Tidy up old terminology in use cases.

This commit is contained in:
dsyer
2008-04-22 15:31:22 +00:00
parent 5c6b37ecc3
commit c1422013be
5 changed files with 38 additions and 19 deletions

View File

@@ -10,8 +10,8 @@ Use Case: Asynchronous Chunk Processing
* Goal
Increased the efficiency of chunk processing by having it execute
asynchronously: each record in a separate thread. Maintain
transactional intergrity of the chunk.
asynchronously: in multiple threads. Maintain transactional
intergrity of the chunk.
* Scope
@@ -119,11 +119,11 @@ Use Case: Asynchronous Chunk Processing
have its own threads - how would each one be able to guide its child
processes to participate in the same transaction?
* Beware a framework that extracts data from an <<<ItemProvider>>>
* Beware a framework that extracts data from an <<<ItemReader>>>
before executing the business logic (e.g. in a
<<<ItemProcessor>>>). It is not enough to allow concurrent
<<<ItemWriter>>>). It is not enough to allow concurrent
processing but simply insist that the individual records are
processed transactionally because the <<<ItemProvider>>> will then
processed transactionally because the <<<ItemReader>>> will then
not be able to participate in the transaction - its next record has
already been passed to the consumer when the transaction starts, so
if there is a rollback then the record is lost.
@@ -131,14 +131,14 @@ Use Case: Asynchronous Chunk Processing
This is the origin of the signature:
+---
public interface ItemProvider {
public interface ItemReader {
Object next();
}
+---
There is no peeking and no iteratror-style <<<hasNext>>>. If there
There is no peeking and no iterator-style <<<hasNext>>>. If there
is a processing problem, transactional clients of the
<<<ItemProvider>>> throw an exception <after> the provider's
<<<ItemReader>>> throw an exception <after> the provider's
<<<next()>>> has been called, but in the same thread (so that
transactional semantics are preserved and the data provider reverts
to its previous state).
@@ -155,5 +155,5 @@ public interface RepeatCallback {
so we can return an object, which is null when the processing has
finished.
In the end we decided against the <<<Object>>> retrun type and went
with a boolean flag to signal (false) for no more processing.
In the end we decided against the <<<Object>>> return type and went
with an exit status to signal for no more processing.

View File

@@ -120,14 +120,14 @@ Use Case: Commit Batch Process Periodically
{{{simple.html}simple}} use case). The iterator could be more than
just a loop that might terminate early: here it could also manage
the file cursor on the input source. In this design there is a
<<<ItemProvider>>> interface that can take care of termination and
<<<ItemReader>>> interface that can take care of termination and
iteration (e.g. iterator-like method signatures).
* Another design idea (more encapsulated and more in keeping with
existing Spring practice) is to make the data source transaction
aware, and for the client use it like a database resource, through a
template. In this case there is a <<<FileInputTemplate>>>. The
<<<ItemProvider>>> needs to be aware of the data source template, so
<<<ItemReader>>> needs to be aware of the data source template, so
that it can terminate when the data is exhausted.
In this version of events there are two kinds of resource in play.
@@ -193,7 +193,7 @@ batchTemplate.iterate(chunkCallback);
+---
The transaction boundary is demarcated at the chunk level
(<<<chunkCallback.doWithRepeat()>>>). Thw termination policy depends
(<<<chunkCallback.doWithRepeat()>>>). The termination policy depends
only on a data source eventually returning null.
* N.B. the chunkSize can be dynamic. E.g., if the chunk is long
@@ -201,7 +201,7 @@ batchTemplate.iterate(chunkCallback);
in case the batch has to be terminated.
* Chunking can also be implemented simply in an
<<<ExecutionHandler>>>. The handler just buffers records up to a
<<<ItemHandler>>>. The handler just buffers records up to a
chunk size, and then executes them all in one step (which might be
transactional). This is easier to implement, and easier to
configure for the clients, but cannot easily be made both concurrent

View File

@@ -76,7 +76,7 @@ Use Case: Copy File to File
* With some external limitations the write-only file source can be
implemented so that within a single JVM it will behave like a
transactional database datasource. We can provide a
<<<FileOutputTemplate>>> that hides the resource acquisition and
<<<FlatFileItemWriter>>> that hides the resource acquisition and
release, and interacts with an existing transaction to provide the
transactional behaviour that is required.

View File

@@ -116,7 +116,26 @@ Use Case: Massively Parallel Batch Processing
** Chunking
The messages from a dispatcher to worker processes consist of a
chunk of items - a set of items to be processed together in a single
transaction (or as the worker sees fit). The dispatcher is usually
single threaded, but this is only a restriction based on the input
data type (if it is a file it is difficult to read in parallel and
maintain restartability). Using a process indicator the dispatcher
could be reading from a database table in a multi-threaded model.
The main restriction is that for restartability the messages between
the dispatcher and workers has to be durable (i.e. JMS or
equivalent). If there is a durable middleware there are no in
principle difficulties with this approach.
The practicalities deserve some discussion. In particular the
dispatcher has to co-ordinate asynchronous replies from its workers,
and also has to avoid overwhelming the workers (so there should be
some throttling). As long as the middleware is durable the
dispatcher can simply wait for replies whenever it thinks there are
workers working. It needs to record this expectation in a durable
form as well, as part of an <<<ExecutionContext>>> for the step.
** Partitioning

View File

@@ -69,12 +69,12 @@ Use Case: Manual Restart After Failure
(persist / rehydrate).
* The initial condition is naturally under control of the
<<<DataProvider>>>. The client need not know about the persistence
<<<ItemReader>>>. The client need not know about the persistence
and rehydration. In fact explicit persistence and rehydration might
be overkill - just relying on the transaction semantics might be
adequate in a lot of cases. The <<<DataProvider>>> would have to be
adequate in a lot of cases. The <<<ItemReader>>> would have to be
aware of the transactions, which we assume are normally demarcated
in the <<<ExecutionHandler>>>. Since the point at which persistence
in the <<<Step>>>. Since the point at which persistence
is needed is tied to transaction commits, there may have to be some
transaction synchronization.