From c1422013becd19c6742df481d6bfae2728509e29 Mon Sep 17 00:00:00 2001 From: dsyer Date: Tue, 22 Apr 2008 15:31:22 +0000 Subject: [PATCH] Tidy up old terminology in use cases. --- src/site/apt/cases/async.apt | 20 ++++++++++---------- src/site/apt/cases/chunks.apt | 8 ++++---- src/site/apt/cases/file-to-file.apt | 2 +- src/site/apt/cases/parallel.apt | 21 ++++++++++++++++++++- src/site/apt/cases/restart.apt | 6 +++--- 5 files changed, 38 insertions(+), 19 deletions(-) diff --git a/src/site/apt/cases/async.apt b/src/site/apt/cases/async.apt index 229036dc6..a954bbcbe 100644 --- a/src/site/apt/cases/async.apt +++ b/src/site/apt/cases/async.apt @@ -10,8 +10,8 @@ Use Case: Asynchronous Chunk Processing * Goal Increased the efficiency of chunk processing by having it execute - asynchronously: each record in a separate thread. Maintain - transactional intergrity of the chunk. + asynchronously: in multiple threads. Maintain transactional + intergrity of the chunk. * Scope @@ -119,11 +119,11 @@ Use Case: Asynchronous Chunk Processing have its own threads - how would each one be able to guide its child processes to participate in the same transaction? - * Beware a framework that extracts data from an <<>> + * Beware a framework that extracts data from an <<>> before executing the business logic (e.g. in a - <<>>). It is not enough to allow concurrent + <<>>). It is not enough to allow concurrent processing but simply insist that the individual records are - processed transactionally because the <<>> will then + processed transactionally because the <<>> will then not be able to participate in the transaction - its next record has already been passed to the consumer when the transaction starts, so if there is a rollback then the record is lost. @@ -131,14 +131,14 @@ Use Case: Asynchronous Chunk Processing This is the origin of the signature: +--- -public interface ItemProvider { +public interface ItemReader { Object next(); } +--- - There is no peeking and no iteratror-style <<>>. If there + There is no peeking and no iterator-style <<>>. If there is a processing problem, transactional clients of the - <<>> throw an exception the provider's + <<>> throw an exception the provider's <<>> has been called, but in the same thread (so that transactional semantics are preserved and the data provider reverts to its previous state). @@ -155,5 +155,5 @@ public interface RepeatCallback { so we can return an object, which is null when the processing has finished. - In the end we decided against the <<>> retrun type and went - with a boolean flag to signal (false) for no more processing. + In the end we decided against the <<>> return type and went + with an exit status to signal for no more processing. diff --git a/src/site/apt/cases/chunks.apt b/src/site/apt/cases/chunks.apt index 204749b7d..bf13d81a1 100644 --- a/src/site/apt/cases/chunks.apt +++ b/src/site/apt/cases/chunks.apt @@ -120,14 +120,14 @@ Use Case: Commit Batch Process Periodically {{{simple.html}simple}} use case). The iterator could be more than just a loop that might terminate early: here it could also manage the file cursor on the input source. In this design there is a - <<>> interface that can take care of termination and + <<>> interface that can take care of termination and iteration (e.g. iterator-like method signatures). * Another design idea (more encapsulated and more in keeping with existing Spring practice) is to make the data source transaction aware, and for the client use it like a database resource, through a template. In this case there is a <<>>. The - <<>> needs to be aware of the data source template, so + <<>> needs to be aware of the data source template, so that it can terminate when the data is exhausted. In this version of events there are two kinds of resource in play. @@ -193,7 +193,7 @@ batchTemplate.iterate(chunkCallback); +--- The transaction boundary is demarcated at the chunk level - (<<>>). Thw termination policy depends + (<<>>). The termination policy depends only on a data source eventually returning null. * N.B. the chunkSize can be dynamic. E.g., if the chunk is long @@ -201,7 +201,7 @@ batchTemplate.iterate(chunkCallback); in case the batch has to be terminated. * Chunking can also be implemented simply in an - <<>>. The handler just buffers records up to a + <<>>. The handler just buffers records up to a chunk size, and then executes them all in one step (which might be transactional). This is easier to implement, and easier to configure for the clients, but cannot easily be made both concurrent diff --git a/src/site/apt/cases/file-to-file.apt b/src/site/apt/cases/file-to-file.apt index 56ea4af8a..9ba24799f 100644 --- a/src/site/apt/cases/file-to-file.apt +++ b/src/site/apt/cases/file-to-file.apt @@ -76,7 +76,7 @@ Use Case: Copy File to File * With some external limitations the write-only file source can be implemented so that within a single JVM it will behave like a transactional database datasource. We can provide a - <<>> that hides the resource acquisition and + <<>> that hides the resource acquisition and release, and interacts with an existing transaction to provide the transactional behaviour that is required. diff --git a/src/site/apt/cases/parallel.apt b/src/site/apt/cases/parallel.apt index dfcc56e6c..0da0d2122 100644 --- a/src/site/apt/cases/parallel.apt +++ b/src/site/apt/cases/parallel.apt @@ -116,7 +116,26 @@ Use Case: Massively Parallel Batch Processing ** Chunking - + The messages from a dispatcher to worker processes consist of a + chunk of items - a set of items to be processed together in a single + transaction (or as the worker sees fit). The dispatcher is usually + single threaded, but this is only a restriction based on the input + data type (if it is a file it is difficult to read in parallel and + maintain restartability). Using a process indicator the dispatcher + could be reading from a database table in a multi-threaded model. + + The main restriction is that for restartability the messages between + the dispatcher and workers has to be durable (i.e. JMS or + equivalent). If there is a durable middleware there are no in + principle difficulties with this approach. + + The practicalities deserve some discussion. In particular the + dispatcher has to co-ordinate asynchronous replies from its workers, + and also has to avoid overwhelming the workers (so there should be + some throttling). As long as the middleware is durable the + dispatcher can simply wait for replies whenever it thinks there are + workers working. It needs to record this expectation in a durable + form as well, as part of an <<>> for the step. ** Partitioning diff --git a/src/site/apt/cases/restart.apt b/src/site/apt/cases/restart.apt index 8a71279d8..4bbd564a6 100644 --- a/src/site/apt/cases/restart.apt +++ b/src/site/apt/cases/restart.apt @@ -69,12 +69,12 @@ Use Case: Manual Restart After Failure (persist / rehydrate). * The initial condition is naturally under control of the - <<>>. The client need not know about the persistence + <<>>. The client need not know about the persistence and rehydration. In fact explicit persistence and rehydration might be overkill - just relying on the transaction semantics might be - adequate in a lot of cases. The <<>> would have to be + adequate in a lot of cases. The <<>> would have to be aware of the transactions, which we assume are normally demarcated - in the <<>>. Since the point at which persistence + in the <<>>. Since the point at which persistence is needed is tied to transaction commits, there may have to be some transaction synchronization.