add use cases to site

spring-batch-integration/src/site/apt/index.apt

@@ -7,4 +7,168 @@
 
 Overview of the Spring Integration Batch Module
 
-  Many of the 2.0 Features we identified for Spring Batch look like they might be efficiently and concisely implemented in Spring Integration.  These are features that can extend Spring Batch, or use Spring batch features in the context of Spring Integration. 
+  Many of the 2.0 Features we identified for Spring Batch look like they might be efficiently and concisely implemented in Spring Integration.  Here is a list of use cases.  These are features that can extend Spring Batch, or use Spring batch features in the context of Spring Integration.  Work in progress by Dave Syer and Jonas Parttner.  Many issues to do with transactionality and synchronous execution have been raised and fixed in Spring Integration as a result of these use cases being prototyped.
+
+*---+---+---+---+---+
+|<<ID>>|<<Description>>|<<Status>>|<<Sub-package>>|<<Comments>>|
+*----
+|1|{{{trigger}Message triggers job}}|Prototype|launch|Relatively complete - maybe look at making handlers strongly typed.  Also lots of opportunities with monitoring progress.|
+*----
+|2|{{{chunking}Chunking and multi-VM job execution}}|Prototype|chunk|Sunny day case works fine (but not packaged yet as a re-usable handler).  Failures might need some analysis.  Use of statefulStepExecutionListener might be improved on?|
+*----
+|3|{{{aggregator}Asynchronous Aggregator}}|Unstarted| |
+*----
+|4|{{{jobflow}Stateful and non-linear jobs}} -> job = flow|Prototype|job|Failure cases need to be analysed - in particular, what happens on restart (after lights out) to messages from the middle of a job.|
+*----
+|5|{{{stepflow}Flexible item processing model}} (as message flow) -> step = flow|Prototype|item|Complete (v. simple).|
+*----
+|6|{{{repeat}Automatic repeat / retry}}|Prototype|retry (unit test)|Works with patched PollingSourceAdapter.|
+*----
+|7|{{{files}Restartable file processing}}|Prototype|file|Seems to hang together.  Not tested thoroughly.|
+*----
+
+  Numbers 2, 4, 5 have also been identified as high level Spring Batch 2.0 Features or themes.  If we implement 1, then we also don't need to do any more scheduling and triggering in Spring Batch.
+
+  Number 6 from the list (repeat/retry) is more of a Spring Integration pattern than a Spring Batch one.  We will try and implement it in Spring Batch first, and see about pushing it out into Spring Integration later (with probably a split of repeat/retry out of Batch at that time).
+
+* Message Triggers Job
+
+  {trigger}Description:
+
+    [[1]] User sends message to channel (maybe through a scheduler)
+
+    [[1]] System interprets message payload as parameters for JobLauncher
+
+    [[1]] System launches job execution
+ 
+    [[1]] If message had a replyTo, System acknowledges with JobExecution
+
+    [[1]] User accepts response and uses it to monitor progress
+
+  Variation:
+
+    [[1]] System waits for job to finish and replies when it is over
+
+    [[1]] User polls for replies and gets notification about end of execution
+
+  Variation:
+
+    [[1]] User wants to block on send and only receive response when job is done
+
+* Chunking and Multi-VM
+
+  {chunking}Description:
+
+    [[1]] Step flushes chunk as message to outgoing channel (repeat up to throttle limit)
+
+    [[1]] Worker thread picks up chunk and processes it
+
+    [[1]] Worker thread replies to response channel
+ 
+    [[1]] Step picks up reply and aggregates the counts
+
+    [[1]] Step blocks until all the requests are satisfied
+
+  TODO: failure modes
+
+* Asynchronous Aggregator
+
+  {aggregator}Job is executed over long period.  Many jobs can be executing concurrently.  
+
+  Description:
+
+    [[1]] Input stage for each job: System reads all items and marks with the job instance id in a durable repository (staging table)
+
+    [[1]] System sends each item (or chunks of items that can be processed together as appropriate) to a channel
+
+    [[1]] Items flow through message pipeline, occasionally pausing until certain conditions are met, possibly for days at a time
+
+    [[1]] Aggregator sits and waits for all items in a job to be finished and then wraps up
+
+* Stateful and non-linear jobs
+
+  {jobflow}Dependencies beyween steps and conditional flow between steps.  Each handler node in a message flow is a step execution, with all the robustness guarantees from the Spring Batch meta data.
+
+  Description:
+
+    [[1]] User launches job
+ 
+    [[1]] System sends message to channel containing job execution
+
+    [[1]] Handler accepts message and executes a step
+
+    [[1]] Handler translates result of step execution into the same form that it accepted the original request
+
+    [[1]] System routes message to next handler, possibly dynamically based on data in the message
+
+    [[1]] Next handler does the same... until one of the routing decisions leads to a reply channel
+
+    [[1]] System receives reply and transfers information to job execution (e.g. status) as necessary
+
+  Variation: failure in one of the handlers
+
+  Variation: restart after failure
+
+* Flexible item processing model
+
+  {stepflow}Description:
+
+    [[1]] Step hands item to ItemWriter
+
+    [[1]] Item is converted to message and sent to synchronous flow
+
+    [[1]] Handler accepts message and does something with item
+
+    [[1]] System routes result to next handler, possibly dynamically
+
+  Variation: failure
+
+    [[1]] Handler throws exception
+
+    [[1]] System propagates exception up to ItemWriter (forces rollback under normal circs - hence synchronous flow)
+
+* Automatic repeat / retry
+
+  {repeat}Description (repeat):
+
+    [[1]] User sends message to channel
+
+    [[1]] System start a transaction and reseives message, then processes it
+
+    [[1]] User sends another message
+
+    [[1]] System receives and processes it in the same transaction
+
+    [[1]] ... repeat ...
+
+    [[1]] System determines that batch is complete and commits transaction
+
+* Restartable file processing
+
+  {files}Large file needs to be processed, so message payload of file contents is not practical.  One line or XML event per message with failover and restartability from Spring Batch.
+
+  Description:
+
+    [[1]] User triggers file processing (sends message, copies file to directory, etc.)
+
+    [[1]] System starts new job
+
+    [[1]] System processes file line by line (or even by event), wrapping each one as a message and sending it to a synchronous flow
+
+    [[1]] System commits periodically (as determined by Spring Batch step configuration)
+
+  Variation: failure and restart
+
+    [[1]] Item processing fails
+
+    [[1]] System aborts job and sends message to failure channel (or failure message to normal reply channel)
+
+    [[1]] Operator fixes problem and triggers restart (another message channel?)
+
+    [[1]] System restarts job for same file at point where it left off
+
+    [[1]] System completes processing
+
+    [[1]] System sends sucess message to reply channel
+
+  Variation: send to asynchronous flow.  Same as main use case but item message is sent to asynchronous flow.  Not as robust because if the lights go out then meesages will be lost, but at least a large file can be split into smaller chunks.