diff --git a/src/site/docbook/reference/scalability.xml b/src/site/docbook/reference/scalability.xml index 96f416bae..cc161bd0a 100644 --- a/src/site/docbook/reference/scalability.xml +++ b/src/site/docbook/reference/scalability.xml @@ -46,13 +46,13 @@ TaskExecutor to your Step configuration, e.g. as an attribute of the tasklet: - - <step id="loading"> + <tasklet reader="stagingReader" processor="stagingProcessor" writer="tradeWriter" commit-interval="1" - task-executor="taskExecutor"/> -]]> + task-executor="taskExecutor"/> +</step> In this example the taskExecutor is a reference to another bean definition, implementing the TaskExecutor @@ -81,23 +81,34 @@
Parallel Steps - As long as the application logic that needs to be parallelised can + As long as the application logic that needs to be parallelized can be split into distinct responsibilities, and assigned to individual steps - then it can be parallelised in a single process. Parallel Step execution + then it can be parallelized in a single process. Parallel Step execution is easy to configure and use, for example, to execute steps (step1,step2) in parallel with step3, you could configure a flow like this: - - - - - - - - - - ]]> + <job id="job1"> + <split id="split1" task-executor="taskExecutor" next="step4"> + <flow> + <step id="step1" ref="s1" next="step2"/> + <step id="step2" ref="s2"/> + </flow> + <flow> + <step id="step3" ref="s3"/> + </flow> + </split> + <step id="step4" ref="s4"/> +</job> + +<beans:bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor"/> + + The configurable "task-executor" attribute is used to specify which + TaskExecutor implementation should be used to execute the individual + flows. The default is SyncTaskExecutor, but an + asynchronous TaskExecutor is required to run the steps in parallel. Note + that the job will ensure that every flow in the split completes before + aggregating the exit statuses and transitioning. See the section on for more detail. @@ -195,11 +206,11 @@ the PartitionStep is shown driving the execution. The PartitionStep configuration looks like this: - - - - -]]> + <bean name="step1:master" class="org.sfw...PartitionStep"> + <property name="partitionHandler" ref="partitionHandler"/> + <property name="stepExecutionSplitter" ref="stepExecutionSplitter"/> + <property name="jobRepository" ref="jobRepository" /> +</bean> There is a simple example which can be copied and extended in the unit test suite for Spring Batch Core (see @@ -222,7 +233,7 @@ re-executed. The PartitionHandler interface can have - specialised implementations for a variety of fabric types: e.g. simple + specialized implementations for a variety of fabric types: e.g. simple RMI remoting, EJB remoting, custom web service, JMS, Java Spaces, shared memory grids (like Terracotta or Coherence), grid execution fabrics (like GridGain). Spring Batch does not contain implementations for any @@ -236,11 +247,11 @@ TaskExecutorPartitionHandler, and it can be configured like this: - - - - -]]> + <bean class="org.sfw..TaskExecutorPartitionHandler"> + <property name="taskExecutor" ref="taskExecutor"/> + <property name="step" ref="step1" /> + <property name="gridSize" value="10" /> +</bean> The gridSize determines the number of separate step executions to create, so it can be matched to the size of the @@ -262,11 +273,11 @@ of an ExecutionContext for each one. The principal method for this in the interface is - public interface StepExecutionSplitter { ... - Set split(StepExecution stepExecution, int gridSize) + Set<StepExecution> split(StepExecution stepExecution, int gridSize) throws JobExecutionException; -}]]> +} So an execution instance for the Master step is passed in, along with a hint about the grid size, and the splitter has to create a set of @@ -283,9 +294,9 @@ parameters for new step executions only (no need to worry about restarts). It has a single method: - partition(int gridSize); -}]]> + public interface Partitioner { + Map<String, ExecutionContext> partition(int gridSize); +} The return value from this method associates a unique name for each step execution (the String), with input @@ -363,10 +374,10 @@ Then the file name can be bound to a step using late binding to the execution context: - - #{stepExecutionContext[fileName]}/* - ]]> + <bean id="itemReader" scope="step" + class="org.sfw...MultiResourceItemReader"> + <property name="resource" value="#{stepExecutionContext[fileName]}/*"/> + </bean>