diff --git a/src/site/docbook/reference/scalability.xml b/src/site/docbook/reference/scalability.xml index 046049319..400b01ecf 100644 --- a/src/site/docbook/reference/scalability.xml +++ b/src/site/docbook/reference/scalability.xml @@ -202,16 +202,21 @@ the PartitionStep is shown driving the execution. The PartitionStep configuration looks like this: - <bean name="step1:master" class="org.sfw...PartitionStep"> - <property name="partitionHandler" ref="partitionHandler"/> - <property name="stepExecutionSplitter" ref="stepExecutionSplitter"/> - <property name="jobRepository" ref="jobRepository" /> -</bean> + <step id="step1.master"> + <partition step="step1" partitioner="partitioner"> + <handler grid-size="10" task-executor="taskExecutor"/> + </partition> +</step> There is a simple example which can be copied and extended in the - unit test suite for Spring Batch Core (see - org.springframework.batch.core.partition - package). + unit test suite for Spring Batch Samples (see + *PartitionJob.xml configuration). + + Spring Batch creates step executions for the partitions called + "step1:partition0", etc., so many people prefer to call the master step + "step1:master" for consistency. With Spring 3.0 you can do this using an + alias for the step (specifying the name attribute + instead of the id).
PartitionHandler @@ -240,10 +245,15 @@ separate threads of execution, using the TaskExecutor strategy from Spring. The implementation is called - TaskExecutorPartitionHandler, and it can be - configured like this: + TaskExecutorPartitionHandler, and it is the + default for a step configured with the XML namespace as above. It can + also be configured explicitly like this: - <bean class="org.spr...TaskExecutorPartitionHandler"> + <step id="step1.master"> + <partition step="step1" handler="handler"/> +</step> + +<bean class="org.spr...TaskExecutorPartitionHandler"> <property name="taskExecutor" ref="taskExecutor"/> <property name="step" ref="step1" /> <property name="gridSize" value="10" /> @@ -257,38 +267,17 @@ The TaskExecutorPartitionHandler is quite useful for IO intensive Steps, like copying large numbers of files or - replicating filesystems into content management systems. + replicating filesystems into content management systems. It can also be + used for remote execution by providing a Step implementation that is a + proxy for a remote invocation (e.g. using Spring Remoting).
- StepExecutionSplitter + Partitioner - The StepExecutionSplitter is responsible - for splitting up a StepExecution into blocks of - work, and providing input parameters for the remote Slaves in the form - of an ExecutionContext for each one. The - principal method for this in the interface is - - public interface StepExecutionSplitter { - ... - Set<StepExecution> split(StepExecution stepExecution, int gridSize) - throws JobExecutionException; -} - - So an execution instance for the Master step is passed in, along - with a hint about the grid size, and the splitter has to create a set of - partitioned StepExecution instances, each with a - different ExecutionContext. - - A convenient generic implementation of StepExecutionSplitter is - provided by Spring Batch, which handles concerns like interpreting the - grid size and handling restart. It is recommended that you use this - implementation (the SimpleStepExecutionSplitter) - and inject specific knowledge of the input data through its - Partitioner property. The Partitioner has a - simpler responsibility: to generate execution contexts as input - parameters for new step executions only (no need to worry about - restarts). It has a single method: + The Partitioner has a simpler responsibility: to generate + execution contexts as input parameters for new step executions only (no + need to worry about restarts). It has a single method: public interface Partitioner { Map<String, ExecutionContext> partition(int gridSize); diff --git a/src/site/docbook/reference/step.xml b/src/site/docbook/reference/step.xml index 767a94f3c..e0ba690b9 100644 --- a/src/site/docbook/reference/step.xml +++ b/src/site/docbook/reference/step.xml @@ -1647,7 +1647,7 @@ itemWriter.write(items); <bean id="flatFileItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobParameters[input.file.name]}" /> + <property name="resource" value="#{jobParameters['input.file.name']}" /> </bean> Both the JobExecution and @@ -1657,12 +1657,12 @@ itemWriter.write(items); <bean id="flatFileItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobExecutionContext[input.file.name]}" /> + <property name="resource" value="#{jobExecutionContext['input.file.name']}" /> </bean> <bean id="flatFileItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{stepExecutionContext[input.file.name]}" /> + <property name="resource" value="#{stepExecutionContext['input.file.name']}" /> </bean> @@ -1671,6 +1671,17 @@ itemWriter.write(items); information. + + If you are using Spring 3.0 (or above) the expressions in + step-scoped beans are in the Spring Expression Language, a powerful + general purpose language with many interesting features. To provide + backward compatibility, if Spring Batch detects the presence of older + versions of Spring it uses a native expression language that is less + powerful, and has slightly different parsing rules. The main difference + is that the map keys in the example above do not need to be quoted with + Spring 2.5, but the quotes are mandatory in Spring 3.0. + +
Step Scope diff --git a/src/site/docbook/reference/testing.xml b/src/site/docbook/reference/testing.xml index 4a586efda..dfe6a8266 100644 --- a/src/site/docbook/reference/testing.xml +++ b/src/site/docbook/reference/testing.xml @@ -62,7 +62,10 @@ public class SkipSampleFunctionalTests extends AbstractJobTests { ... }@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "/simple-job-launcher-context.xml", "/jobs/skipSampleJob.xml" }) -public class SkipSampleFunctionalTests extends AbstractJobTests { +public class SkipSampleFunctionalTests { + + @Autowired + private JobLauncherTestUtils jobLauncherTestUtils; private SimpleJdbcTemplate simpleJdbcTemplate; @@ -80,7 +83,8 @@ public class SkipSampleFunctionalTests extends AbstractJobTests { i, "customer" + i); } - JobExecution jobExecution = this.launchJob(); + JobExecution jobExecution = jobLauncherTestUtils.launchJob().getStatus(); + Assert.assertEquals("COMPLETED", jobExecution.getExitStatus()); } @@ -99,7 +103,81 @@ public class SkipSampleFunctionalTests extends AbstractJobTests { targeted tests by allowing the test to set up data for just that step and to validate its results directly. - JobExecution jobExecution = this.launchStep("loadFileStep"); + JobExecution jobExecution = jobLauncherTestUtils.launchStep("loadFileStep"); +
+ +
+ Testing Step-Scoped Components + + Often the components that are configured for your steps at runtime + use step scope and late binding to inject context from the step or job + execution. These are tricky to test as standalone components unless you + have a way to set the context as if they were in a step execution. That is + the goal of two components in Spring Batch: the + StepScopeTestExecutionListener and the + StepScopeTestUtils. + + The listener is declared at the class level, and its job is to + create a step execution context for each test method. For example: + + @ContextConfiguration +@TestExecutionListeners( { DependencyInjectionTestExecutionListener.class, + StepScopeTestExecutionListener.class }) +@RunWith(SpringJUnit4ClassRunner.class) +public class StepScopeTestExecutionListenerIntegrationTests { + + // This component is defined step-scoped, so it cannot be injected unless + // a step is active... + @Autowired + private ItemReader<String> reader; + + public StepExecution getStepExection() { + StepExecution execution = MetaDataInstanceFactory.createStepExecution(); + execution.getExecutionContext().putString("input.data", "foo,bar,spam"); + return execution; + } + + @Test + public void testReader() { + // The reader is initialized and bound to the input data + assertNotNull(reader.read()); + } + +} + + There are two TestExecutionListeners, one + from the regular Spring Test framework and handles dependency injection + from the configured application context, injecting the reader, and the + other is the Spring Batch + StepScopeTestExecutionListener. It works by looking + for a factory method in the test case for a + StepExecution, and using that as the context for + the test method, as if that execution was active in a Step at runtime. The + factory method is detected by its signature (it just has to return a + StepExecution). If a factory method is not provided + then a default StepExecution is created. + + The listener approach is convenient if you want the duration of the + step scope to be the execution of the test method. For a more flexible, + but more invasive approach you can use the + StepScopeTestUtils. For example, to count the + number of items available in the reader above: + + int count = StepScopeTestUtils.doInStepScope(stepExecution, + new Callable<Integer>() { + + public Integer call() throws Exception { + + int count = 0; + + while (reader.read() != null) { + count++; + } + + return count; + + } +});