diff --git a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java index aaf15b2dc..177d58d6e 100644 --- a/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java +++ b/spring-batch-core/src/main/java/org/springframework/batch/core/configuration/annotation/EnableBatchProcessing.java @@ -37,7 +37,7 @@ import org.springframework.transaction.PlatformTransactionManager; *
* @Configuration
* @EnableBatchProcessing
- * @Import(DataSourceCnfiguration.class)
+ * @Import(DataSourceConfiguration.class)
* public class AppConfig {
*
* @Autowired
@@ -167,4 +167,4 @@ public @interface EnableBatchProcessing {
*/
boolean modular() default false;
-}
\ No newline at end of file
+}
diff --git a/src/site/docbook/reference/job.xml b/src/site/docbook/reference/job.xml
index f6b8dae94..997f97eaf 100644
--- a/src/site/docbook/reference/job.xml
+++ b/src/site/docbook/reference/job.xml
@@ -226,12 +226,101 @@ catch (JobRestartException e) {
+
+ Java Config
+
+ Spring 3 brought the ability to configure applications via java instead
+ of XML. As of Spring Batch 2.2.0, batch jobs can be configured using the same
+ java config. There are two components for the java based configuration:
+ the @EnableBatchConfiguration annotation and two builders.
+
+ The @EnableBatchProcessing works similarly to the other
+ @Enable* annotations in the Spring family. In this case,
+ @EnableBatchProcessing provides a base configuration for
+ building batch jobs. Within this base configuration, an instance of
+ StepScope is createded in addition to a number of beans made
+ available to be autowired:
+
+
+
+
+ JobRepository - bean name "jobRepository"
+
+
+ JobLauncher - bean name "jobLauncher"
+
+
+ JobRegistry - bean name "jobRegistry"
+
+
+ PlatformTransactionManager - bean name "transactionManager"
+
+
+ JobBuilderFactory - bean name "jobBuilders"
+
+
+ StepBuilderFactory - bean name "stepBuilders"
+
+
+
+ The core interface for this configuration is the BatchConfigurer .
+ The default implementation provides the beans mentioned above and requires a
+ DataSource as a bean within the context to be provided. This data
+ source will be used by the JobRepository .
+
+
+
+ Only one configuration class needs to have the
+ @EnableBatchProcessing annotation. Once you have a class
+ annotated with it, you will have all of the above available.
+
+
+ With the base configuration in place, a user can use the provided builder factories
+ to configure a job. Below is an example of a two step job configured via the
+ JobBuilderFactory and the StepBuilderFactory .
+
+ @Configuration
+@EnableBatchProcessing
+@Import(DataSourceCnfiguration.class)
+public class AppConfig {
+
+ @Autowired
+ private JobBuilderFactory jobs;
+
+ @Autowired
+ private StepBuilderFactory steps;
+
+ @Bean
+ public Job job() {
+ return jobs.get("myJob").start(step1()).next(step2()).build();
+ }
+
+ @Bean
+ protected Step step1(ItemReader<Person> reader, ItemProcessor<Person, Person> processor, ItemWriter<Person> writer) {
+ return steps.get("step1")
+ .<Person, Person> chunk(10)
+ .reader(reader)
+ .processor(processor)
+ .writer(writer)
+ .build();
+ }
+
+ @Bean
+ protected Step step2(Tasklet tasklet) {
+ return steps.get("step2")
+ .tasklet(tasklet)
+ .build();
+ }
+}
+
+
+
-
+
Configuring a JobRepository
-
+
As described in earlier, the
JobRepository
@@ -246,7 +335,7 @@ catch (JobRestartException e) {
collaborators. However, there are still a few configuration options
available:
-
+
]]>
-
+
None of the configuration options listed above are required except
the id. If they are not set, the defaults shown above will be used. They
@@ -265,7 +354,7 @@ catch (JobRestartException e) {
length of the long VARCHAR columns in the sample schema scripts
- used to store things like exit code descriptions. If you don't modify the schema and you don't use multi-byte characters you shouldn't need to change it.
+ used to store things like exit code descriptions. If you don't modify the schema and you don't use multi-byte characters you shouldn't need to change it.
Transaction Configuration for the JobRepository
@@ -297,7 +386,7 @@ catch (JobRestartException e) {
-
@@ -315,7 +404,7 @@ catch (JobRestartException e) {
classpath.
-
+
Changing the Table Prefix
@@ -342,7 +431,7 @@ catch (JobRestartException e) {
-
+
In-Memory Repository
@@ -354,7 +443,7 @@ catch (JobRestartException e) {
this reason, Spring batch provides an in-memory Map version of the job
repository:
-
]]>
@@ -373,7 +462,7 @@ catch (JobRestartException e) {
ResourcelessTransactionManager useful.
-
+
Non-standard Database Types in a Repository
@@ -404,7 +493,7 @@ catch (JobRestartException e) {
on and wire one up manually in the normal Spring way.
-
+
@@ -777,7 +866,7 @@ public class JobLauncherController {
JobRepository , it can be easily configured via a
factory bean:
- ]]>
Earlier in this
@@ -787,7 +876,7 @@ public class JobLauncherController {
JobExplorer is working with the same tables, it
too needs the ability to set a prefix:
- p:tablePrefix="BATCH_" ]]>
@@ -899,30 +988,30 @@ public class JobLauncherController {
List getExecutions(long instanceId) throws NoSuchJobInstanceException;
- List getJobInstances(String jobName, int start, int count)
+ List getJobInstances(String jobName, int start, int count)
throws NoSuchJobException;
Set getRunningExecutions(String jobName) throws NoSuchJobException;
String getParameters(long executionId) throws NoSuchJobExecutionException;
- Long start(String jobName, String parameters)
+ Long start(String jobName, String parameters)
throws NoSuchJobException, JobInstanceAlreadyExistsException;
- Long restart(long executionId)
+ Long restart(long executionId)
throws JobInstanceAlreadyCompleteException, NoSuchJobExecutionException,
NoSuchJobException, JobRestartException;
- Long startNextInstance(String jobName)
- throws NoSuchJobException, JobParametersNotFoundException, JobRestartException,
+ Long startNextInstance(String jobName)
+ throws NoSuchJobException, JobParametersNotFoundException, JobRestartException,
JobExecutionAlreadyRunningException, JobInstanceAlreadyCompleteException;
- boolean stop(long executionId)
+ boolean stop(long executionId)
throws NoSuchJobExecutionException, JobExecutionNotRunningException;
String getSummary(long executionId) throws NoSuchJobExecutionException;
- Map getStepExecutionSummaries(long executionId)
+ Map getStepExecutionSummaries(long executionId)
throws NoSuchJobExecutionException;
Set getJobNames();
@@ -996,8 +1085,8 @@ public class JobLauncherController {
as shown below:
RetryTemplate
+
+ The retry functionality was pulled out of Spring Batch as of 2.2.0.
+ It is now part of a new library, Spring Retry.
+
+
To make processing more robust and less prone to failure, sometimes
it helps to automatically retry a failed operation in case it might
succeed on a subsequent attempt. Errors that are susceptible to this kind
@@ -22,13 +27,13 @@
<T> T execute(RetryCallback<T> retryCallback) throws Exception;
- <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback)
+ <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback)
throws Exception;
- <T> T execute(RetryCallback<T> retryCallback, RetryState retryState)
+ <T> T execute(RetryCallback<T> retryCallback, RetryState retryState)
throws Exception, ExhaustedRetryException;
- <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback,
+ <T> T execute(RetryCallback<T> retryCallback, RecoveryCallback<T> recoveryCallback,
RetryState retryState) throws Exception;
} The basic callback is a simple interface that allows you to
@@ -276,7 +281,7 @@ template.execute(new RetryCallback<Foo>() {
BackOffContext start(RetryContext context);
- void backOff(BackOffContext backOffContext)
+ void backOff(BackOffContext backOffContext)
throws BackOffInterruptedException;
}A BackoffPolicy is free to implement
diff --git a/src/site/docbook/reference/whatsnew.xml b/src/site/docbook/reference/whatsnew.xml
index ef639bf6a..0c66f1d51 100644
--- a/src/site/docbook/reference/whatsnew.xml
+++ b/src/site/docbook/reference/whatsnew.xml
@@ -2,427 +2,129 @@
- What's New in Spring Batch 2.0
+ What's New in Spring Batch 2.2
- The Spring Batch 2.0 release has six major themes:
+ The Spring Batch 2.2 release has six major themes:
- Java 5
+ Spring Data Integration
- Non Sequential Step Execution
+ Java Configuration
- Chunk oriented processing
+ Spring Retry
- Meta Data enhancements
-
-
-
- Scalability
-
-
-
- Configuration
+ Job Parameters
-
- Java 5
+
+ Spring Data Integration
- The 1.x releases of Spring Batch were all based on Java 1.4. This
- prevented the framework from using many enhancements provided in Java 5
- such as generics, parameterized types, etc. The entire framework has been
- updated to utilize these features. As a result, Java
- 1.4 is no longer supported. Most of the interfaces developers
- work with have been updated to support generic types. As an example, the
- ItemReader interface from 1.1 is below:
+ Since the 2.0 release of Spring Batch, the Spring Data project has brought
+ support for the NoSQL movement to Spring. The 2.2 release of Spring Batch has added
+ support for MongoDB, Neo4j and Gemfire natively through the Spring Data abstractions.
- public interface ItemReader {
-
- Object read() throws Exception;
-
- void mark() throws MarkFailedException;
-
- void reset() throws ResetFailedException;
-}
-
- As you can see, the read method returns an
- Object . The 2.0 version is below:
-
- public interface ItemReader<T> {
-
- T read() throws Exception, UnexpectedInputException, ParseException;
-
-}
-
- As you can see, ItemReader now supports the
- generic type, T , which is returned from read. You
- may also notice that mark and
- reset have been removed. This is due to step
- processing strategy changes, which are discussed below. Many other
- interfaces have been similarly updated.
+ This release has also added support for writing to any custom Spring Data Repository a
+ user may write. The RepositoryItemReader and
+ RepositoryItemWriter each wrap a repository implementation (
+ PagingAndSortingRepository and CrudRepository
+ respectively) to retrieve data from and persist data to.
-
- Chunk Oriented Processing
+
+ Java Configuration
- Previously, the default processing strategy provided by Spring Batch
- was item-oriented processing:
+ Until 2.2.0 the only option for configuring a job was via XML (either through the batch DSL or
+ by hand). However, in 2.2.0, Java based configuration has been added as a way to define Spring Batch
+ Jobs. To support this new configuration option, an annotation and builder classes have been added. What
+ was previously defined as this:
-
-
-
-
+ <batch>
+ <job-repository/>
-
-
-
-
+ <job id="myJob">
+ <step id="step1".../>
+ <step id="step2".../>
+ </job>
- In item-oriented processing, the ItemReader
- returns one Object (the 'item') which is then
- handed to the ItemWriter , periodically committing
- when the number of items hits the commit interval. For example, if the
- commit interval is 5, ItemReader and
- ItemWriter will each be called 5 times. This is
- illustrated in a simplified code example below:
+ <beans:bean id="transactionManager".../>
- for(int i = 0; i < commitInterval; i++){
- Object item = itemReader.read();
- itemWriter.write(item);
+ <beans:bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
+ <beans:property name="jobRepository" ref="jobRepository"/>
+ </beans:bean>
+</batch>
+
+
+ Can now be configured using the @EnableBatchProcessing annotation and the
+ provided JobBuilderFactory and StepBuilderFactory as show below:
+
+ @Configuration
+ @EnableBatchProcessing
+ @Import(DataSourceCnfiguration.class)
+ public class AppConfig {
+
+ @Autowired
+ private JobBuilderFactory jobs;
+
+ @Bean
+ public Job job() {
+ return jobs.get("myJob").start(step1()).next(step2()).build();
+ }
+
+ @Bean
+ protected Step step1() {
+ ...
+ }
+
+ @Bean
+ protected Step step2() {
+ ...
+ }
}
- Both the ItemReader and
- ItemWriter interfaces were completely geared toward
- this approach:
-
- public interface ItemReader {
-
- Object read() throws Exception;
-
- void mark() throws MarkFailedException;
-
- void reset() throws ResetFailedException;
-}
-
- public interface ItemWriter {
-
- void write(Object item) throws Exception;
-
- void flush() throws FlushFailedException;
-
- void clear() throws ClearFailedException;
-}
-
- Because the 'scope' of the processing was one item, supporting
- rollback scenarios required additional methods, which is what
- mark , reset ,
- flush , and clear
- provided. If, after successfully reading and writing 2 items, the third
- has an error while writing, the transaction would need to be rolled back.
- In this case, the clear method on the writer
- would be called, indicating that it should clear
- its buffer, and reset would be called on the
- ItemReader , indicating that it should return back
- to the last position it was at when mark was
- called. (Both mark and
- flush are called on commit)
-
- In 2.0, this strategy has been changed to a chunk-oriented
- approach:
-
-
-
-
-
-
-
-
-
-
-
- Using the same example from above, if the commit interval is five,
- read will be called 5 times, and write once. The items read will be
- aggregated into a list, that will ultimately be written out, as the
- simplified example below illustrates:
-
- List items = new Arraylist();
-for(int i = 0; i < commitInterval; i++){
- items.add(itemReader.read());
-}
-itemWriter.write(items);
-
- This approach not only allows for much simpler processing and
- scalability approaches, it also makes the
- ItemReader and ItemWriter
- interfaces much cleaner:
-
- public interface ItemReader<T> {
-
- T read() throws Exception, UnexpectedInputException, ParseException;
-
-}
-
- public interface ItemWriter<T> {
-
- void write(List<? extends T> items) throws Exception;
-
-}
-
- As you can see, the interfaces no longer contain the
- mark , reset ,
- flush , and clear
- methods. This makes the creation of readers and writers much more
- straightforward for developers. In the case of
- ItemReader , the interface is now forward-only. The
- framework will buffer read items for developers in the case of rollback
- (though there are exceptions if the underlying resource is transactional
- see: ).
- ItemWriter is also simplified, since it gets the
- entire 'chunk' of items at once, rather than one at a time, it can decide
- to flush any resources (such as a file or hibernate session) before
- returning control to the Step . More detailed
- information on chunk-oriented processing can be found in . Reader and writer implementation
- information can be found in .
-
-
- ItemProcessor
-
- Previously, Step s had only two
- dependencies, ItemReader and
- ItemWriter :
-
-
-
-
-
-
-
-
-
-
-
- The basic configuration above is fairly robust. However, there are
- many cases where the item needs to be transformed before writing. In 1.x
- this can be achieved using the composite pattern:
-
-
-
-
-
-
-
-
-
-
-
- This approach works. However, it requires an extra layer between
- either the reader or the writer and the Step .
- Furthermore, the ItemWriter would need to be
- registered separately as an ItemStream with the
- Step . For this reason, the
- ItemTransfomer was renamed to
- ItemProcessor and moved up to the same level as
- ItemReader and
- ItemWriter :
-
-
-
-
-
-
-
-
-
-
-
+ The @EnableBatchProcessing annotation makes a number
+ of common dependencies available for autowiring by default. This list includes a
+ JobRepsitory , JobLauncher ,
+ JobRegistry , PlatformTransactionManager ,
+ JobBuilderFactory , and a StepBuilderFactory .
+ More information on how to configure Jobs and Steps with the new
+ Java config can be found in
-
- Configuration Enhancements
+
+ Spring Retry
- Until 2.0, the only option for configuring batch jobs has been
- normal spring bean configuration. However, in 2.0 there is a new namespace
- for configuration. For example, in 1.1, configuring a job looked like the
- following:
-
- <bean id="footballJob"
- class="org.springframework.batch.core.job.SimpleJob">
- <property name="steps">
- <list>
- <!-- Step bean details ommitted for clarity -->
- <bean id="playerload"/>
- <bean id="gameLoad"/>
- <bean id="playerSummarization"/>
- </list>
- </property>
- <property name="jobRepository" ref="jobRepository" />
-</bean>
-
- In 2.0, the equivalent would be:
-
- <job id="footballJob">
- <!-- Step bean details ommitted for clarity -->
- <step id="playerload" next="gameLoad"/>
- <step id="gameLoad" next="playerSummarization"/>
- <step id="playerSummarization"/>
-</job>
-
- More information on how to configure Jobs and Steps with the new
- namespace can be found in , and .
+ The ability to retry an operation via the RetryTemplate
+ has always been a feature of Spring Batch. That ability has been identified as a
+ useful feature for other frameworks (Spring Integration for example). With the 2.2.0
+ release, the retry logic has been extracted from Spring Batch into it's own library
+ called Spring Retry. With this change, there are two main impacts. The first is
+ that the majority of the org.springframework.batch.retry package
+ has been moved into this new library. With that move, the package name has also
+ dropped the batch to become org.springframework.retry .
-
- Meta Data Access Improvements
+
+ Job Parameters
- The JobRepository interface represents basic
- CRUD operations with Job meta-data. However, it may
- also be useful to query the meta-data. For that reason, the
- JobExplorer and JobOperator
- interfaces have been created:
+ Prior to the 2.2.0 release of Spring Batch, all parameters pass to a job execution
+ were used as part of the identity of the job. This limited the ability to change job
+ parameters during a rerun of a job. To accommodate this use case, 2.2.0 introduced the
+ idea of non-identifying job parameters.
-
-
-
-
-
-
-
-
-
-
- More information on the new meta data features can be found in . It is also worth noting that Jobs can now
- be stopped via the database, removing the requirement to maintain a handle
- to the JobExecution on the JVM the job was launched
- in.
+ By default, job parameters in 2.2.0 are still identifying. However, Spring Batch
+ now allows a user to specify a parameter not be used in the identity of a job instance.
+ In order to support this change, the domain model for batch changed. Before 2.2.0, job
+ parameters were associated with a JobInstance . 2.2.0 and beyond,
+ they are associated with a JobExecution . This also required the
+ underlying database schema for the job repository to change.
-
- Non Sequential Step Execution
-
- 2.0 has also seen improvements in how steps can be configured.
- Rather than requiring that they solely be sequential:
-
-
-
-
-
-
-
-
-
-
-
- They may now be conditional:
-
-
-
-
-
-
-
-
-
-
-
- This new 'conditional flow' support is made easy to configure via
- the new namespace:
-
- <job id="job">
- <step id="stepA">
- <next on="FAILED" to="stepB" />
- <next on="*" to="stepC" />
- </step>
- <step id="stepB" next="stepC" />
- <step id="stepC" />
-</job>
-
- More details on how to configure non sequential steps can be found
- in .
-
-
-
- Scalability
-
- Spring Batch 1.x was always intended as a single VM, possibly
- multi-threaded model, but many features were built into it that support
- parallel execution in multiple processes. Many projects have successfully
- implemented a scalable solution relying on the quality of service features
- of Spring Batch to ensure that processing only happens in the correct
- sequence. In 2.0 those features have been exposed more explicitly. There
- are two approaches to scalability: remote chunking, and
- partitioning.
-
-
- Remote Chunking
-
- Remote chunking is a technique for dividing up the work of a step
- without any explicit knowledge of the structure of the data. Any input
- source can be split up dynamically by reading it in a single process (as
- per normal in 1.x) and sending the items as a chunk to a remote worker
- process. The remote process implements a listener pattern, responding to
- the request, processing the data and sending an asynchronous reply. The
- transport for the request and reply has to be durable with guaranteed
- delivery and a single consumer, and those features are readily available
- with any JMS implementation. But Spring Batch is building the remote
- chunking feature on top of Spring Integration, therefore it is agnostic
- to the actual implementation of the message middleware. More details can
- be found in
-
-
-
- Partitioning
-
- Partitioning is an alternative approach which in contrast depends
- on having some knowledge of the structure of the input data, like a
- range of primary keys, or the name of a file to process. The advantage
- of this model is that the processors of each element in a partition can
- act as if they are a single step in a normal Spring Batch job. They
- don't have to implement any special or new patterns, which makes them
- easy to configure and test. Partitioning in principle is more scalable
- than remote chunking because there is no serialization bottleneck
- arising from reading all the input data in one place.
-
- In Spring Batch 2.0 partitioning is supported by two interfaces:
- PartitionHandler and
- StepExecutionSplitter . The
- PartitionHandler is the one that knows about the
- execution fabric - it has to transmit requests to remote steps and
- collect the results using whatever grid or remoting technology is
- available. PartitionHandler is an SPI, and Spring
- Batch provides one implementation out of the box for local execution
- through a TaskExecutor . This will be useful
- immediately when parallel processing of heavily IO bound tasks is
- required, since in those cases remote execution only complicates the
- deployment and doesn't necessarily help much with the performance. Other
- implementations will be specific to the execution fabric. (e.g. one of
- the grid providers such as IBM, Oracle, Terracotta, Appistry etc.),
- Spring Batch makes no preference for any of grid provider over another.
- More details can be found in
-
-