Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

copy samples docos over from trunk

Browse Source
This commit is contained in:
dsyer
2008-07-29 17:09:05 +00:00
parent 42237215a2
commit 29c4b77e4a
1 changed files with 478 additions and 193 deletions
Download Patch File Download Diff File Expand all files Collapse all files

671
spring-batch-samples/src/site/apt/index.apt
View File

@@ -46,37 +46,45 @@ Spring Batch Samples
Here is a list of samples with checks to indicate which features each one demonstrates:
*----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
|<<Job / Feature>> | <<delimited input>> | <<fixed-length input>> | <<xml input>> | <<multiline input>> | <<db driving query input>> | <<db cursor input>> | <<delimited output>> | <<fixed-length output>> | <<xml output>> | <<multiline output>> | <<db output>> | <<skip>> | <<restart>> | <<automatic mapping>> | <<JMX>> | <<validation>> | <<delegation>>
|<<Job / Feature>> | <<delimited input>> | <<fixed-length input>> | <<xml input>> | <<multiline input>> | <<db driving query input>> | <<db cursor input>> | <<delimited output>> | <<fixed-length output>> | <<xml output>> | <<multiline output>> | <<db output>> | <<skip>> | <<restart>> | <<automatic mapping>> | <<asynch launch>> | <<validation>> | <<delegation>> | <<write behind>> |
*----
adhocLoopJob | | | | | | | | | | | | | | | x | |
{{adhocLoop}} | | | | | | | | | | | | | | |x | | | |
*----
beanWrapperMapperSample | | x | | | | | | | | | x | | | x | | x |
{{batchUpdate}} | | | | | |x | | | | | | | | | | | |x |
*----
compositeProcessorSample | | | | | | | | x | | | x | | | | | x |
{{beanWrapperMapperSample}} | |x | | | | | | | | |x | | |x | |x | | |
*----
delegatingJob | | | | | | | | | | | | | | | | | x
{{compositeItemWriterSample}}| | | | | | | |x | | |x | | | | |x | | |
*----
fixedLengthImportJob | | x | | | | | | | | | x | | | | | x |
{{delegating}} | | | | | | | | | | | | | | | | |x | |
*----
hibernateJob | | | | | | x | | | | | x | | | | | |
{{fixedLengthImport}} | |x | | | | | | | | |x | | | | |x | | |
*----
ibatisJob | | | | | x | | | | | | x | | | | | |
{{football}} |x | | | | |x | | | | |x | | | | | | | |
*----
infiniteLoopJob | | | | | | | | | | | | | | | | |
{{hibernate}} | | | | | |x | | | | |x | | | | | | |x |
*----
multilineJob | | x | | x | | | | | | | | | | | | |
{{ibatis}} | | | | |x | | | | | |x | | | | | | | |
*----
multilineOrderJob | x | | | x | | | | x | | x | | | | | | |
{{multiline}} | |x | |x | | | | | | | | | | | | | | |
*----
fotballlJob | x | | | | | x | | | | | x | | | | | |
{{multilineOrder}} |x | | |x | | | |x | |x | | | | | | | | |
*----
restartSample | | x | | | | | | | | | x | | x | | | x |
{{parallel}} | |x | | | | | | | | |x | | | | |x | | |
*----
simpleTaskletJob | | | | | | | | | | | | | | | | |
{{quartzSample}} |x | | | | |x | | | | |x | | | |x | | | |
*----
tradeJob | x | | | | x | | x | | | | x | | | | | x |
{{restartSample}} | |x | | | | | | | | |x | |x | | |x | | |
*----
xmlStaxJob | | | x | | | | | | x | | | | | | | |
{{retrySample}} | | | | | | | | | | | | | | | |x | | |
*----
{{skipSample}} |x | | | |x | |x | | | |x |x | | | |x | | |
*----
{{tasklet}} | | | | | | | | | | | | | | | | | | |
*----
{{trade}} |x | | | |x | |x | | | |x | | | | |x | | |
*----
{{xmlStax}} | | |x | | | | | |x | | | | | | | | | |
*----
* Common Sample Source Structures
@@ -102,84 +110,93 @@ xmlStaxJob | | | x | | | | | | x | | | | | | | |
Each job consists of several steps, these steps are defined in steps
property.
** Tasklet Job
* Adhoc Loop and JMX Demo ({adhocLoop})
The goal is to show the simplest use of the batch framework with a
single job with a single step, which cleans up a directory and runs
a system command.
This job is simply an infinite loop. It runs forever so it is
useful for testing features to do with stopping and starting jobs.
It is used, for instance, as one of the jobs that can be run from
JMX using the Eclipse launch configuration "jmxLauncher".
<Description:> This job is defined by <<<taskletJob.xml>>> file. The
<<<Job>>> itself is defined by the bean definition with
<<<id="taskletJob">>>. In this example we have two steps.
The JMX launcher uses an additional XML configuration file
(adhoc-job-launcher-context.xml) to set up a <<<JobLauncher>>> for
running jobs asynchronously (i.e. in a background thread). This
follows the same pattern as the {{{quartzSample}Quartz sample}}, so
see that section for more details of the <<<JobLauncher>>>
configuration.
* The first step defines a tasklet that is responsible for
clearing out a directory though a custom <<<Tasklet>>>. Each
tasklet has an <<<execute()>>> method which is called by the
step. All processing of business data should be handled by this
method.
The rest of the configuration for this demo consists of exposing
some components from the application context as JMX managed beans.
The <<<JobLauncher>>> is exposed as a stripped down interface
<<<ExportedJobLauncher>>>, so that it can be controlled from a
remote client (such as JConsole from the JDK) which does not have
Spring Batch on the classpath. See the Spring Core Reference Guide
for more details on how to customise the JMX configuration.
* The second step uses another tasklet to execute a system (OS)
command line.
* Batch Update ({batchUpdate})
<XML definition:> taskletJob.xml
The purpose of this sample is to show to usage of the
<<<BatchSqlUpdateItemWriter>>> to make efficient updates to a
database table.
You can visualize the Spring configuration of a job through
Spring-IDE. See {{{http://springide.org/blog/}Spring IDE}}. The
source view of the configuration is as follows:
The <<<BatchSqlUpdateItemWriter>>> accepts a special form of
<<<PreparedStatementSetter>>> as a (mandatory) dependency. This is
responsible for copying fields from the item to be written to a
<<<PreparedStatement>>> matching the SQL query that has been
injected. The implementation of the
<<<CustomerCreditUpdatePreparedStatementSetter>>> shows best
practice of keeping all the information needed for the execution in
one place, since it contains a static constant value (<<<QUERY>>>)
which is used to configure the query for the writer.
+---
<bean id="taskletJob" parent="simpleJob">
<property name="steps">
<list>
<bean id="deleteFilesInDir" parent="taskletStep">
<property name="tasklet">
<bean
class="org.springframework.batch.sample.tasklet.FileDeletingTasklet">
<property name="directoryResource"
ref="directory" />
</bean>
</property>
</bean>
<bean id="executeSystemCommand" parent="taskletStep">
<property name="tasklet">
<bean
class="org.springframework.batch.sample.tasklet.SystemCommandTasklet">
<property name="command" value="echo hello" />
<!-- 5 second timeout for the command to complete -->
<property name="timeout" value="5000" />
</bean>
</property>
</bean>
</list>
</property>
* BeanWrapperMapper Sample ({beanWrapperMapperSample})
</bean>
This sample shows the use of automatic mapping from fields in a file
to a domain object. The <<<Trade>>> and <<<Person>>> objects needed
by the job are created from the Spring configuration using prototype
beans, and then their properties are set using the
<<<BeanWrapperFieldSetMapper>>>, which sets properties of the
prototype according to the field names in the file.
<bean id="directory"
class="org.springframework.core.io.FileSystemResource">
<constructor-arg value="target/test-outputs/test-dir" />
</bean>
+---
Nested property paths are resolved in the same way as normal Spring
binding occurs, but with a little extra leeway in terms of spelling
and capitalisation. Thus for instance, the <<<Trade>>> object has a
property called <<<customer>>> (lower case), but the file has been
configured to have a column name <<<CUSTOMER>>> (upper case), and
the mapper will accept the values happily. Underscores instead of
camel-casing (e.g. <<<CREDIT_CARD>>> instead of <<<creditCard>>>)
also work.
For simplicity we are only displaying the job configuration itself
and leaving out the details of the supporting batch execution
environment configuration.
* Composite ItemWriter Sample ({compositeItemWriterSample})
** Fixed Length Import Job
This shows a common use case using a composite pattern, composing
instances of other framework readers or writers. It is also quite
common for business-specific readers or writers to wrap
off-the-shelf components in a similar way.
In this job the composite pattern is used just to make duplicate
copies of the output data. The delegates for the
<<<CompositeItemWriter>>> have to be separately registered as
streams in the <<<Step>>> where they are used, in order for the step
to be restartable. This is a common feature of all delegate
patterns.
* Delegating Sample ({delegating})
This sample shows the delegate pattern again, and also the
<<<ItemReaderAdapter>>> which is used to adapt a POJO to the
<<<ItemReader>>> interface.
* Fixed Length Import Job ({fixedLengthImport})
The goal is to demonstrate a typical scenario of importing data
from a fixed-length file to database
<Description:> This job shows a more typical scenario, when reading
input data and processing the data is cleanly separated. The data
provider is responsible for reading input and mapping each record to
a domain object, which is then passed to the module processor. The
module processor handles the processing of the domain objects, in
this case it only writes them to database.
<XML definition:> fixedLengthImportJob.xml
<Input source:> file with fixed row structure
This job shows a typical scenario, when reading input data and
processing the data is cleanly separated. The data provider is
responsible for reading input and mapping each record to a domain
object, which is then passed to the module processor. The module
processor handles the processing of the domain objects, in this case
it only writes them to database.
In this example we are using a simple fixed length record structure
that can be found in the project at
@@ -214,109 +231,7 @@ xmlStaxJob | | | x | | | | | | x | | | | | | | |
object
* Multiline Order Job
The goal is to demostrate how to handle a more complex file input
format, where a record meant for processing inludes nested records
and spans multiple lines
<XML definition:> multilineOrderJob.xml
<Input source:> file with multiline records. OrderDataProvider is
an example of a non-default programmatic data provider. It reads
input until it detects that the multiline record has finished and
encapsulates the record in a single domain object.
<Output target:> file with multiline records. The concrete
<<<ItemWriter>>> passes the object to a an injected 'delegate
writer' which in this case writes the output to a file. The writer
in this case demonstrates how to write multiline output using a
custom aggregator transformer.
* Quartz Sample
The goal is to demonstrate how to schedule job execution using
Quartz scheduler. In this case there is no unit test to launch the
sample because it just re-uses the football job. There is a main
method in <<<QuartzBatchLauncher>>> and an Eclipse launch
configuration which runs it with empty arguments. The main method
is very basic - it is intended only as a guide to how Quartz might
be used in principle.
<XML definition:> <<<quartz-job-launcher.xml>>>, also re-uses
<<<footballJob.xml>>>
The configuration declares a <<<JobLauncher>>> bean. The launcher
bean is different from the other samples only in that it uses an
asynchronous task executor, so that the jobs are launched in a
separate thread to the main method:
+---
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor">
<bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
</property>
</bean>
+---
Also, a Quartz <<<JobDetail>>> is defined using a Spring
<<<JobDetailBean>>> as a convenience.
+--
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="0/10 * * * * ?" />
</bean>
</property>
</bean>
+--
Finally, a trigger with a scheduler is defined that will launch the
job detail every 10 seconds:
+---
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="0/10 * * * * ?" />
</bean>
</property>
</bean>
+---
The job is thus scheduled to run every 10 seconds. In fact it
should be successful on the first attempt, so the second and
subsequent attempts should through a
<<<JobInstanceAlreadyCompleteException>>>. In a production system,
the job detail would probably be modified to account for this
exception (e.g. catch it and re-submit with a new set of job
parameters). The point here is that Spring Batch guarantees that
the job execution is idempotent - you can never inadvertently
process the same data twice.
* Trade Job
The goal is to show a reasonably complex scenario, that would
resemble the real-life usage of the framework.
<Description:> This job has 3 steps. First, data about trades are
imported from a file to database. Second, the trades are read from
the database and credit on customer accounts is decreased
appropriately. Last, a report about customers is exported to a file.
<XML definition:> <<<tradeJob.xml>>> - the job definition,
<<<tradeJobIo.xml>>> - input and output configuration
<Description:> This job has 3 steps. First, data about trades is
imported from a file to database. Second, the data about trades is
read from the database and credit on customer accounts is decreased
appropriately. Last, a report about customers is exported to a file.
* Football Job
* Football Job ({football})
This is a (American) Football statistics loading job. We gave it the
id of <<<footballJob>>> in our configuration file. Before diving
@@ -363,10 +278,10 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
receptions, rushes, and total touchdowns.
Our example batch job is going to load both files into a database,
and then combine each to summarize how each player performed for a
and then combine each to summarise how each player performed for a
particular year. Although this example is fairly trivial, it shows
multiple types of input, and the general style is a common batch
scenario. That is, summarizing a very large dataset so that it can
scenario. That is, summarising a very large dataset so that it can
be more easily manipulated or viewed by an online web-based
application. In an enterprise solution the third step, the reporting
step, could be implemented through the use of Eclipse BIRT or one of
@@ -425,10 +340,10 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
ref="playerFileItemReader" />
<property name="itemWriter">
<bean
class="org.springframework.batch.sample.item.writer.PlayerItemWriter">
class="org.springframework.batch.sample.domain.football.internal.internal.PlayerItemWriter">
<property name="playerDao">
<bean
class="org.springframework.batch.sample.dao.JdbcPlayerDao">
class="org.springframework.batch.sample.domain.football.internal.internal.JdbcPlayerDao">
<property name="dataSource"
ref="dataSource" />
</bean>
@@ -448,7 +363,7 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
the developer can remain solely concerned with their business
logic.
* <ItemReaderr> – the item reader is the source of the information
* <ItemReader> – the item reader is the source of the information
pipe. At the most basic level input is read in from an input
source, parsed into a domain object and returned. In this way, the
good batch architecture practice of ensuring all data has been
@@ -486,7 +401,7 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
Following the general flow of the batch job, the next step is to
describe how each line of the file will be parsed from its string
representation into a domain object. The first thing the provider
will need is an InputSource, which is provided as part of the Spring
will need is an <<<ItemReader>>>, which is provided as part of the Spring
Batch infrastructure. Because the input is flat-file based, a
<<<FlatFileItemReader>>> is used:
@@ -504,7 +419,7 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
</property>
<property name="fieldSetMapper">
<bean
class="org.springframework.batch.sample.mapping.PlayerFieldSetMapper" />
class="org.springframework.batch.sample.domain.football.internal.internal.PlayerFieldSetMapper" />
</property>
</bean>
+---
@@ -525,7 +440,7 @@ AbduKa00,1996,mia,15,nyg,0,0,0,0,0,17,96,,14,0
<<<FieldSetMapper>>> interface. There is a single method,
<<<mapLine()>>>, which maps <<<FieldSet>>>s the same way that
developers are comfortable mapping <<<ResultSet>>>s into Java
<<<Object>>>s, either by index or fieldname. This behavior is by
<<<Object>>>s, either by index or field name. This behaviour is by
intention and design similar to the <<<RowMapper>>> passed into a
<<<JdbcTemplate>>>. You can see this below:
@@ -581,7 +496,7 @@ public class
<property name="dataSource" ref="dataSource" />
<property name="mapper">
<bean
class="org.springframework.batch.sample.mapping.PlayerSummaryMapper" />
class="org.springframework.batch.sample.domain.football.internal.internal.PlayerSummaryMapper" />
</property>
<property name="sql">
<value>
@@ -596,7 +511,7 @@ games.player_id group by games.player_id, games.year_no
</bean>
+----
The SqlCursorInputSource has three dependences:
The <<<JdbcCursorItemReader>>> has three dependences:
* A <<<DataSource>>>
@@ -613,5 +528,375 @@ games.player_id group by games.player_id, games.year_no
sample application you can execute the JUnit test
<<<FootballJobFunctionalTests>>>, and you'll see an output showing
each of the records as they are processed. Please keep in mind that
AoP is used to wrap the <<<ItemWriter>> and output each record as it
AoP is used to wrap the <<<ItemWriter>>> and output each record as it
is processed to the logger, which may impact performance.
* Hibernate {hibernate}
The purpose of this sample is to show a typical usage of Hibernate
as an ORM tool in the input and output of a job.
The job uses a <<<HibernateCursorItemReader>>> for the input, where
a simple HQL query is used to supply items. It also uses a
non-framework <<<ItemWriter>>> wrapping a DAO, which perhaps was
written as part of an online system.
The output reliability and robustness are improved by the use of the
<<<HibernateAwareItemWriter>>> from the framework. One of its roles
is to buffer items and flush them explicitly, rather than implicitly
on a transaction boundary (which would be the default). This
"write-behind" behaviour is provided by Hibernate implicitly, but we
need to take control of it so that the skip and retry features
provided by Spring Batch can work effectively. Thus the other role
of the <<<HibernateAwareItemWriter>>> is to watch out for failures
and flush aggressively when an item is seen from a previously failed
chunk. In this way there will always be a failure at some point
immediately after the bad item was written, and the item is then
easily identifiable.
* Ibatis ({ibatis})
The goal of this sample is to show the use of Ibatis as a query
mapping tool. Its features are similar to the Hibernate sample, but
it uses Ibatis to drive its input and output.
* Multiline ({multiline})
The goal of this sample is to show some common tricks with multiline
records in file input jobs.
The input file in this case consists of two groups of trades
delimited by special lines in a file (BEGIN and END):
+---
BEGIN
UK21341EAH4597898.34customer1
UK21341EAH4611218.12customer2
END
BEGIN
UK21341EAH4724512.78customer2
UK21341EAH4810809.25customer3
UK21341EAH4985423.39customer4
END
+---
The goal of the job is to operate on the two groups, so the item
type is naturally <<<List<Trade>>>>. To get these items delivered
from an item reader we employ two components from Spring Batch: the
<<<AggregateItemReader>>> and the
<<<PrefixMatchingCompositeLineTokenizer>>>. The latter is
responsible for recognising the difference between the trade data
and the delimiter records. The former is responsible for
aggregating the trades from each group into a <<<List>>> and handing
out the list from its <<<read()>>> method. To help these components
perform their responsibilities we also provide some business
knowledge about the data in the form of a <<<FieldSetMapper>>>
(<<<TradeFieldSetMapper>>>). The <<<TradeFieldSetMapper>>> checks
its input for the delimiter fields (BEGIN, END) and if it detects
them, returns the special tokens that <<<AggregateItemReader>>>
needs. Otherwise it maps the input into a <<<Trade>>> object.
* Multiline Order Job ({multilineOrder})
The goal is to demonstrate how to handle a more complex file input
format, where a record meant for processing includes nested records
and spans multiple lines
The input source is file with multiline records.
<<<OrderItemReader>>> is an example of a non-default programmatic
item reader. It reads input until it detects that the multiline
record has finished and encapsulates the record in a single domain
object.
The output target is a file with multiline records. The concrete
<<<ItemWriter>>> passes the object to a an injected 'delegate
writer' which in this case writes the output to a file. The writer
in this case demonstrates how to write multiline output using a
custom aggregator transformer.
* Parallel Sample ({parallel})
The purpose of this sample is to show multi-threaded step execution
using the Process Indicator pattern.
The job reads data from the same file as the
{{{fixedLengthImport}Fixed Length Import}} sample, but instead of
writing it out directly it goes through a staging table, and the
staging table is read in a multi-threaded step. Note that for such
a simple example where the item processing was not expensive, there
is unlikely to be much if any benefit in using a multi-threaded
step.
Multi-threaded step execution is easy to configure using Spring
Batch, but there are some limitations. Most of the out-of-the-box
<<<ItemReader>>> and <<<ItemWriter>>> implementations are not
designed to work in this scenario because they need to be
restartable and they are also stateful. There should be no surprise
about this, and reading a file (for instance) is usually fast enough
that multi-threading that part of the process is not likely to
provide much benefit, compared to the cost of managing the state.
The best strategy to cope with restart state from multiple
concurrent threads depends on the kind of input source involved:
* For file-based input (and output) restart sate is practically
impossible to manage. Spring Batch does not provide any features
or samples to help with this use case.
* With message middleware input it is trivial to manage restarts,
since there is no state to store (if a transaction rolls back the
messages are returned to the destination they came from).
* With database input state management is still necessary, but it
isn't particularly difficult. The easiest thing to do is rely on
a Process Indicator in the input data, which is a column in the
data indicating for each row if it has been processed or not. The
flag is updated inside the batch transaction, and then in the case
of a failure the updates are lost, and the records will show as
un-processed on a restart.
This last strategy is implemented in the <<<StagingItemReader>>>.
Its companion, the <<<StagingItemWriter>>> is responsible for
setting up the data in a staging table which contains the process
indicator. The reader is then driven by a simple SQL query that
includes a where clause for the processed flag, i.e.
+---
SELECT ID FROM BATCH_STAGING WHERE JOB_ID=? AND PROCESSED=? ORDER BY ID
+---
It is then responsible for updating the processed flag (which
happens inside the main step transaction).
* Quartz Sample ({quartz})
The goal is to demonstrate how to schedule job execution using
Quartz scheduler. In this case there is no unit test to launch the
sample because it just re-uses the football job. There is a main
method in <<<JobRegistryBackgroundJobRunner>>> and an Eclipse launch
configuration which runs it with arguments to pick up the football
job.
The additional XML configuration for this job is in
<<<quartz-job-launcher.xml>>>, and it also re-uses
<<<footballJob.xml>>>
The configuration declares a <<<JobLauncher>>> bean. The launcher
bean is different from the other samples only in that it uses an
asynchronous task executor, so that the jobs are launched in a
separate thread to the main method:
+---
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor">
<bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
</property>
</bean>
+---
Also, a Quartz <<<JobDetail>>> is defined using a Spring
<<<JobDetailBean>>> as a convenience.
+--
<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="org.springframework.batch.sample.quartz.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="footballJob"/>
<entry key="jobLocator" value-ref="jobRegistry"/>
<entry key="jobLauncher" value-ref="jobLauncher"/>
</map>
</property>
</bean>
+--
Finally, a trigger with a scheduler is defined that will launch the
job detail every 10 seconds:
+---
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="jobDetail" />
<property name="cronExpression" value="0/10 * * * * ?" />
</bean>
</property>
</bean>
+---
The job is thus scheduled to run every 10 seconds. In fact it
should be successful on the first attempt, so the second and
subsequent attempts should through a
<<<JobInstanceAlreadyCompleteException>>>. In a production system,
the job detail would probably be modified to account for this
exception (e.g. catch it and re-submit with a new set of job
parameters). The point here is that Spring Batch guarantees that
the job execution is idempotent - you can never inadvertently
process the same data twice.
* Restart Sample ({restartSample})
The goal of this sample is to show how a job can be restarted after
a failure and continue processing where it left off.
To simulate a failure we "fake" a failure on the fourth record
though the use of a sample component
<<<ExceptionThrowingItemReaderProxy>>>. This is a stateful reader
that counts how many records it has processed and throws a planned
exception in a specified place. Since we re-use the same instance
when we restart the job it will not fail the second time.
* Retry Sample ({retrySample})
The purpose of this sample is to show how to use the automatic retry
capabilities of Spring Batch.
The retry is configured in the step through the
<<<SkipLimitStepFactoryBean>>>:
+---
<bean id="step1" parent="simpleStep"
class="org.springframework.batch.core.step.item.SkipLimitStepFactoryBean">
...
<property name="retryLimit" value="3" />
<property name="retryableExceptionClasses" value="java.lang.Exception" />
</bean>
+---
Failed items will cause a rollback for all <<<Exception>>> types, up
to a limit of 3 attempts. On the 4th attempt, the failed item would
be skipped, and there would be a callback to a
<<<ItemSkipListener>>> if one was provided (via the "listeners"
property of the step factory bean).
An <<<ItemReader>>> is provided that will generate unique
<<<Trade>>> data by just incrementing a counter. Note that it uses
the counter in its <<<mark()>>> and <<<reset()>>> methods so that
the same content is returned after a rollback. The same content is
returned, but the instance of <<<Trade>>> is different, which means
that the implementation of <<<equals()>>> in the <<<Trade>>> object
is important. This is because to identify a failed item on retry
(so that the number of attempts can be counted) the framework by
default uses <<<Object.equals()>>> to compare the recently failed
item with a cache of previously failed items. Without implementing
a field-based <<<equals()>>> method for the domain object, our job
will spin round the retry for potentially quite a long time before
failing because the default implementation of <<<equals()>>> is
based on object reference, not on field content.
* Skip Sample ({skipSample})
The purpose of this sample is to show how to use the skip features
of Spring Batch. Since skip is really just a special case of retry
(with limit 0), the details are quite similar to the {{{retrySample}Retry
Sample}}, but the use case is less artificial, since it
is based on the {{{trade}Trade Sample}}.
The failure condition is still artificial, since it is triggered by
a special <<<ItemWriter>>> wrapper (<<<ItemTrackingItemWriter>>>).
The plan is that a certain item (the third) will fail business
validation in the writer, and the system can then respond by
skipping it. We also configure the step so that it will not roll
back on the validation exception, since we know that it didn't
invalidate the transaction, only the item. This is done through the
transaction attribute:
+---
<bean id="step2" parent="skipLimitStep">
<property name="skipLimit" value="1" />
<!-- No rollback for exceptions that are marked with "+" in the tx attributes -->
<property name="transactionAttribute"
value="+org.springframework.batch.item.validator.ValidationException" />
....
</bean>
+---
The format for the transaction attribute specification is given in
the Spring Core documentation (e.g. see the Javadocs for
{{{http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/transaction/interceptor/TransactionAttributeEditor.html}TransactionAttributeEditor}}).
* Tasklet Job ({tasklet})
The goal is to show the simplest use of the batch framework with a
single job with a single step, which cleans up a directory and runs
a system command.
<Description:> The
<<<Job>>> itself is defined by the bean definition with
<<<id="taskletJob">>>. In this example we have two steps.
* The first step defines a tasklet that is responsible for
clearing out a directory though a custom <<<Tasklet>>>. Each
tasklet has an <<<execute()>>> method which is called by the
step. All processing of business data should be handled by this
method.
* The second step uses another tasklet to execute a system (OS)
command line.
You can visualise the Spring configuration of a job through
Spring-IDE. See {{{http://springide.org/blog/}Spring IDE}}. The
source view of the configuration is as follows:
+---
<bean id="taskletJob" parent="simpleJob">
<property name="steps">
<list>
<bean id="deleteFilesInDir" parent="taskletStep">
<property name="tasklet">
<bean
class="org.springframework.batch.sample.tasklet.FileDeletingTasklet">
<property name="directoryResource"
ref="directory" />
</bean>
</property>
</bean>
<bean id="executeSystemCommand" parent="taskletStep">
<property name="tasklet">
<bean
class="org.springframework.batch.sample.common.SystemCommandTasklet">
<property name="command" value="echo hello" />
<!-- 5 second timeout for the command to complete -->
<property name="timeout" value="5000" />
</bean>
</property>
</bean>
</list>
</property>
</bean>
<bean id="directory"
class="org.springframework.core.io.FileSystemResource">
<constructor-arg value="target/test-outputs/test-dir" />
</bean>
+---
For simplicity we are only displaying the job configuration itself
and leaving out the details of the supporting batch execution
environment configuration.
* Trade Job ({trade})
The goal is to show a reasonably complex scenario, that would
resemble the real-life usage of the framework.
This job has 3 steps. First, data about trades are imported from a
file to database. Second, the trades are read from the database and
credit on customer accounts is decreased appropriately. Last, a
report about customers is exported to a file.
* XML Input Output ({xmlStax})
The goal here is to show the use of XML input and output through
streaming and Spring OXM marshallers and unmarshallers.
The job has a single step that copies <<<Trade>>> data from one XML
file to another. It uses XStream for the object XML conversion,
because this is simple to configure for basic use cases like this
one. See
{{{http://static.springframework.org/spring-ws/sites/1.5/reference/html/oxm.html}Spring
OXM documentation}} for details of other options.