Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Reapplied changes overwritten by rev 2959

Browse Source
This commit is contained in:
dhgarrette
2009-02-11 22:11:03 +00:00
parent aff922709d
commit a205993ace
5 changed files with 4711 additions and 4585 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
docs/src/site/docbook/reference/domain.xml
View File

@@ -21,8 +21,8 @@
</listitem>
<listitem>
<para>simple and default implementations that allowed for quick
adoption and ease of use out-of-the-box</para>
<para>simple and default implementations that allow for quick adoption
and ease of use out-of-the-box</para>
</listitem>
<listitem>
@@ -47,13 +47,13 @@
<imageobject role="fo">
<imagedata align="center"
fileref="images/spring-batch-reference-model.png"
format="PNG" scale="70" />
format="PNG" />
</imageobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/spring-batch-reference-model.png"
format="PNG" scale="85" />
format="PNG" scale="80" />
</imageobject>
<caption><para>Figure 2.1: Batch Stereotypes</para></caption>
@@ -63,7 +63,7 @@
language of batch. A Job has one to many steps, which has exactly one
ItemReader, ItemProcessor, and ItemWriter. A job needs to be launched
(JobLauncher), and meta data about the currently running process needs to be
stored (JobRepository)</para>
stored (JobRepository).</para>
<section id="job">
<title id="jobStereotypes">Job</title>
@@ -78,7 +78,7 @@
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/job-heirarchy.png"
scale="90" />
scale="80" />
</imageobject>
<imageobject role="fo">
@@ -133,11 +133,11 @@
case of this job, there will be one logical
<classname>JobInstance</classname> per day. For example, there will be a
January 1st run, and a January 2nd run. If the January 1st run fails the
first time and is run again the next day, it's still the January 1st
run. (Usually this corresponds with the data its processing as well,
meaning the January 1st run processes data for January 1st, etc) That is
to say, each <classname>JobInstance</classname> can have multiple
executions. (<classname>JobExecution</classname> is discussed in more
first time and is run again the next day, it is still the January 1st
run. (Usually this corresponds with the data it is processing as well,
meaning the January 1st run processes data for January 1st, etc).
Therefore, each <classname>JobInstance</classname> can have multiple
executions (<classname>JobExecution</classname> is discussed in more
detail below) and only one <classname>JobInstance</classname>
corresponding to a particular <classname>Job</classname> can be running
at a given time. The definition of a <classname>JobInstance</classname>
@@ -151,11 +151,11 @@
likely be a business decision, it is left up to the
<classname>ItemReader</classname> to decide. What using the same
<classname>JobInstance</classname> will determine, however, is whether
or not the 'state' (i.e. The ExecutionContext, which is discussed below)
from previous executions will be used. Using a new
<classname>JobInstance</classname> will mean 'start from the beginning'
and using an existing instance will generally mean 'start from where you
left off'.</para>
or not the 'state' (i.e. the <classname>ExecutionContext</classname>,
which is discussed below) from previous executions will be used. Using a
new <classname>JobInstance</classname> will mean 'start from the
beginning' and using an existing instance will generally mean 'start
from where you left off'.</para>
</section>
<section id="jobParameters">
@@ -165,15 +165,15 @@
differs from <classname>Job</classname>, the natural question to ask is:
"how is one <classname>JobInstance</classname> distinguished from
another?" The answer is: <classname>JobParameters</classname>.
<classname>JobParameters</classname> are any set of parameters used to
start a batch job, which can be used for identification or even as
<classname>JobParameters</classname> is a set of parameters used to
start a batch job. They can be used for identification or even as
reference data during the run:</para>
<para><mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/job-stereotypes-parameters.png"
scale="90" />
scale="80" />
</imageobject>
<imageobject role="fo">
@@ -204,10 +204,10 @@
will not be considered complete unless the execution completes
successfully. Using the EndOfDay <classname>Job</classname> described
above as an example, consider a <classname>JobInstance</classname> for
01-01-2008 that failed the first time it was run. If it is ran again,
01-01-2008 that failed the first time it was run. If it is run again
with the same job parameters as the first run (01-01-2008), a new
JobExecution will be created. However, there will still be only one
<classname>JobInstance</classname>.</para>
<classname>JobExecution</classname> will be created. However, there will
still be only one <classname>JobInstance</classname>.</para>
<para>A <classname>Job</classname> defines what a job is and how it is
to be executed, and <classname>JobInstance</classname> is a purely
@@ -226,9 +226,9 @@
<entry>status</entry>
<entry>A <classname>BatchStatus</classname> object that
indicates the status of the execution. While it's running, it's
BatchStatus.STARTED, if it fails it's BatchStatus.FAILED, and if
it finishes successfully it's BatchStatus.COMPLETED</entry>
indicates the status of the execution. While running, it's
BatchStatus.STARTED, if it fails, it's BatchStatus.FAILED, and
if it finishes successfully, it's BatchStatus.COMPLETED</entry>
</row>
<row>
@@ -259,18 +259,19 @@
<entry>createTime</entry>
<entry>A <classname>java.util.Date</classname> representing the
current system time when the JobExecution was first persisted.
The job may not have been started yet (and thus has no start
time), but it will always have a createTime, which is required
by the framework for managing job level
ExecutionContexts.</entry>
current system time when the <classname>JobExecution</classname>
was first persisted. The job may not have been started yet (and
thus has no start time), but it will always have a createTime,
which is required by the framework for managing job level
<classname>ExecutionContext</classname>s.</entry>
</row>
<row>
<entry>lastUpdated</entry>
<entry>A <classname>java.util.Date</classname> representing the
last time a JobExecution was persisted.</entry>
last time a <classname>JobExecution</classname> was
persisted.</entry>
</row>
<row>
@@ -389,17 +390,17 @@
will be kicked off again for 01-01, starting where it left off and
completing successfully at 9:30. Because it's now the next day, the
01-02 job must be run as well, which is kicked off just afterwards at
9:31, and completes in it's normal one hour time at 10:30. There is no
9:31, and completes in its normal one hour time at 10:30. There is no
requirement that one <classname>JobInstance</classname> be kicked off
after another, unless there is potential for the two jobs to attempt to
access the same data, causing issues with locking at the database level.
It is entirely up to the scheduler to determine when a
<classname>Job</classname> should be run. Since they're separate
JobInstances, Spring Batch will make no attempt to stop them from being
run concurrently. (Attempting to run the same
<classname>JobInstance</classname>s, Spring Batch will make no attempt
to stop them from being run concurrently. (Attempting to run the same
<classname>JobInstance</classname> while another is already running will
result in a <classname>JobExecutionAlreadyRunningException</classname>
being thrown) There should now be an extra entry in both the
being thrown). There should now be an extra entry in both the
<classname>JobInstance</classname> and
<classname>JobParameters</classname> tables, and two extra entries in
the <classname>JobExecution</classname> table:</para>
@@ -554,7 +555,7 @@
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/jobHeirarchyWithSteps.png"
scale="90" />
scale="80" />
</imageobject>
<imageobject role="fo">
@@ -568,12 +569,12 @@
<para>A <classname>StepExecution</classname> represents a single attempt
to execute a <classname>Step</classname>. A new
<classname>StepExecution</classname> will be created each time a Step is
run, similar to <classname>JobExecution</classname>. However, if a step
fails to execute because the step before it fails, there will be no
execution persisted for it. A <classname>StepExecution</classname> will
only be created when it's <classname>Step</classname> is actually
started.</para>
<classname>StepExecution</classname> will be created each time a
<classname>Step</classname> is run, similar to
<classname>JobExecution</classname>. However, if a step fails to execute
because the step before it fails, there will be no execution persisted
for it. A <classname>StepExecution</classname> will only be created when
its <classname>Step</classname> is actually started.</para>
<para>Step executions are represented by objects of the
<classname>StepExecution</classname> class. Each execution contains a
@@ -596,8 +597,8 @@
<entry>A <classname>BatchStatus</classname> object that
indicates the status of the execution. While it's running, the
status is BatchStatus.STARTED, if it fails the status is
BatchStatus.FAILED, and if it finishes successfully the status
status is BatchStatus.STARTED, if it fails, the status is
BatchStatus.FAILED, and if it finishes successfully, the status
is BatchStatus.COMPLETED</entry>
</row>
@@ -670,15 +671,15 @@
<row>
<entry>processSkipCount</entry>
<entry>The number of times process has failed, resulting in a
skipped item.</entry>
<entry>The number of times <methodname>process</methodname> has
failed, resulting in a skipped item.</entry>
</row>
<row>
<entry>filterCount</entry>
<entry>The number of items that have been 'filtered' by the
ItemProcessor</entry>
<classname>ItemProcessor</classname>.</entry>
</row>
<row>
@@ -848,14 +849,14 @@
this scenario. This value will be updated just before each commit by the
framework, and can contain multiple rows corresponding to entries within
the <classname>ExecutionContext</classname>. Being notified before a
commit requires one of the various StepListeners, or an
<classname>ItemStream</classname>, which are discussed in more detail
later in this guide. As with the previous example, it is assumed that the
Job is restarted the next day. When it is restarted, the values from the
<classname>ExecutionContext</classname> of the last run are reconstituted
from the database, and when the <classname>ItemReader</classname> is
opened, it can check to see if it has any stored state in the context, and
initialize itself from there:</para>
commit requires one of the various <classname>StepListener</classname>s,
or an <classname>ItemStream</classname>, which are discussed in more
detail later in this guide. As with the previous example, it is assumed
that the <classname>Job</classname> is restarted the next day. When it is
restarted, the values from the <classname>ExecutionContext</classname> of
the last run are reconstituted from the database, and when the
<classname>ItemReader</classname> is opened, it can check to see if it has
any stored state in the context, and initialize itself from there:</para>
<programlisting> if (executionContext.containsKey(getKey(LINES_READ_COUNT))) {
log.debug("Initializing for restart. Restart data is: " + executionContext);
@@ -898,7 +899,7 @@
exists per <classname>StepExecution</classname> at any given time. Clients
of the <classname>ExecutionContext</classname> should be careful because
this creates a shared keyspace, so care should be taken when putting
values in to ensure no data is overwritten, however, the
values in to ensure no data is overwritten. However, the
<classname>Step</classname> stores absolutely no data in the context, so
there is no way to adversely affect the framework.</para>
@@ -915,7 +916,7 @@
</programlisting>
<para>As noted in the comment, ecStep will not equal ecJob, they are two
<para>As noted in the comment, ecStep will not equal ecJob; they are two
different <classname>ExecutionContext</classname>s. The one scoped to the
<classname>Step</classname> will be saved at every commit point in the
<classname>Step</classname>, whereas the one scoped to the
@@ -981,7 +982,7 @@
the output of a <classname>Step</classname>, one item at a time.
Generally, an item writer has no knowledge of the input it will receive
next, only the item that was passed in its current invocation. More
details about the <classname>ItemWriter</classname> interface and it's
details about the <classname>ItemWriter</classname> interface and its
various implementations can be found in <xref
linkend="readersAndWriters" /></para>
</section>
@@ -994,10 +995,10 @@
<classname>ItemReader</classname> reads one item, and the
<classname>ItemWriter</classname> writes them, the
<classname>ItemProcessor</classname> provides access to transform or apply
other business processing. If while processing the item it's determined
that it's not valid, returning null indicates that it should not be
written out. More details about the ItemProcessor interface can be found
in <xref linkend="readersAndWriters" />.</para>
other business processing. If, while processing the item, it is determined
that the item is not valid, returning null indicates that the item should
not be written out. More details about the ItemProcessor interface can be
found in <xref linkend="readersAndWriters" />.</para>
</section>
<section>

1680
docs/src/site/docbook/reference/job.xml
View File

File diff suppressed because it is too large Load Diff

5469
docs/src/site/docbook/reference/readersAndWriters.xml
View File

File diff suppressed because it is too large Load Diff

1225
docs/src/site/docbook/reference/schema-appendix.xml
View File

File diff suppressed because it is too large Load Diff

797
docs/src/site/docbook/reference/whatsnew.xml
View File

@@ -1,389 +1,408 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="whatsNew">
<title>What's new in Spring Batch 2.0</title>
<para>The Spring Batch 2.0 release has six major themes:</para>
<itemizedlist>
<listitem>
<para>Java 5</para>
</listitem>
<listitem>
<para>Non Sequential Step Execution</para>
</listitem>
<listitem>
<para>Chunk oriented processing</para>
</listitem>
<listitem>
<para>Meta Data enhancements</para>
</listitem>
<listitem>
<para>Scalability</para>
</listitem>
<listitem>
<para>Configuration</para>
</listitem>
</itemizedlist>
<section>
<title id="s.2.1.6">Java 5</title>
<para>The 1.x release of Spring Batch were all based on Java 1.4. This
prevented the framework from using many enhancements provided in Java 5
such as generics, parameterized types, etc. The entire framework has been
updated to utilize these features. <emphasis role="bold">Java 1.4 is no
longer supported.</emphasis> Most of the interfaces developers work with
have been updated to support generic types. As an example, the ItemReader
interface from 1.1 is below:</para>
<programlisting>
public interface ItemReader {
Object read() throws Exception;
void mark() throws MarkFailedException;
void reset() throws ResetFailedException;
}
</programlisting>
<para>As you can see, the <methodname>read</methodname> method returns an
Object. The 2.0 version is below:</para>
<programlisting>
public interface ItemReader&lt;T&gt; {
T read() throws Exception, UnexpectedInputException, ParseException;
}
</programlisting>
<para>As you can see, <classname>ItemReader</classname> now supports the
generic type, T, which is returned from read. You may also notice that
mark and reset have been removed. This is due to step processing strategy
changes, which are discussed below. Many other interfaces have been
similarly updated. </para>
</section>
<section>
<title>Chunk Oriented Processing</title>
<para>Previously, the default processing strategy provided by Spring Batch
was item-oriented processing:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/item-oriented-processing.png" scale="90"
width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/item-oriented-processing.png" scale="90"
width="50%" />
</imageobject>
</mediaobject>
<para>In item-oriented processing, the ItemReader returns one Object (the
'item) which is then handed to the ItemWriter, periodically committing
when the number of items hits the commit interval. For example, if the
commit interval is 5, ItemReader and ItemWriter will each be called 5
times. This is illustrated in a simplified code example below:</para>
<programlisting>
for(int i = 0; i &lt; commitInterval; i++){
Object item = itemReader.read();
itemWriter.write(item);
}
</programlisting>
<para>Both the ItemReader and ItemWriter interfaces were completely geared
toward this approach:</para>
<programlisting>
public interface ItemReader {
Object read() throws Exception;
void mark() throws MarkFailedException;
void reset() throws ResetFailedException;
}
</programlisting>
<programlisting> public interface ItemWriter {
void write(Object item) throws Exception;
void flush() throws FlushFailedException;
void clear() throws ClearFailedException;
}
</programlisting>
<para>Because the 'scope' of the processing was one item, supporting
rollback scenarios required additional methods, which is what mark, reset,
flush, and clear provided. If, after successfully reading and writing 2
items, the third had an error while writing, the transaction would need to
be rolled back. In this case, the clear method on the writer would be
called, indicating that it should clear its buffer, and reset would be
called on the ItemReader, indicating that it should return back to the
last position it was at when mark was called. (Both mark and flush are
called on commit)</para>
<para>In 2.0, this strategy has been changed to a chunk-oriented
approach:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/simplified-chunk-oriented-processing.png"
scale="90" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/simplified-chunk-oriented-processing.png"
scale="90" width="60%" />
</imageobject>
</mediaobject>
<para>Using the same example from above, if the commit interval is five,
read will be called 5 times, and write once. The items read will be
aggregated into a list, that will ultimately be written out, as the
simplified example below illustrates:</para>
<programlisting>
List items = new Arraylist();
for(int i = 0; i &lt; commitInterval; i++){
items.add(itemReader.read());
}
itemWriter.write(items);
</programlisting>
<para>This approach not only allows for much simpler processing and
scalability approaches, it also makes the ItemReader and ItemWriter
interfaces much cleaner:</para>
<programlisting>
public interface ItemReader&lt;T&gt; {
T read() throws Exception, UnexpectedInputException, ParseException;
}
</programlisting>
<programlisting>
public interface ItemWriter&lt;T&gt; {
void write(List&lt;? extends T&gt; items) throws Exception;
}
</programlisting>
<para>As you can see, the interfaces no longer contain the mark, reset,
flush, and clear methods. This makes the creation of readers and writers
much more straightforward for developers. In the case of
<classname>ItemReader</classname>, the interface is now forward-only. The
framework will buffer read items for developers in the case of rollback.
(There are exceptions if the underlying resource is transactional see:
<xref linkend="transactionalReaders" />) ItemWriter is also simplified,
since it gets the entire 'chunk' of items at once, rather than one at a
time, it can decide to flush any resources (such as a file or hibernate
session) before returning control to the <classname>Step</classname>. More
detailed information on chunk-oriented processing can be found in <xref
linkend="chunkOrientedProcessing" />. Reader and writer implementation
information can be found in 80<xref linkend="readersAndWriters" /></para>
<section>
<title>ItemProcessor</title>
<para>Previously, Steps had only two dependencies,
<classname>ItemReader</classname> and
<classname>ItemWriter</classname>:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/1-1-step.png" scale="80"
width="50%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" fileref="images/1-1-step.png" scale="80"
width="50%" />
</imageobject>
</mediaobject>
<para>The basic configuration above is fairly robust. However, there are
many cases where the item needs to be transformed before writing. In 1.x
this can be achieved using the composite pattern:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/composite-transformer.png"
scale="" width="70%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" fileref="images/composite-transformer.png"
scale="65" width="" />
</imageobject>
</mediaobject>
<para>This approach works, however, it requires an extra layer between
either the reader or the writer and the <classname>Step</classname>.
Furthermore, the <classname>ItemWriter</classname> would need to be
registered separately as an <classname>ItemStream</classname> with the
<classname>Step</classname>. For this reason, the ItemTransfomer was
renamed to ItemProcessor and moved up to the same level as ItemReader
and ItemWriter:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/step.png" scale=""
width="50%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" contentwidth="480"
fileref="images/step.png" scale="60" width="" />
</imageobject>
</mediaobject>
</section>
</section>
<section>
<title>Configuration enhancements</title>
<para>Until 2.0, the only option for configuring batch jobs has been
normal spring bean configuration. However, in 2.0 there is a new namespace
for configuration. For example, in 1.1, configuring a job looked like the
following:</para>
<programlisting>
&lt;bean id="footballJob"
class="org.springframework.batch.core.job.SimpleJob"&gt;
&lt;property name="steps"&gt;
&lt;list&gt;
&lt;!-- Step Bean details ommitted for clarity --&gt;
&lt;bean id="playerload" parent="simpleStep" /&gt;
&lt;bean id="gameLoad" parent="simpleStep" /&gt;
&lt;bean id="playerSummarization" parent="simpleStep" /&gt;
&lt;/list&gt;
&lt;/property&gt;
&lt;property name="jobRepository" ref="jobRepository" /&gt;
&lt;/bean&gt;
</programlisting>
<para>In 2.0, the equivalent would be:</para>
<programlisting>
&lt;job id="footballJob"&gt;
&lt;step name="playerload" next="gameLoad"/&gt;
&lt;step name="gameLoad" next="playerSummarization"/&gt;
&lt;step name="playerSummarization"/&gt;
&lt;/job&gt;
</programlisting>
<para>More information on how to configure Jobs and Steps with the new
namespace can be found in <xref linkend="configureJob" />, and <xref
linkend="configureStep" />.</para>
</section>
<section>
<title>Meta Data access improvements</title>
<para>The <classname>JobRepository</classname> interface represents basic
CRUD operations with <classname>Job</classname> meta-data. However, it can
be useful to query the meta-data. For that reason, the
<classname>JobExplorer</classname> and <classname>JobOperator</classname>
interfaces have been created:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/job-repository-advanced.png"
scale="90" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" fileref="images/job-repository-advanced.png"
scale="70" width="" />
</imageobject>
</mediaobject>
<para>More information on the new meta data features can be found in <xref
linkend="advancedMetaData" />. It is also worth noting that Jobs can now
be stopped via the database, removing the requirement to maintain a handle
to the <classname>JobExecution</classname> on the JVM the job was launched
in.</para>
</section>
<section>
<title>Non Sequential Step Execution</title>
<para>2.0 has also seen improvements in how steps can be configured.
Rather than requiring that they solely be sequential:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/sequential-flow.png"
scale="" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" fileref="images/sequential-flow.png"
scale="80" width="" />
</imageobject>
</mediaobject>
<para>They may now be conditional:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/conditional-flow.png"
scale="80" width="40%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" fileref="images/conditional-flow.png"
scale="80" width="40%" />
</imageobject>
</mediaobject>
<para>This new 'conditional flow' support is made easy to configure via
the new namespace:</para>
<programlisting>
&lt;job id="job"&gt;
&lt;step name="stepA"&gt;
&lt;next on="FAILED" to="stepB" /&gt;
&lt;next on="*" to="stepC" /&gt;
&lt;/step&gt;
&lt;step name="stepB" next="stepC" /&gt;
&lt;step name="stepC" /&gt;
&lt;/job&gt;
</programlisting>
<para>More details on how to configure non sequential steps can be found
in <xref linkend="controllingStepFlow" /></para>
</section>
</chapter>
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="whatsNew">
<title>What's new in Spring Batch 2.0</title>
<para>The Spring Batch 2.0 release has six major themes:</para>
<itemizedlist>
<listitem>
<para>Java 5</para>
</listitem>
<listitem>
<para>Non Sequential Step Execution</para>
</listitem>
<listitem>
<para>Chunk oriented processing</para>
</listitem>
<listitem>
<para>Meta Data enhancements</para>
</listitem>
<listitem>
<para>Scalability</para>
</listitem>
<listitem>
<para>Configuration</para>
</listitem>
</itemizedlist>
<section>
<title id="s.2.1.6">Java 5</title>
<para>The 1.x releases of Spring Batch were all based on Java 1.4. This
prevented the framework from using many enhancements provided in Java 5
such as generics, parameterized types, etc. The entire framework has been
updated to utilize these features. As a result, <emphasis role="bold">Java
1.4 is no longer supported.</emphasis> Most of the interfaces developers
work with have been updated to support generic types. As an example, the
<classname>ItemReader</classname> interface from 1.1 is below:</para>
<programlisting>
public interface ItemReader {
Object read() throws Exception;
void mark() throws MarkFailedException;
void reset() throws ResetFailedException;
}
</programlisting>
<para>As you can see, the <methodname>read</methodname> method returns an
<classname>Object</classname>. The 2.0 version is below:</para>
<programlisting>
public interface ItemReader&lt;T&gt; {
T read() throws Exception, UnexpectedInputException, ParseException;
}
</programlisting>
<para>As you can see, <classname>ItemReader</classname> now supports the
generic type, T, which is returned from read. You may also notice that
<methodname>mark</methodname> and <methodname>reset</methodname> have been
removed. This is due to step processing strategy changes, which are
discussed below. Many other interfaces have been similarly updated.</para>
</section>
<section>
<title>Chunk Oriented Processing</title>
<para>Previously, the default processing strategy provided by Spring Batch
was item-oriented processing:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/item-oriented-processing.png" scale="90"
width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/item-oriented-processing.png" scale="90"
width="50%" />
</imageobject>
</mediaobject>
<para>In item-oriented processing, the <classname>ItemReader</classname>
returns one <classname>Object</classname> (the 'item') which is then
handed to the <classname>ItemWriter</classname>, periodically committing
when the number of items hits the commit interval. For example, if the
commit interval is 5, <classname>ItemReader</classname> and
<classname>ItemWriter</classname> will each be called 5 times. This is
illustrated in a simplified code example below:</para>
<programlisting>
for(int i = 0; i &lt; commitInterval; i++){
Object item = itemReader.read();
itemWriter.write(item);
}
</programlisting>
<para>Both the <classname>ItemReader</classname> and
<classname>ItemWriter</classname> interfaces were completely geared toward
this approach:</para>
<programlisting>
public interface ItemReader {
Object read() throws Exception;
void mark() throws MarkFailedException;
void reset() throws ResetFailedException;
}
</programlisting>
<programlisting> public interface ItemWriter {
void write(Object item) throws Exception;
void flush() throws FlushFailedException;
void clear() throws ClearFailedException;
}
</programlisting>
<para>Because the 'scope' of the processing was one item, supporting
rollback scenarios required additional methods, which is what
<methodname>mark</methodname>, <methodname>reset</methodname>,
<methodname>flush</methodname>, and <methodname>clear</methodname>
provided. If, after successfully reading and writing 2 items, the third
has an error while writing, the transaction would need to be rolled back.
In this case, the <methodname>clear</methodname> method on the writer
would be called, indicating that it should <methodname>clear</methodname>
its buffer, and <methodname>reset</methodname> would be called on the
<classname>ItemReader</classname>, indicating that it should return back
to the last position it was at when <methodname>mark</methodname> was
called. (Both <methodname>mark</methodname> and
<methodname>flush</methodname> are called on commit)</para>
<para>In 2.0, this strategy has been changed to a chunk-oriented
approach:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/simplified-chunk-oriented-processing.png"
scale="90" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/simplified-chunk-oriented-processing.png"
scale="90" width="60%" />
</imageobject>
</mediaobject>
<para>Using the same example from above, if the commit interval is five,
read will be called 5 times, and write once. The items read will be
aggregated into a list, that will ultimately be written out, as the
simplified example below illustrates:</para>
<programlisting>
List items = new Arraylist();
for(int i = 0; i &lt; commitInterval; i++){
items.add(itemReader.read());
}
itemWriter.write(items);
</programlisting>
<para>This approach not only allows for much simpler processing and
scalability approaches, it also makes the
<classname>ItemReader</classname> and <classname>ItemWriter</classname>
interfaces much cleaner:</para>
<programlisting>
public interface ItemReader&lt;T&gt; {
T read() throws Exception, UnexpectedInputException, ParseException;
}
</programlisting>
<programlisting>
public interface ItemWriter&lt;T&gt; {
void write(List&lt;? extends T&gt; items) throws Exception;
}
</programlisting>
<para>As you can see, the interfaces no longer contain the
<methodname>mark</methodname>, <methodname>reset</methodname>,
<methodname>flush</methodname>, and <methodname>clear</methodname>
methods. This makes the creation of readers and writers much more
straightforward for developers. In the case of
<classname>ItemReader</classname>, the interface is now forward-only. The
framework will buffer read items for developers in the case of rollback
(though there are exceptions if the underlying resource is transactional
see: <xref linkend="transactionalReaders" />).
<classname>ItemWriter</classname> is also simplified, since it gets the
entire 'chunk' of items at once, rather than one at a time, it can decide
to flush any resources (such as a file or hibernate session) before
returning control to the <classname>Step</classname>. More detailed
information on chunk-oriented processing can be found in <xref
linkend="chunkOrientedProcessing" />. Reader and writer implementation
information can be found in <xref linkend="readersAndWriters" />.</para>
<section>
<title>ItemProcessor</title>
<para>Previously, <classname>Step</classname>s had only two
dependencies, <classname>ItemReader</classname> and
<classname>ItemWriter</classname>:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/1-1-step.png" scale="80"
width="50%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/1-1-step.png" scale="80"
width="50%" />
</imageobject>
</mediaobject>
<para>The basic configuration above is fairly robust. However, there are
many cases where the item needs to be transformed before writing. In 1.x
this can be achieved using the composite pattern:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/composite-transformer.png"
scale="" width="70%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/composite-transformer.png"
scale="65" width="" />
</imageobject>
</mediaobject>
<para>This approach works. However, it requires an extra layer between
either the reader or the writer and the <classname>Step</classname>.
Furthermore, the <classname>ItemWriter</classname> would need to be
registered separately as an <classname>ItemStream</classname> with the
<classname>Step</classname>. For this reason, the
<classname>ItemTransfomer</classname> was renamed to
<classname>ItemProcessor</classname> and moved up to the same level as
<classname>ItemReader</classname> and
<classname>ItemWriter</classname>:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/step.png" scale=""
width="50%" />
</imageobject>
<imageobject role="fo">
<imagedata align="center" contentwidth="480"
fileref="images/step.png"
scale="60" width="" />
</imageobject>
</mediaobject>
</section>
</section>
<section>
<title>Configuration enhancements</title>
<para>Until 2.0, the only option for configuring batch jobs has been
normal spring bean configuration. However, in 2.0 there is a new namespace
for configuration. For example, in 1.1, configuring a job looked like the
following:</para>
<programlisting>
&lt;bean id="footballJob"
class="org.springframework.batch.core.job.SimpleJob"&gt;
&lt;property name="steps"&gt;
&lt;list&gt;
&lt;!-- Step Bean details ommitted for clarity --&gt;
&lt;bean id="playerload" parent="simpleStep" /&gt;
&lt;bean id="gameLoad" parent="simpleStep" /&gt;
&lt;bean id="playerSummarization" parent="simpleStep" /&gt;
&lt;/list&gt;
&lt;/property&gt;
&lt;property name="jobRepository" ref="jobRepository" /&gt;
&lt;/bean&gt;
</programlisting>
<para>In 2.0, the equivalent would be:</para>
<programlisting>
&lt;job id="footballJob"&gt;
&lt;step name="playerload" next="gameLoad"/&gt;
&lt;step name="gameLoad" next="playerSummarization"/&gt;
&lt;step name="playerSummarization"/&gt;
&lt;/job&gt;
</programlisting>
<para>More information on how to configure Jobs and Steps with the new
namespace can be found in <xref linkend="configureJob" />, and <xref
linkend="configureStep" />.</para>
</section>
<section>
<title>Meta Data access improvements</title>
<para>The <classname>JobRepository</classname> interface represents basic
CRUD operations with <classname>Job</classname> meta-data. However, it may
also be useful to query the meta-data. For that reason, the
<classname>JobExplorer</classname> and <classname>JobOperator</classname>
interfaces have been created:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/job-repository-advanced.png"
scale="90" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/job-repository-advanced.png"
scale="70" width="" />
</imageobject>
</mediaobject>
<para>More information on the new meta data features can be found in <xref
linkend="advancedMetaData" />. It is also worth noting that Jobs can now
be stopped via the database, removing the requirement to maintain a handle
to the <classname>JobExecution</classname> on the JVM the job was launched
in.</para>
</section>
<section>
<title>Non Sequential Step Execution</title>
<para>2.0 has also seen improvements in how steps can be configured.
Rather than requiring that they solely be sequential:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/sequential-flow.png"
scale="" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/sequential-flow.png"
scale="80" width="" />
</imageobject>
</mediaobject>
<para>They may now be conditional:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/conditional-flow.png"
scale="80" width="" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="images/conditional-flow.png"
scale="80" width="" />
</imageobject>
</mediaobject>
<para>This new 'conditional flow' support is made easy to configure via
the new namespace:</para>
<programlisting>
&lt;job id="job"&gt;
&lt;step name="stepA"&gt;
&lt;next on="FAILED" to="stepB" /&gt;
&lt;next on="*" to="stepC" /&gt;
&lt;/step&gt;
&lt;step name="stepB" next="stepC" /&gt;
&lt;step name="stepC" /&gt;
&lt;/job&gt;
</programlisting>
<para>More details on how to configure non sequential steps can be found
in <xref linkend="controllingStepFlow" />.</para>
</section>
</chapter>