Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

BATCH-674: Reorganized and updated reference documentation.

Browse Source
This commit is contained in:
lucasward
2009-01-26 06:49:03 +00:00
parent b0f182e951
commit 7c01de7aed
27 changed files with 2116 additions and 2412 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/models/Figures.ppt
View File

Binary file not shown.

BIN
docs/models/diagrams.ppt Executable file
View File

Binary file not shown.

33
docs/src/site/docbook/reference/Discard.xml
View File

@@ -1,33 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<chapter>
<title>Discard</title>
<section>
<title id="s.5.2">Support Stereotypes</title>
<para>While item readers and writers serve as the main entry points for
item-oriented processing, they might be supplemented by a number of
support classes that perform specific tasks within the reader / writer
lifecycle. These support stereotypes are useful for dividing the work of
item readers and writers into reusable pieces, as well as abstracting away
the details of processing, such as interaction with external systems.
Additionally, they give us another opportunity to leverage the powerful
configuration features of the Spring framework, as we can switch between
several beans implementing these support interfaces without changing the
driving item reader or writer.</para>
<section>
<title id="s.2.4.2">Item Transformers</title>
<para>An item transformer is a class that is capable of taking an object
and changing it somehow before processing occurs. For instance, an item
transformer my alter an object by changing its properties or by
replacing it with another object entirely, such as a wrapper or
derivative object. It can also be defined as an adaptor, allowing an
object of one type to be converted for use as an object of a second
type.</para>
</section>
</section>
</chapter>

17
docs/src/site/docbook/reference/batch-job-testing.xml
View File

@@ -1,17 +0,0 @@
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="batch-job-testing">
<title>Batch Unit and Integration Tests</title>
<section>
<title id="bjt.1">Unit Testing</title>
<para>Document Batch Job Unit Testing features. This includes the use of Mock Objects,
embedded database (HSQLDB), etc. </para>
</section>
<section>
<title id="bjt.2">Integration Testing</title>
<para>Document how to test against the targeted database, applications, etc.</para>
</section>
</chapter>

17
docs/src/site/docbook/reference/batch-launch.xml
View File

@@ -1,17 +0,0 @@
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="batch-launch">
<title>Run Tier - Launching Batch Jobs</title>
<section>
<title id="bl.1">Mapping Batch Error Codes to Launch Client Error Codes</title>
<para>Mapping Batch Error Codes to Launch Client Error Codes</para>
</section>
<section>
<title id="bl.2">Launch Batch from Command Line</title>
<para>Document Command Line Launching</para>
</section>
<section>
<title id="bl.3">Launch Batch On Demand</title>
<para>Document Launching Batch Jobs on Demand</para>
</section>
</chapter>

52
docs/src/site/docbook/reference/batch-performance-testing.xml
View File

@@ -1,52 +0,0 @@
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="batch-performance-testing">
<title>Batch Performance Testing</title>
<section>
<title id="bpt.1">Performance Testing Overview</title>
<para>A batch performance test team needs to have the following at their disposal:
</para>
<para>
<itemizedlist>
<listitem><para>
Define performance Targets -
</para></listitem>
<listitem><para>
Establishing the requirements for a performance testing environment -
</para></listitem>
<listitem><para>
Performance Data - generating adequate volumes of realistic data for performance testing
</para></listitem>
<listitem><para>
Performance Tools -
</para></listitem>
<listitem><para>
Performance Team Roles - Tool SME's, performance DBA.
</para></listitem>
</itemizedlist>
</para>
</section>
<section>
<title id="bpt.2">Defining Performance Targets</title>
<para></para>
</section>
<section>
<title id="bpt.3">Establishing Performance requirements and installing the environment.</title>
<para>Establishing the requirements for the performance environment. </para>
</section>
<section>
<title id="bpt.4">Performance Data</title>
<para>Generating adequate volumes of realistic data for performance testing.</para>
</section>
<section>
<title id="bpt.5">Performance Tools</title>
<para></para>
</section>
<section>
<title id="bpt.6">Performance Team Roles</title>
<para>Tool SME's, performance dba's, environment experts (OS, JVM, etc.)</para>
</section>
</chapter>

232
docs/src/site/docbook/reference/common-patterns.xml Normal file
View File

@@ -0,0 +1,232 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="patterns">
<title>Common Batch Patterns</title>
<section>
<title>Introduction</title>
<para>Some batch jobs can be assembled purely from off-the-shelf
components in Spring Batch, mostly the <classname>ItemReader</classname>
and <classname>ItemWriter</classname> implementations. Where this is not
possible (the majority of cases) the main API entry points for application
developers are the <classname>Tasklet</classname>,
<classname>ItemReader</classname>, <classname>ItemWriter</classname> and
the various listener interfaces. Most simple batch jobs will be able to
use off-the-shelf input from a Spring Batch
<classname>ItemReader</classname>, but it is very often the case that
there are custom concerns in the processing and writing, which normally
leads developers to implement an <classname>ItemWriter</classname>, or
<classname>ItemTransformer</classname>.</para>
<para>Here we provide a few examples of common patterns in custom business
logic, mainly using the listener interfaces . It should be noted that an
<classname>ItemReader</classname> or <classname>ItemWriter</classname> can
implement the listener interfaces as well if appropriate.</para>
</section>
<section>
<title>Logging Item Processing and Failures</title>
<para>A common use case is the need for special handling of errors in a
step, item by item, perhaps logging to a special channel, or inserting a
record into a database. The <classname>StepHandlerStep</classname>
(created from the step factory beans) allows users to implement this use
case with a simple <classname>ItemReadListener</classname>, for errors on
read, and an <classname>ItemWriteListener</classname>, for errors on
write. The below code snippets illustrate a listener that logs both read
and write failures:</para>
<programlisting>public class ItemFailureLoggerListener extends ItemListenerSupport {
private static Log logger = LogFactory.getLog("item.error");
public void onReadError(Exception ex) {
logger.error("Encountered error on read", e);
}
public void onWriteError(Exception ex, Object item) {
logger.error("Encountered error on write", e);
}
}</programlisting>
<para>Having implemented this listener it must be registered with the
step:</para>
<programlisting>&lt;bean id="simpleStep"
class="org.springframework.batch.core.step.item.SimpleStepFactoryBean" &gt;
...
&lt;property name="listeners"&gt;
&lt;bean class="org.example...ItemFailureLoggerListener"/&gt;
&lt;/property&gt;
&lt;/bean&gt;</programlisting>
<para>Remember that if your listener does anything in an
<code>onError()</code> method, it will be inside a transaction that is
going to be rolled back. If you need to use a transactional resource such
as a database inside an <code>onError()</code> method, consider adding a
declarative transaction to that method (see Spring Core Reference Guide
for details), and giving its propagation attribute the value
REQUIRES_NEW.</para>
</section>
<section>
<title>Stopping a Job Manually for Business Reasons</title>
<para>Spring Batch provides a stop() method through the JobLauncher
interface, but this is really aimed at the operator, rather than the
application programmer. Sometimes it is more convenient or makes more
sense to stop a job execution from within the business logic.</para>
<para>The simplest thing to do is to throw a RuntimeException (one that
isn't retried indefinitely or skipped), For example, a custom exception
type could be used, as in the example below:</para>
<programlisting>public class PoisonPillItemWriter extends AbstractItemWriter {
public void write(Object item) throws Exception {
if (isPoisonPill(item)) {
throw new PoisonPillException("Posion pill detected: "+item);
}
}
}</programlisting>
<para>Another simple way to stop a step from executing is to simply return
<code>null</code> from the <classname>ItemReader</classname>:</para>
<programlisting>public class EarlyCompletionItemReader extends AbstractItemReader {
private ItemReader delegate;
public void setDelegate(ItemReader delegate) { ... }
public Object read() throws Exception {
Object item = delegate.read();
if (isEndItem(item)) {
return null; // end the step here
}
return item;
}
}</programlisting>
<para>The previous example actually relies on the fact that there is a
default implementation of the <classname>CompletionPolicy</classname>
strategy which signals a complete batch when the item to be processed is
null. A more sophisticated completion policy could be implemented and
injected into the <classname>Step</classname> through the
<classname>RepeatOperationsStepFactoryBean</classname>:</para>
<programlisting>&lt;bean id="simpleStep"
class="org.springframework.batch.core.step.item.RepeatOperationsStepFactoryBean" &gt;
...
&lt;property name="chunkOperations"&gt;
&lt;bean class="org.springframework.batch.repeat.support.RepeatTemplate"&gt;
&lt;property name="completionPolicy"&gt;
&lt;bean class="org.example...SpecialCompletionPolicy"/&gt;
&lt;/property&gt;
&lt;/bean&gt;
&lt;/property&gt;
&lt;/bean&gt;</programlisting>
<para>An alternative is to set a flag in the
<classname>StepExecution</classname>, which is checked by the
<classname>Step</classname> implementations in the framework in between
item processing. To implement this alternative, we need access to the
current StepExecution, and this can be achieved by implementing a
StepListener and registering it with the Step. Here is an example of a
listener that sets the flag:</para>
<programlisting>public class CustomItemWriter extends ItemListenerSupport implements StepListener {
private StepExecution stepExecution;
public void beforeStep(StepExecution stepExecution) {
this.stepExecution = stepExecution;
}
public void afterRead(Object item) {
if (isPoisonPill(item)) {
stepExecution.setTerminateOnly(true);
}
}
}</programlisting>
<para>The default behaviour here when the flag is set is for the step to
throw a <classname>JobInterruptedException</classname>. This can be
controlled through the <classname>StepInterruptionPolicy</classname>, but
the only choice is to throw or not throw an exception, so this is always
an abnormal ending to a job.</para>
</section>
<section>
<title>Adding a Footer Record</title>
<para>A very common requirement is to aggregate information during the
output process and to append a record at the end of a file summarizing the
data, or providing a checksum. This can also be achieved with a callbacks
in the step, normally as part of a custom
<classname>ItemWriter</classname>. In this case, since a job is
accumulating state that should not be lost if the job aborts, the
<classname>ItemStream</classname> interface should be implemented:</para>
<programlisting>public class CustomItemWriter extends AbstractItemWriter implements
ItemStream, StepListener
{
private static final String TOTAL_AMOUNT_KEY = "total.amount";
private ItemWriter delegate;
private double totalAmount = 0.0;
public void setDelegate(ItemWriter delegate) { ... }
public ExitStatus afterStep(StepExecution stepExecution) {
// Add the footer record here...
delegate.write("Total Amount Processed: " + totalAmount);
}
public void open(ExecutionContext executionContext) {
if (executionContext.containsKey(TOTAL_AMOUNT_KEY) {
totalAmount = executionContext.getDouble(TOTAL_AMOUNT_KEY);
}
}
public void update(ExecutionContext executionContext) {
executionContext.setDouble(TOTAL_AMOUNT_KEY, totalAmount);
}
public void write(Object item) {
delegate.write(item);
totalAmount += ((Trade) item).getAmount();
}
}</programlisting>
<para>The custom writer in the example is stateful (it maintains its total
in an instance variable <varname>totalAmount</varname>), but the state is
stored through the <classname>ItemStream</classname> interface in the
<classname>ExecutionContext</classname>. In this way we can be sure that
when the <code>open()</code> callback is received on a restart. The
framework garuntees we always get the last value that was committed. It
should be noted that it is not always necessary to implement ItemStream.
For example, if the ItemWriter is re-runnable, in the sense that it
maintains its own state in a transactional resource like a database, there
is no need to maintain state within the writer itself.</para>
</section>
</chapter>

737
docs/src/site/docbook/reference/core.xml → docs/src/site/docbook/reference/domain.xml
View File

@@ -4,130 +4,69 @@
<chapter id="core">
<title>The Domain Language of Batch</title>
<section>
<title>Introduction</title>
<para>To any experienced batch architect, the overall concepts of batch
processing used in Spring Batch should be familiar and comfortable. There
are “Jobs” and “Steps” and developer supplied processing units called
ItemReaders and ItemWriters. However, because of the Spring patterns,
operations, templates, callbacks, and idioms, there are opportunities for
the following:<itemizedlist>
<listitem>
<para>significant improvement in adherence to a clear separation of
concerns</para>
</listitem>
<para>To any experienced batch architect, the overall concepts of batch
processing used in Spring Batch should be familiar and comfortable. There
are “Jobs” and “Steps” and developer supplied processing units called
ItemReaders and ItemWriters. However, because of the Spring patterns,
operations, templates, callbacks, and idioms, there are opportunities for
the following:<itemizedlist>
<listitem>
<para>significant improvement in adherence to a clear separation of
concerns</para>
</listitem>
<listitem>
<para>clearly delineated architectural layers and services provided as
interfaces</para>
</listitem>
<listitem>
<para>clearly delineated architectural layers and services provided
as interfaces</para>
</listitem>
<listitem>
<para>simple and default implementations that allowed for quick
adoption and ease of use out-of-the-box</para>
</listitem>
<listitem>
<para>simple and default implementations that allowed for quick
adoption and ease of use out-of-the-box</para>
</listitem>
<listitem>
<para>significantly enhanced extensibility</para>
</listitem>
</itemizedlist></para>
<listitem>
<para>significantly enhanced extensibility</para>
</listitem>
</itemizedlist></para>
<para>The diagram below is simplified version of the batch reference
architecture that has been used for decades. It provides an overview of the
components that make up the domain language of batch processing. This
architecture framework is a blueprint that has been proven through decades
of implementations on the last several generations of platforms
(COBOL/Mainframe, C++/Unix, and now Java/anywhere). JCL and COBOL developers
are likely to be as comfortable with the concepts as C++, C# and Java
developers. Spring Batch provides a physical implementation of the layers,
components and technical services commonly found in robust, maintainable
systems used to address the creation of simple to complex batch
applications, with the infrastructure and extensions to address very complex
processing needs.</para>
<para>The diagram below is only a slight variation of the batch reference
architecture that has been used for decades. It provides an overview of
the high level components, technical services, and basic operations
required by a batch architecture. This architecture framework is a
blueprint that has been proven through decades of implementations on the
last several generations of platforms (COBOL/Mainframe, C++/Unix, and now
Java/anywhere). JCL and COBOL developers are likely to be as comfortable
with the concepts as C++, C# and Java developers. Spring Batch provides a
physical implementation of the layers, components and technical services
commonly found in robust, maintainable systems used to address the
creation of simple to complex batch applications, with the infrastructure
and extensions to address very complex processing needs.</para>
</section>
<mediaobject>
<imageobject role="fo">
<imagedata align="center"
fileref="src/site/docbook/reference/images/spring-batch-reference-model.png"
format="PNG" />
</imageobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/spring-batch-reference-model.png"
format="PNG" scale="80" />
</imageobject>
<caption><para>Figure 2.1: Batch Stereotypes</para></caption>
</mediaobject>
<para>The diagram above highlights the key concepts that make up the domain
language of batch. A Job has one to many steps, which has exactly one
ItemReader, ItemProcessor, and ItemWriter. A job needs to be launched
(JobLauncher), and meta data about the currently running process needs to be
stored (JobRepository)</para>
<section>
<title id="s.2">Batch Application Style Interactions and Services</title>
<mediaobject>
<imageobject role="fo">
<imagedata align="center"
fileref="src/site/docbook/reference/images/spring-batch-reference-model.png"
format="PNG" />
</imageobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/spring-batch-reference-model.png"
format="PNG" />
</imageobject>
<caption><para>Figure 2.1: Batch Stereotypes</para></caption>
</mediaobject>
<para>The above diagram highlights the interactions and key services
provided by the Spring Batch framework. The colors used are important to
understanding the responsibilities of a developer in Spring Batch. Grey
represents an external application such as an enterprise scheduler or a
database. It's important to note that scheduling is grey, and should thus
be considered separate from Spring Batch. Blue represents application
architecture services. In most cases these are provided by Spring Batch
with out of the box implementations, but an architecture team may make
specific implementations that better address their specific needs. Yellow
represents the pieces that must be configured by a developer. For example,
a job schedule needs to be configured so that the job is kicked off at the
appropriate time. A job configuration file also needs to be created, which
defines how a job will be run. It is also worth noting that the
<classname>ItemReader</classname> and <classname>ItemWriter</classname>
used by an application may just as easily be a custom one made by a
developer for their specific batch job, rather than one provided by Spring
Batch or an architecture team.</para>
<para>The Batch Application Style is organized into four logical tiers,
which include Run, Job, Application, and Data. The primary goal for
organizing an application according to the tiers is to embed what is known
as "separation of concerns" within the system. These tiers can be
conceptual but may prove effective in mapping the deployment of the
artifacts onto physical components like Java runtimes and integration with
data sources and targets. Effective separation of concerns results in
reducing the impact of change to the system. The four conceptual tiers
containing batch artifacts are:</para>
<para><itemizedlist>
<listitem>
<para><emphasis role="bold">Run Tier:</emphasis> The Run Tier is
concerned with the scheduling and launching of the application. A
vendor product is typically used in this tier to allow time-based
and interdependent scheduling of batch jobs as well as providing
parallel processing capabilities.</para>
</listitem>
<listitem>
<para><emphasis role="bold">Job Tier:</emphasis> The Job Tier is
responsible for the overall execution of a batch job. It
sequentially executes batch steps, ensuring that all steps are in
the correct state and all appropriate policies are enforced.</para>
</listitem>
<listitem>
<para><emphasis role="bold">Application Tier:</emphasis> The
Application Tier contains components required to execute the
program. It contains specific tasks that address required batch
functionality and enforces policies around execution (e.g., commit
intervals, capture of statistics, etc.)</para>
</listitem>
<listitem>
<para><emphasis role="bold">Data Tier:</emphasis> The Data Tier
provides integration with the physical data sources that might
include databases, files, or queues.</para>
</listitem>
</itemizedlist></para>
</section>
<section>
<title id="jobStereotypes">Job Stereotypes</title>
<title id="jobStereotypes">Job</title>
<para>This section describes stereotypes relating to the concept of a
batch job. A <classname>Job</classname> is an entity that encapsulates an
@@ -138,7 +77,8 @@
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/job-heirarchy.png" />
<imagedata align="center" fileref="images/job-heirarchy.png"
scale="80" />
</imageobject>
<imageobject role="fo">
@@ -147,51 +87,40 @@
</imageobject>
</mediaobject>
<section>
<title id="s.2.1.1">Job</title>
<para>In Spring Batch, a Job is simply a continer for Steps. It combines
multiple steps that belong logically together in a flow and allows for
configuration of properties global to all steps, such as restartability.
The job configuration contains: </para>
<para>A job is represented by a Spring bean that implements the
<classname>Job</classname> interface and contains all of the information
necessary to define the operations performed by a job. A job
configuration is typically contained within a Spring XML configuration
file and the job's name is determined by the "id" attribute associated
with the job configuration bean. The job configuration contains</para>
<itemizedlist>
<listitem>
<para>The simple name of the job</para>
</listitem>
<itemizedlist>
<listitem>
<para>The simple name of the job</para>
</listitem>
<listitem>
<para>Definition and ordering of Steps</para>
</listitem>
<listitem>
<para>Definition and ordering of Steps</para>
</listitem>
<listitem>
<para>Whether or not the job is restartable</para>
</listitem>
</itemizedlist>
<listitem>
<para>Whether or not the job is restartable</para>
</listitem>
</itemizedlist>
<para>A default simple implementation of the <classname>Job</classname>
interface is provided by Spring Batch in the form of the
<classname>SimpleJob</classname> class which creates some standard
functionality on top of <classname>Job</classname>, however the batch
namespace abstracts away the need to instaniate it directly. Instead, the
<code>&lt;job&gt;</code> tag can be used:</para>
<para>A default simple implementation of the <classname>Job</classname>
interface is provided by Spring Batch in the form of the
<classname>SimpleJob</classname> class which creates some standard
functionality on top of <classname>Job</classname>, namely a standard
execution logic that all jobs should utilize. In general, all jobs
should be defined using a bean of type
<classname>SimpleJob</classname>:</para>
<programlisting>
&lt;job id="footballJob"&gt;
&lt;step name="playerload" next="gameLoad"/&gt;
&lt;step name="gameLoad" next="playerSummarization"/&gt;
&lt;step name="playerSummarization"/&gt;
&lt;/job&gt;
<programlisting> &lt;bean id="footballJob"
class="org.springframework.batch.core.job.SimpleJob"&gt;
&lt;property name="steps"&gt;
&lt;list&gt;
&lt;!-- Step Bean details ommitted for clarity --&gt;
&lt;bean id="playerload" parent="simpleStep" /&gt;
&lt;bean id="gameLoad" parent="simpleStep" /&gt;
&lt;bean id="playerSummarization" parent="simpleStep" /&gt;
&lt;/list&gt;
&lt;/property&gt;
&lt;property name="restartable" value="true" /&gt;
&lt;/bean&gt;</programlisting>
</section>
</programlisting>
<section>
<title id="s.2.1.2">JobInstance</title>
@@ -238,13 +167,28 @@
another?" The answer is: <classname>JobParameters</classname>.
<classname>JobParameters</classname> are any set of parameters used to
start a batch job, which can be used for identification or even as
reference data during the run. In the example above, where there are two
instances, one for January 1st, and another for January 2nd, there is
really only one Job, one that was started with a job parameter of
01-01-2008 and another that was started with a parameter of 01-02-2008.
Thus, the contract can be defined as: <classname>JobInstance</classname>
= <classname>Job</classname> + <classname>JobParameters</classname>.
This allows a developer to effectively control how you a
reference data during the run:</para>
<para><mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/job-stereotypes-parameters.png"
scale="80" />
</imageobject>
<imageobject role="fo">
<imagedata align="center"
fileref="src/site/docbook/reference/images/job-heirarchy.png" />
</imageobject>
</mediaobject></para>
<para>In the example above, where there are two instances, one for
January 1st, and another for January 2nd, there is really only one Job,
one that was started with a job parameter of 01-01-2008 and another that
was started with a parameter of 01-02-2008. Thus, the contract can be
defined as: <classname>JobInstance</classname> =
<classname>Job</classname> + <classname>JobParameters</classname>. This
allows a developer to effectively control how a
<classname>JobInstance</classname> is defined, since they control what
parameters are passed in.</para>
</section>
@@ -258,10 +202,10 @@
<classname>JobInstance</classname> corresponding to a given execution
will not be considered complete unless the execution completes
successfully. Using the EndOfDay <classname>Job</classname> described
above as an example, consider a JobInstance for 01-01-2008 that failed
the first time it was run. If it is ran again, with the same job
parameters as the first run (01-01-2008), a new JobExecution will be
created. However, there will still be only one
above as an example, consider a <classname>JobInstance</classname> for
01-01-2008 that failed the first time it was run. If it is ran again,
with the same job parameters as the first run (01-01-2008), a new
JobExecution will be created. However, there will still be only one
<classname>JobInstance</classname>.</para>
<para>A <classname>Job</classname> defines what a job is and how it is
@@ -334,6 +278,15 @@
<entry>The 'property bag' containing any user data that needs to
be persisted between executions.</entry>
</row>
<row>
<entry>failureExceptions</entry>
<entry>The list of exceptions encountered during the execution
of a <classname>Job</classname>. These can be useful if more
than one exception is encountered during the failure of a
<classname>Job</classname>.</entry>
</row>
</tbody>
</tgroup>
</table>
@@ -573,26 +526,29 @@
</section>
<section>
<title id="s.2.1">Step Stereotypes</title>
<title id="s.2.1">Step</title>
<para>A <classname>Step</classname> is a domain object that encapsulates
an independent, sequential phase of a batch job. Therefore, every
<classname>Job</classname> is composed entirely of one or more steps. A
<classname>Step</classname> should be thought of as a unique processing
stream that will be executed in sequence. For example, if you have one
step that loads a file into a database, another that reads from the
database, validates the data, preforms processing, and then writes to
another table, and another that reads from that table and writes out to a
file. Each of these steps will be performed completely before moving on to
the next step. The file will be completely read into the database before
step 2 can begin. As with <classname>Job</classname>, a
<classname>Step</classname> contains all of the information necessary to
define and control the actual batch processing. This is a necessarily
vague description because the contents of any given
<classname>Step</classname> are at the discretion of the developer writing
a <classname>Job</classname>. A Step can be as simple or complex as the
developer desires. A simple <classname>Step</classname> might load data
from a file into the database, requiring little or no code. (depending
upon the implementations used) A more complex <classname>Step</classname>
may have complicated business rules that are applied as part of the
processing. As with <classname>Job</classname>, a
<classname>Step</classname> has an individual
<classname>StepExecution</classname> that corresponds with a unique
<classname>JobExecution</classname>:</para>
<mediaobject>
<imageobject role="html">
<imagedata align="center" fileref="images/jobHeirarchyWithSteps.png" />
<imagedata align="center" fileref="images/jobHeirarchyWithSteps.png"
scale="80" />
</imageobject>
<imageobject role="fo">
@@ -601,30 +557,6 @@
</imageobject>
</mediaobject>
<section>
<title id="step">Step</title>
<para>A <classname>Step</classname> contains all of the information
necessary to define and control the actual batch processing. This is a
necessarily vague description because the contents of any given
<classname>Step</classname> are at the discretion of the developer
writing a <classname>Job</classname>. A Step can be as simple or complex
as the developer desires. A simple <classname>Step</classname> might
load data from a file into the database, requiring little or no code.
(depending upon the implementations used) A more complex
<classname>Step</classname> may have complicated business rules that are
applied as part of the processing.</para>
<para>Steps are defined by instantiating implementations of the
<classname>Step</classname> interface. Two step implementation classes
are available in the Spring Batch framework, and they are each discussed
in detail in Chatper 4 of this guide. For most situations, the
<classname>StepHandlerStep</classname> implementation is sufficient, but
for situations where only one call is needed, such as a stored procedure
call or a wrapper around existing script, a
<classname>TaskletStep</classname> may be a better option.</para>
</section>
<section>
<title id="stepExecution">StepExecution</title>
@@ -753,181 +685,173 @@
</tgroup>
</table>
</section>
</section>
<section>
<title>ExecutionContext</title>
<section>
<title>ExecutionContext</title>
<para>An <classname>ExecutionContext</classname> represents a collection
of key/value pairs that are persisted and controlled by the framework in
order to allow developers a place to store persistent state that is
scoped to a <classname>StepExecution</classname> or
<classname>JobExecution</classname>. For those familiar with Quartz, it
is very similar to <classname>JobDataMap</classname>. The best usage
example is restart. Using flat file input as an example, while
processing individual lines, the framework periodically persists the
<classname>ExecutionContext</classname> at commit points. This allows
the <classname>ItemReader</classname> to store its state in case a fatal
error occurs during the run, or even if the power goes out. All that is
needed is to put the current number of lines read into the context, and
the framework will do the rest:</para>
<para>An <classname>ExecutionContext</classname> represents a collection
of key/value pairs that are persisted and controlled by the framework in
order to allow developers a place to store persistent state that is scoped
to a <classname>StepExecution</classname> or
<classname>JobExecution</classname>. For those familiar with Quartz, it is
very similar to <classname>JobDataMap</classname>. The best usage example
is to facilitate restart. Using flat file input as an example, while
processing individual lines, the framework periodically persists the
<classname>ExecutionContext</classname> at commit points. This allows the
<classname>ItemReader</classname> to store its state in case a fatal error
occurs during the run, or even if the power goes out. All that is needed
is to put the current number of lines read into the context, and the
framework will do the rest:</para>
<programlisting>executionContext.putLong(getKey(LINES_READ_COUNT), reader.getPosition());</programlisting>
<programlisting>executionContext.putLong(getKey(LINES_READ_COUNT), reader.getPosition());</programlisting>
<para>Using the EndOfDay example from the Job Stereotypes section as an
example, assume there's one step: 'loadData', that loads a file into the
database. After the first failed run, the meta data tables would look
like the following:</para>
<para>Using the EndOfDay example from the Job Stereotypes section as an
example, assume there's one step: 'loadData', that loads a file into the
database. After the first failed run, the meta data tables would look like
the following:</para>
<para><table>
<title>BATCH_JOB_INSTANCE</title>
<para><table>
<title>BATCH_JOB_INSTANCE</title>
<tgroup cols="2">
<tbody>
<row>
<entry>JOB_INSTANCE_ID</entry>
<tgroup cols="2">
<tbody>
<row>
<entry>JOB_INSTANCE_ID</entry>
<entry>JOB_NAME</entry>
</row>
<entry>JOB_NAME</entry>
</row>
<row>
<entry>1</entry>
<row>
<entry>1</entry>
<entry>EndOfDayJob</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_JOB_PARAMS</title>
<entry>EndOfDayJob</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_JOB_PARAMS</title>
<tgroup cols="4">
<tbody>
<row>
<entry>JOB_INSTANCE_ID</entry>
<tgroup cols="4">
<tbody>
<row>
<entry>JOB_INSTANCE_ID</entry>
<entry>TYPE_CD</entry>
<entry>TYPE_CD</entry>
<entry>KEY_NAME</entry>
<entry>KEY_NAME</entry>
<entry>DATE_VAL</entry>
</row>
<entry>DATE_VAL</entry>
</row>
<row>
<entry>1</entry>
<row>
<entry>1</entry>
<entry>DATE</entry>
<entry>DATE</entry>
<entry>schedule.Date</entry>
<entry>schedule.Date</entry>
<entry>2008-01-01 00:00:00</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_JOB_EXECUTION</title>
<entry>2008-01-01 00:00:00</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_JOB_EXECUTION</title>
<tgroup cols="5">
<tbody>
<row>
<entry>JOB_EXECUTION_ID</entry>
<tgroup cols="5">
<tbody>
<row>
<entry>JOB_EXECUTION_ID</entry>
<entry>JOB_INSTANCE_ID</entry>
<entry>JOB_INSTANCE_ID</entry>
<entry>START_TIME</entry>
<entry>START_TIME</entry>
<entry>END_TIME</entry>
<entry>END_TIME</entry>
<entry>STATUS</entry>
</row>
<entry>STATUS</entry>
</row>
<row>
<entry>1</entry>
<row>
<entry>1</entry>
<entry>1</entry>
<entry>1</entry>
<entry>2008-01-01 21:00:23.571</entry>
<entry>2008-01-01 21:00:23.571</entry>
<entry>2008-01-01 21:30:17.132</entry>
<entry>2008-01-01 21:30:17.132</entry>
<entry>FAILED</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_STEP_EXECUTION</title>
<entry>FAILED</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_STEP_EXECUTION</title>
<tgroup cols="6">
<tbody>
<row>
<entry>STEP_EXECUTION_ID</entry>
<tgroup cols="6">
<tbody>
<row>
<entry>STEP_EXECUTION_ID</entry>
<entry>JOB_EXECUTION_ID</entry>
<entry>JOB_EXECUTION_ID</entry>
<entry>STEP_NAME</entry>
<entry>STEP_NAME</entry>
<entry>START_TIME</entry>
<entry>START_TIME</entry>
<entry>END_TIME</entry>
<entry>END_TIME</entry>
<entry>STATUS</entry>
</row>
<entry>STATUS</entry>
</row>
<row>
<entry>1</entry>
<row>
<entry>1</entry>
<entry>1</entry>
<entry>1</entry>
<entry>loadDate</entry>
<entry>loadDate</entry>
<entry>2008-01-01 21:00:23.571</entry>
<entry>2008-01-01 21:00:23.571</entry>
<entry>2008-01-01 21:30:17.132</entry>
<entry>2008-01-01 21:30:17.132</entry>
<entry>FAILED</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_EXECUTION_CONTEXT</title>
<entry>FAILED</entry>
</row>
</tbody>
</tgroup>
</table><table>
<title>BATCH_STEP_EXECUTION_CONTEXT</title>
<tgroup cols="4">
<tbody>
<row>
<entry>EXECUTION_ID</entry>
<tgroup cols="2">
<tbody>
<row>
<entry>STEP_EXECUTION_ID</entry>
<entry>TYPE_CD</entry>
<entry>SHORT_CONTEXT</entry>
</row>
<entry>KEY_NAME</entry>
<row>
<entry>1</entry>
<entry>LONG_VAL</entry>
</row>
<entry>{piece.count=40321}</entry>
</row>
</tbody>
</tgroup>
</table>In this case, the <classname>Step</classname> ran for 30 minutes
and processed 40,321 'pieces', which would represent lines in a file in
this scenario. This value will be updated just before each commit by the
framework, and can contain multiple rows corresponding to entries within
the <classname>ExecutionContext</classname>. Being notified before a
commit requires one of the various StepListeners, or an
<classname>ItemStream</classname>, which are discussed in more detail
later in this guide. As with the previous example, it is assumed that the
Job is restarted the next day. When it is restarted, the values from the
<classname>ExecutionContext</classname> of the last run are reconstituted
from the database, and when the <classname>ItemReader</classname> is
opened, it can check to see if it has any stored state in the context, and
initialize itself from there:</para>
<row>
<entry>1</entry>
<entry>LONG</entry>
<entry>piece.count</entry>
<entry>40321</entry>
</row>
</tbody>
</tgroup>
</table>In this case, the <classname>Step</classname> ran for 30
minutes and processed 40,321 'pieces', which would represent lines in a
file in this scenario. This value will be updated just before each
commit by the framework, and can contain multiple rows corresponding to
entries within the <classname>ExecutionContext</classname>. Being
notified before a commit requires one of the various StepListeners, or
an <classname>ItemStream</classname>, which are discussed in more detail
later in this guide. As with the previous example, it is assumed that
the Job is restarted the next day. When it is restarted, the values from
the <classname>ExecutionContext</classname> of the last run are
reconstituted from the database, and when the
<classname>ItemReader</classname> is opened, it can check to see if it
has any stored state in the context, and initialize itself from
there:</para>
<programlisting> if (executionContext.containsKey(getKey(LINES_READ_COUNT))) {
<programlisting> if (executionContext.containsKey(getKey(LINES_READ_COUNT))) {
log.debug("Initializing for restart. Restart data is: " + executionContext);
long lineCount = executionContext.getLong(getKey(LINES_READ_COUNT));
@@ -940,70 +864,75 @@
}
}</programlisting>
<para>In this case, after the above code is executed, the current line
will be 40,322, allowing the <classname>Step</classname> to start again
from where it left off. The <classname>ExecutionContext</classname> can
also be used for statistics that need to be persisted about the run
itself. For example, if a flat file contains orders for processing that
exist across multiple lines, it may be necessary to store how many
orders have been processed (which is much different from than the number
of lines read) so that an email can be sent at the end of the
<classname>Step</classname> with the total orders processed in the body.
The framework handles storing this for the developer, in order to
correctly scope it with an individual
<classname>JobInstance</classname>. It can be very difficult to know
whether an existing <classname>ExecutionContext</classname> should be
used or not. For example, using the 'EndOfDay' example from above, when
the 01-01 run starts again for the second time, the framework recognizes
that it is the same <classname>JobInstance</classname> and on an
individual <classname>Step</classname> basis, pulls the
<classname>ExecutionContext</classname> out of the database and hands it
as part of the <classname>StepExecution</classname> to the
<classname>Step</classname> itself. Conversely, for the 01-02 run the
framework recognizes that it is a different instance, so an empty
context must be handed to the <classname>Step</classname>. There are
many of these types of determinations that the framework makes for the
developer to ensure the state is given to them at the correct time. It
is also important to note that exactly one
<classname>ExecutionContext</classname> exists per
<classname>StepExecution</classname> at any given time. Clients of the
<classname>ExecutionContext</classname> should be careful because this
creates a shared keyspace, so care should be taken when putting values
in to ensure no data is overwritten, however, the
<classname>Step</classname> stores absolutely no data in the context, so
there is no way to adversely affect the framework.</para>
</section>
<para>In this case, after the above code is executed, the current line
will be 40,322, allowing the <classname>Step</classname> to start again
from where it left off. The <classname>ExecutionContext</classname> can
also be used for statistics that need to be persisted about the run
itself. For example, if a flat file contains orders for processing that
exist across multiple lines, it may be necessary to store how many orders
have been processed (which is much different from than the number of lines
read) so that an email can be sent at the end of the
<classname>Step</classname> with the total orders processed in the body.
The framework handles storing this for the developer, in order to
correctly scope it with an individual <classname>JobInstance</classname>.
It can be very difficult to know whether an existing
<classname>ExecutionContext</classname> should be used or not. For
example, using the 'EndOfDay' example from above, when the 01-01 run
starts again for the second time, the framework recognizes that it is the
same <classname>JobInstance</classname> and on an individual
<classname>Step</classname> basis, pulls the
<classname>ExecutionContext</classname> out of the database and hands it
as part of the <classname>StepExecution</classname> to the
<classname>Step</classname> itself. Conversely, for the 01-02 run the
framework recognizes that it is a different instance, so an empty context
must be handed to the <classname>Step</classname>. There are many of these
types of determinations that the framework makes for the developer to
ensure the state is given to them at the correct time. It is also
important to note that exactly one <classname>ExecutionContext</classname>
exists per <classname>StepExecution</classname> at any given time. Clients
of the <classname>ExecutionContext</classname> should be careful because
this creates a shared keyspace, so care should be taken when putting
values in to ensure no data is overwritten, however, the
<classname>Step</classname> stores absolutely no data in the context, so
there is no way to adversely affect the framework.</para>
<para>It is also important to note that there is at least one
<classname>ExecutionContext</classname> per
<classname>JobExecution</classname>, and one for every
<classname>StepExecution</classname>. For example, consider the following
code snippet:</para>
<programlisting>
ExecutionContext ecStep = stepExecution.getExecutionContext();
ExecutionContext ecJob = jobExecution.getExecutionContext();
//ecStep does not equal ecJob
</programlisting>
<para>As noted in the comment, ecStep will not equal ecJob, they are two
different <classname>ExecutionContext</classname>s. The one scoped to the
<classname>Step</classname> will be saved at every commit point in the
<classname>Step</classname>, wheras the one scoped to the
<classname>Job</classname> will be saved in between every
<classname>Step</classname> execution.</para>
</section>
<section>
<title>JobRepository</title>
<para><classname>JobRepository</classname> is the persistence mechanism
for all of the Stereotypes mentioned above. When a job is first launched,
a <classname>JobExecution</classname> is obtained by calling the
repository's <methodname>createJobExecution</methodname> method, and
during the course of execution, <classname>StepExecution</classname> and
<classname>JobExecution</classname> are persisted by passing them to the
repository:</para>
for all of the Stereotypes mentioned above. It provides CRUD operations
for <classname>JobLauncher</classname>, <classname>Job</classname>, and
<classname>Step</classname> implementations. When a
<classname>Job</classname> is first launched, a
<classname>JobExecution</classname> is obtained from the repository, and
during the course of execution <classname>StepExecution</classname> and
<classname>JobExecution</classname> implementations are persisted by
passing them to the repository:</para>
<programlisting> public interface JobRepository {
<programlisting>
&lt;job-repository id="jobRepository"/&gt;
public JobExecution createJobExecution(Job job, JobParameters jobParameters)
throws JobExecutionAlreadyRunningException, JobRestartException;
void add(StepExecution stepExecution);
void update(JobExecution jobExecution);
void update(StepExecution stepExecution);
void updateExecutionContext(StepExecution stepExecution);
StepExecution getLastStepExecution(JobInstance jobInstance, Step step);
int getStepExecutionCount(JobInstance jobInstance, Step step);
}
</programlisting>
</section>
@@ -1027,25 +956,6 @@
<classname>Job</classname>.</para>
</section>
<section>
<title>JobLocator</title>
<para><classname>JobLocator</classname> represents an interface for
locating a <classname>Job</classname>:</para>
<programlisting> public interface JobLocator {
Job getJob(String name) throws NoSuchJobException;
}</programlisting>
<para>This interface is very necessary due to the nature of Spring itself.
Because it can't be guaranteed that one
<classname>ApplicationContext</classname> equals one
<classname>Job</classname>, an abstraction is needed to obtain a
<classname>Job</classname> for a given name. It becomes especially useful
when launching jobs from within a Java EE application server.</para>
</section>
<section>
<title id="s.5.1.1">Item Reader</title>
@@ -1080,15 +990,4 @@
that it's not valid, returning null indicates that it should not be
written out.</para>
</section>
<section>
<title id="s.2.1.6">Tasklet</title>
<para>A <classname>Tasklet</classname> represents the execution of a
logical unit of work, as defined by its implementation of the Spring Batch
provided <classname>Tasklet</classname> interface. A
<classname>Tasklet</classname> is useful for encapsulating processing
logic that is not natural to split into read-(transform)-write phases,
such as invoking a system command or a stored procedure.</para>
</section>
</chapter>

43
docs/src/site/docbook/reference/execution.xml
View File

@@ -13,8 +13,7 @@
<mediaobject>
<imageobject role="html">
<imagedata align="center"
fileref="images/spring-batch-reference-model.png"
width="75%" />
fileref="images/spring-batch-reference-model.png" />
</imageobject>
<imageobject role="fo">
@@ -457,8 +456,8 @@
&lt;/bean&gt;</programlisting>
<para>The databaseType property indicates the type of incrementer that
must be used. Options include: "db2", "db2zos", "derby", "hsql", "mysql",
"oracle", and "postgres".</para>
must be used. Options include: "db2", "db2zos", "derby", "hsql",
"mysql", "oracle", and "postgres".</para>
</section>
<section>
@@ -1152,9 +1151,9 @@
&lt;property name="retryableExceptionClasses" value="org.springframework.dao.DeadlockLoserDataAccessException" /&gt;</emphasis>
&lt;/bean&gt;</programlisting>
<para>The <classname>SkipLimitStepFactoryBean</classname> requires
a limit for the number of times an individual item can be retried, and
a list of Exceptions that are 'retryable'.</para>
<para>The <classname>SkipLimitStepFactoryBean</classname> requires a
limit for the number of times an individual item can be retried, and a
list of Exceptions that are 'retryable'.</para>
</section>
<section>
@@ -1421,11 +1420,11 @@
&lt;property name="jobRepository" ref="repository" /&gt;
&lt;/bean&gt;</programlisting>
<note>
<para>TaskletStep will automatically register the tasklet as
<classname>StepExecutionListener</classname> if it implements
this interface</para>
</note>
<note>
<para>TaskletStep will automatically register the tasklet as
<classname>StepExecutionListener</classname> if it implements this
interface</para>
</note>
<section>
<title>TaskletAdapter</title>
@@ -1565,16 +1564,14 @@
<section>
<title>Logging Item Processing and Failures</title>
<para>A common use case is the need for special handling of
errors in a step, item by item, perhaps logging to a special
channel, or inserting a record into a
database. The <classname>StepHandlerStep</classname> (created
from the step factory beans) allows users to implement this use
case with a simple
<classname>ItemReadListener</classname>, for errors on read, and an
<classname>ItemWriteListener</classname>, for errors on write. The below
code snippets illustrate a listener that logs both read and write
failures:</para>
<para>A common use case is the need for special handling of errors in a
step, item by item, perhaps logging to a special channel, or inserting a
record into a database. The <classname>StepHandlerStep</classname>
(created from the step factory beans) allows users to implement this use
case with a simple <classname>ItemReadListener</classname>, for errors
on read, and an <classname>ItemWriteListener</classname>, for errors on
write. The below code snippets illustrate a listener that logs both read
and write failures:</para>
<programlisting>public class ItemFailureLoggerListener extends ItemListenerSupport {
@@ -1771,4 +1768,4 @@
itself.</para>
</section>
</section>
</chapter>
</chapter>

BIN
docs/src/site/docbook/reference/images/chunk-oriented-processing.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
docs/src/site/docbook/reference/images/conditional-flow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
docs/src/site/docbook/reference/images/job-heirarchy.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 29 KiB

BIN
docs/src/site/docbook/reference/images/job-launcher-sequence-async.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 41 KiB

BIN
docs/src/site/docbook/reference/images/job-launcher-sequence-sync.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 34 KiB

BIN
docs/src/site/docbook/reference/images/job-stereotypes-parameters.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
docs/src/site/docbook/reference/images/jobHeirarchyWithSteps.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 25 KiB

BIN
docs/src/site/docbook/reference/images/sequential-flow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
docs/src/site/docbook/reference/images/spring-batch-layers.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.5 KiB

After

Width:  |  Height:  |  Size: 18 KiB

BIN
docs/src/site/docbook/reference/images/spring-batch-reference-model.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

BIN
docs/src/site/docbook/reference/images/step.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

10
docs/src/site/docbook/reference/index.xml
View File

@@ -48,17 +48,21 @@
<xi:include href="spring-batch-intro.xml" />
<xi:include href="core.xml" />
<xi:include href="domain.xml" />
<xi:include href="job.xml" />
<xi:include href="step.xml" />
<xi:include href="readersAndWriters.xml" />
<xi:include href="execution.xml" />
<xi:include href="repeat.xml" />
<xi:include href="retry.xml" />
<xi:include href="testing.xml" />
<xi:include href="common-patterns.xml" />
<xi:include href="appendix.xml" />

990
docs/src/site/docbook/reference/job.xml
View File

File diff suppressed because it is too large Load Diff

11
docs/src/site/docbook/reference/namespace.xml
View File

@@ -1,11 +0,0 @@
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="namespace">
<title>Namespace Support</title>
<section>
<title id="ns.1">Namespace Support</title>
<para>Document Namespace support here. </para>
</section>
</chapter>

62
docs/src/site/docbook/reference/outline.xml
View File

@@ -1,62 +0,0 @@
<?xml version='1.0'?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" "http://www.oasis-open.org/docbook/xml/4.0/docbookx.dtd">
<book lang="en">
<bookinfo>
<title>The Spring Batch - Reference Documentation</title>
<corpauthor>Wayne Lund, Waseem Malik, Lucas Ward, Scott Wintermute,
Kerry O&apos;Brien, Tomi Vanek</corpauthor>
<date>May 2007</date>
</bookinfo>
<section>
<title id="c.1">Chapter 1: Spring Batch Introduction</title>
<para><ulink url="spring-batch-intro.html">Overview of the Spring Batch Architecture</ulink> - the Spring Batch Reference Model</para>
</section>
<section>
<title id="c.2"><ulink url="infrastructure.html">Chapter 2: The Spring Batch Infrastructure</ulink></title>
<para>Infrastructure covers Repeat Template, I/O facilities and the RetryTemplate</para>
</section>
<section>
<title id="c.3"><ulink url="core.html">Chapter 3: Spring Batch Core</ulink></title>
<para>Describe the domain language of batch and how the pieces fit together.</para>
</section>
<section>
<title id="c.4"><ulink url="execution.html">Chapter 4: Spring Batch Execution</ulink></title>
<para>Describe the simple batch execution environment.</para>
</section>
<section>
<title id="c.5"><ulink url="application.html">Chapter 5: Spring Batch Applications</ulink></title>
<para>Describe the solution space for spring batch.</para>
</section>
<section>
<title id="c.6"><ulink url="samples.html">Chapter 6: Spring Batch Samples</ulink></title>
<para>The documentation for samples goes here</para>
</section>
<section>
<title id="c.7"><ulink url="batch-job-testing.html">Chapter 7: Unit and Integration Testing Batch Jobs</ulink></title>
<para>The documentation for unit and integration testing of batch jobs goes here.</para>
</section>
<section>
<title id="c.8"><ulink url="batch-performance-testing.html">Chapter 8: Performance Testing Batch Jobs</ulink></title>
<para>How to performance test batch jobs.</para>
</section>
<section>
<title id="9"><ulink url="glossary.html">Chapter 9: Glossary</ulink></title>
<para>(Should this be Appendix A?) The Batch Glossary documents common terms used in the batch processing domain.</para>
</section>
<section>
<title>More sections that may come later?</title>
<para>More advanced info about building Spring Batch, how to contribute, JMS integration, management with JMX, the Spring Batch data model, integration with schedulers (eg quartz)</para>
</section>
</book>

10
docs/src/site/docbook/reference/partitioned-containers.xml
View File

@@ -1,10 +0,0 @@
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<chapter id="Partitioned Containers">
<title>Batch IO Support</title>
<section>
<title id="s.1">Partitioned Containers</title>
<para>Much stuff goes here.</para>
</section>
</chapter>

392
docs/src/site/docbook/reference/spring-batch-intro.xml
View File

@@ -4,209 +4,201 @@
<chapter id="spring-batch-intro">
<title>Spring Batch Introduction</title>
<para>Many applications within the enterprise domain require bulk processing
to perform business operations in mission critical environments. These
business operations include automated, complex processing of large volumes
of information that is most efficiently processed without user interaction.
These operations typically include time based events (e.g. month-end
calculations, notices or correspondence), periodic application of complex
business rules processed repetitively across very large data sets (e.g.
Insurance benefit determination or rate adjustments), or the integration of
information that is received from internal and external systems that
typically requires formatting, validation and processing in a transactional
manner into the system of record. Batch processing is used to process
billions of transactions every day for enterprises.</para>
<para>Spring Batch is a lightweight, comprehensive batch framework designed
to enable the development of robust batch applications vital for the daily
operations of enterprise systems. Spring Batch builds upon the productivity,
POJO-based development approach, and general ease of use capabilities people
have come to know from the Spring Framework, while making it easy for
developers to access and leverage more advance enterprise services when
necessary. Spring Batch is not a scheduling framework. There are many good
enterprise schedulers available in both the commercial and open source
spaces such as Quartz, Tivoli, Control-M, etc. It is intended to work in
conjunction with a scheduler, not replace a scheduler.</para>
<para>Spring Batch provides reusable functions that are essential in
processing large volumes of records, including logging/tracing, transaction
management, job processing statistics, job restart, skip, and resource
management. It also provides more advance technical services and features
that will enable extremely high-volume and high performance batch jobs
though optimization and partitioning techniques. Simple as well as complex,
high-volume batch jobs can leverage the framework in a highly scalable
manner to process significant volumes of information.</para>
<section>
<title id="s.1">Introduction</title>
<title id="s.1.4">Background</title>
<para>Many applications within the enterprise domain require bulk
processing to perform business operations in mission critical
environments. These business operations include automated, complex
processing of large volumes of information that is most efficiently
processed without user interaction. These operations typically include
time based events (e.g. month-end calculations, notices or
correspondence), periodic application of complex business rules processed
repetitively across very large data sets (e.g. Insurance benefit
determination or rate adjustments), or the integration of information that
is received from internal and external systems that typically requires
formatting, validation and processing in a transactional manner into the
system of record. Batch processing is used to process billions of
transactions every day for enterprises.</para>
<para>While open source software projects and associated communities have
focused greater attention on web-based and SOA messaging-based
architecture frameworks, there has been a notable lack of focus on
reusable architecture frameworks to accommodate Java-based batch
processing needs, despite continued needs to handle such processing within
enterprise IT environments. The lack of a standard, reusable batch
architecture has resulted in the proliferation of many one-off, in-house
solutions developed within client enterprise IT functions.</para>
<para>Spring Batch is a lightweight, comprehensive batch framework
designed to enable the development of robust batch applications vital for
the daily operations of enterprise systems. Spring Batch builds upon the
productivity, POJO-based development approach, and general ease of use
capabilities people have come to know from the Spring Framework, while
making it easy for developers to access and leverage more advance
enterprise services when necessary. Spring Batch is not a scheduling
framework. There are many good enterprise schedulers available in both the
commercial and open source spaces such as Quartz, Tivoli, Control-M, etc.
It is intended to work in conjunction with a scheduler, not replace a
scheduler.</para>
<para>SpringSource and Accenture have collaborated to change this.
Accenture's hands-on industry and technical experience in implementing
batch architectures, SpringSource's depth of technical experience, and
Spring's proven programming model together mark a natural and powerful
partnership to create high-quality, market relevant software aimed at
filling an important gap in enterprise Java. Both companies are also
currently working with a number of clients solving similar problems
developing Spring-based batch architecture solutions. This has provided
some useful additional detail and real-life constraints helping to ensure
the solution can be applied to the real-world problems posed by clients.
For these reasons and many more, SpringSource and Accenture have teamed to
collaborate on the development of Spring Batch.</para>
<para>Spring Batch provides reusable functions that are essential in
processing large volumes of records, including logging/tracing,
transaction management, job processing statistics, job restart, skip, and
resource management. It also provides more advance technical services and
features that will enable extremely high-volume and high performance batch
jobs though optimization and partitioning techniques. Simple as well as
complex, high-volume batch jobs can leverage the framework in a highly
scalable manner to process significant volumes of information.</para>
<para>Accenture has contributed previously proprietary batch processing
architecture frameworks, based upon decades worth of experience in
building batch architectures with the last several generations of
platforms, (i.e., COBOL/Mainframe, C++/Unix, and now Java/anywhere) to the
Spring Batch project along with committer resources to drive support,
enhancements, and the future roadmap.</para>
<section>
<title id="s.1.4">Background</title>
<para>While open source software projects and associated communities
have focused greater attention on web-based and SOA messaging-based
architecture frameworks, there has been a notable lack of focus on
reusable architecture frameworks to accommodate Java-based batch
processing needs, despite continued needs to handle such processing
within enterprise IT environments. The lack of a standard, reusable
batch architecture has resulted in the proliferation of many one-off,
in-house solutions developed within client enterprise IT
functions.</para>
<para>SpringSource and Accenture have collaborated to change this.
Accenture's hands-on industry and technical experience in implementing
batch architectures, SpringSource's depth of technical experience, and
Spring's proven programming model together mark a natural and powerful
partnership to create high-quality, market relevant software aimed at
filling an important gap in enterprise Java. Both companies are also
currently working with a number of clients solving similar problems
developing Spring-based batch architecture solutions. This has provided
some useful additional detail and real-life constraints helping to
ensure the solution can be applied to the real-world problems posed by
clients. For these reasons and many more, SpringSource and Accenture
have teamed to collaborate on the development of Spring Batch.</para>
<para>Accenture has contributed previously proprietary batch processing
architecture frameworks, based upon decades worth of experience in
building batch architectures with the last several generations of
platforms, (i.e., COBOL/Mainframe, C++/Unix, and now Java/anywhere) to
the Spring Batch project along with committer resources to drive
support, enhancements, and the future roadmap.</para>
<para>The collaborative effort between Accenture and SpringSource aims
to promote the standardization of software processing approaches,
frameworks, and tools that can be consistently leveraged by enterprise
users when creating batch applications. Companies and government
agencies desiring to deliver standard, proven solutions to their
enterprise IT environments will benefit from Spring Batch.</para>
</section>
<section>
<title id="s-b-i.1.2">Usage Scenarios</title>
<para>A typical batch program generally reads a large number of records
from a database, file, or queue, processes the data in some fashion, and
then writes back data in a modified form. Spring Batch automates this
basic batch iteration, providing the capability to process similar
transactions as a set, typically in an offline environment without any
user interaction. Batch jobs are part of most IT projects and Spring
Batch is the only open source framework that provides a robust,
enterprise-scale solution.</para>
<para>Business Scenarios <itemizedlist>
<listitem>
<para>Commit batch process periodically</para>
</listitem>
<listitem>
<para>Concurrent batch processing: parallel processing of a
job</para>
</listitem>
<listitem>
<para>Staged, enterprise message-driven processing</para>
</listitem>
<listitem>
<para>Massively parallel batch processing</para>
</listitem>
<listitem>
<para>Manual or scheduled restart after failure</para>
</listitem>
<listitem>
<para>Sequential processing of dependent steps (with extensions to
workflow-driven batches)</para>
</listitem>
<listitem>
<para>Partial processing: skip records (e.g. on rollback)</para>
</listitem>
<listitem>
<para>Whole-batch transaction: for cases with a small batch size
or existing stored procedures/scripts</para>
</listitem>
</itemizedlist></para>
<para>Technical Objectives <itemizedlist>
<listitem>
<para>Batch developers use the Spring programming model:
concentrate on business logic; let the framework take care of
infrastructure.</para>
</listitem>
<listitem>
<para>Clear separation of concerns between the infrastructure, the
batch execution environment, and the batch application.</para>
</listitem>
<listitem>
<para>Provide common, core execution services as interfaces that
all projects can implement.</para>
</listitem>
<listitem>
<para>Provide simple and default implementations of the core
execution interfaces that can be used ‘out of the box’.</para>
</listitem>
<listitem>
<para>Easy to configure, customize, and extend services, by
leveraging the spring framework in all layers.</para>
</listitem>
<listitem>
<para>All existing core services should be easy to replace or
extend, without any impact to the infrastructure layer.</para>
</listitem>
<listitem>
<para>Provide a simple deployment model, with the architecture
JARs completely separate from the application, built using
Maven.</para>
</listitem>
</itemizedlist></para>
</section>
<section>
<title id="s.1.1">Spring Batch Architecture</title>
<para>Spring Batch is designed with extensibility and a diverse group of
end users in mind. The figure below shows a sketch of the layered
architecture that supports the extensibility and ease of use for
end-user developers. <mediaobject>
<imageobject role="fo">
<imagedata align="center"
fileref="src/site/docbook/reference/images/spring-batch-layers.png"
format="PNG" />
</imageobject>
<imageobject role="html">
<imagedata align="center" fileref="images/spring-batch-layers.png"
format="PNG" />
</imageobject>
<caption><para>Figure 1.1: Spring Batch Layered
Architecture</para></caption>
</mediaobject></para>
<para>This layered architecture highlights three major high level
components: Application, Core, and Infrastructure. The application
contains all batch jobs and custom code written by developers using
Spring Batch. The Batch Core contains the core runtime classes necessary
to launch and control a batch job. It includes things such as a
<classname>JobLauncher</classname>, <classname>Job</classname>, and
<classname>Step</classname> implementations. Both Application and Core
are built on top of a common infrastructure. This infrastructure
contains common readers and writers, and services such as the
<classname>RetryTemplate</classname>, which are used both by application
developers(<classname>ItemReader</classname> and
<classname>ItemWriter</classname>) and the core framework itself.
(retry)</para>
</section>
<para>The collaborative effort between Accenture and SpringSource aims to
promote the standardization of software processing approaches, frameworks,
and tools that can be consistently leveraged by enterprise users when
creating batch applications. Companies and government agencies desiring to
deliver standard, proven solutions to their enterprise IT environments
will benefit from Spring Batch.</para>
</section>
</chapter>
<section>
<title id="s-b-i.1.2">Usage Scenarios</title>
<para>A typical batch program generally reads a large number of records
from a database, file, or queue, processes the data in some fashion, and
then writes back data in a modified form. Spring Batch automates this
basic batch iteration, providing the capability to process similar
transactions as a set, typically in an offline environment without any
user interaction. Batch jobs are part of most IT projects and Spring Batch
is the only open source framework that provides a robust, enterprise-scale
solution.</para>
<para>Business Scenarios <itemizedlist>
<listitem>
<para>Commit batch process periodically</para>
</listitem>
<listitem>
<para>Concurrent batch processing: parallel processing of a
job</para>
</listitem>
<listitem>
<para>Staged, enterprise message-driven processing</para>
</listitem>
<listitem>
<para>Massively parallel batch processing</para>
</listitem>
<listitem>
<para>Manual or scheduled restart after failure</para>
</listitem>
<listitem>
<para>Sequential processing of dependent steps (with extensions to
workflow-driven batches)</para>
</listitem>
<listitem>
<para>Partial processing: skip records (e.g. on rollback)</para>
</listitem>
<listitem>
<para>Whole-batch transaction: for cases with a small batch size or
existing stored procedures/scripts</para>
</listitem>
</itemizedlist></para>
<para>Technical Objectives <itemizedlist>
<listitem>
<para>Batch developers use the Spring programming model: concentrate
on business logic; let the framework take care of
infrastructure.</para>
</listitem>
<listitem>
<para>Clear separation of concerns between the infrastructure, the
batch execution environment, and the batch application.</para>
</listitem>
<listitem>
<para>Provide common, core execution services as interfaces that all
projects can implement.</para>
</listitem>
<listitem>
<para>Provide simple and default implementations of the core
execution interfaces that can be used ‘out of the box’.</para>
</listitem>
<listitem>
<para>Easy to configure, customize, and extend services, by
leveraging the spring framework in all layers.</para>
</listitem>
<listitem>
<para>All existing core services should be easy to replace or
extend, without any impact to the infrastructure layer.</para>
</listitem>
<listitem>
<para>Provide a simple deployment model, with the architecture JARs
completely separate from the application, built using Maven.</para>
</listitem>
</itemizedlist></para>
</section>
<section>
<title id="s.1.1">Spring Batch Architecture</title>
<para>Spring Batch is designed with extensibility and a diverse group of
end users in mind. The figure below shows a sketch of the layered
architecture that supports the extensibility and ease of use for end-user
developers. <mediaobject>
<imageobject role="fo">
<imagedata align="center" contentdepth="" contentwidth=""
fileref="src/site/docbook/reference/images/spring-batch-layers.png"
format="PNG" scalefit="" width="338" />
</imageobject>
<imageobject role="html">
<imagedata align="center" fileref="images/spring-batch-layers.png"
format="PNG" scalefit="" width="35%" />
</imageobject>
<caption><para>Figure 1.1: Spring Batch Layered
Architecture</para></caption>
</mediaobject></para>
<para>This layered architecture highlights three major high level
components: Application, Core, and Infrastructure. The application
contains all batch jobs and custom code written by developers using Spring
Batch. The Batch Core contains the core runtime classes necessary to
launch and control a batch job. It includes things such as a
<classname>JobLauncher</classname>, <classname>Job</classname>, and
<classname>Step</classname> implementations. Both Application and Core are
built on top of a common infrastructure. This infrastructure contains
common readers and writers, and services such as the
<classname>RetryTemplate</classname>, which are used both by application
developers(<classname>ItemReader</classname> and
<classname>ItemWriter</classname>) and the core framework itself.
(retry)</para>
</section>
</chapter>

1922
docs/src/site/docbook/reference/step.xml
View File

File diff suppressed because it is too large Load Diff