Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Browse Source
This commit is contained in:
wxlund
2008-02-13 01:49:18 +00:00
parent 2197107100
commit 4a7e796045
2 changed files with 355 additions and 160 deletions
Download Patch File Download Diff File Expand all files Collapse all files

513
docs/src/site/docbook/reference/core.xml
View File

@@ -177,175 +177,370 @@
<para>Physical Resources in the Data Tier that are the source and
target of ItemReaders and Writers like Message Queues, Databases,
Files and Print Queues.</para>
</listitem>
</itemizedlist>
<section>
<title id="s.2.1">Batch Domain Stereotypes</title>
<para>We will discuss each of these Batch domain Stereotypes
individually. This section describes stereotypes relating to the concept of a batch job.
A job is an entity that encapsulates an entire batch process.</para>
<section>
<title id="s.2.1.1">Job Configuration</title>
<para>
The job configuration could be described as the heart of the Spring Batch framework.
It is represented by a Spring bean of class _JobConfiguration_ and contains all of
the information necessary to define the operations performed by a job. A job configuration
is typically contained within a Spring XML configuration file and the job's name is
determined by the "id" attribute associated with the job configuration bean. The job configuration contains:
</para>
<itemizedlist>
<listitem>
<para> The simple name of the job</para>
</listitem>
<listitem>
<para>Definition and ordering of [Step Configurations|#Step Configuration]</para>
</listitem>
<listitem>
<para>The limit of how many times this job may be started</para>
</listitem>
<listitem>
<para>Whether or not the job is restartable</para>
</listitem>
</itemizedlist>
The mechanics of defining a job configuration will be discussed in the next chapter. [Provide a link]
</section>
<section>
<title id="s.2.1.2">Job Instance</title>
<para>A job instance refers to the business concept of a single job invocation. In other words, suppose
you have a job called "foo" that is run three times a day. There will be one "foo" configuration,
and each time "foo" is supposed to run would be an instance of the "foo" job. Each instance would be
uniquely identified as each one represents a distinct batch need. Further, each instance might
have attempted several times to complete its work. Each attempt is represented by a [#Job Execution],
described below. A job instance is not considered to be complete until an associated job execution
completes successfully. As such, a single job instance may have many executions.
</para>
<para>For example, a unique instance might be identified by just a job name, or by the combination of a
job name and a scheduled date. Using this second type of identification, we might have two distinct
instances, "foo-01-01-2008" and "foo-01-02-2008." Although these two instances would share the same
configuration, they would each have their own set of executions and the successful completion of one
instance would not affect the status of the other.
</para>
<para>Job instances are represented by objects of the _JobInstance_ class, which are created when the
job is executed. Each job instance contains references to related [Step Instances|#Step Instance]
and a _JobIdentifier_ that uniquely identifies this job instance.
</para>
</section>
<section>
<title id="s.2.1.3">Job Execution</title>
<para>
A job execution refers to the technical concept of a single attempt to run a job. It is a single attempt to execute the logic represented by a job instance. A job execution may end in failure or success, but the job instance corresponding to a given execution will not be marked as complete unless the execution completes successfully.
</para>
<para>
For instance, if we have a job instance "foo-01-01-2008" that fails to successfully complete its work the first time it is run, when we attempt to run it again, a new job execution will be created. If our "foo" configuration is restartable, we may begin our second job execution from a restart point. Otherwise, our job execution will start from the beginning. In either case, we will see that our single job instance has had two job executions.
</para>
<para>
Job executions are represented by objects of the _JobExecution_ class. These job executions are created by an implementation of the _JobExecutorFacade_ interface from a given _JobInstance_ corresponding to a unique _JobIdentifier_. Each job execution contains a reference to its corresponding job instance, related Step Executions and step/chunk context data.
</para>
</section>
<section>
<title id="s.2.1.4">Step Stereotypes</title>
<para>
This section describes stereotypes relating to the concept of a batch step. A step is an entity that encapsulates a single, independent phase of a batch job. Therefore, every batch job is composed entirely of one or more batch steps.
</para>
</section>
<section>
<title id="s.2.1.5">Step Configuration</title>
<para>
The step configuration contains all of the information necessary to define a discrete set of business logic within a job configuration. This is a necessarily vague description because the contents of any given step configuration are at the discretion of the developer writing your jobs. A step can be as narrowly defined as a single line of code or as broadly defined as necessary to complete the entire work of your job. There are several factors that will affect the breadth of your step configurations.
</para>
<itemizedlist>
<listitem>
<para>Re-usability - step definitions can be shared between jobs</para>
</listitem>
<listitem>
<para>Transaction Management - depending on your desired transaction strategy, you may divide the work of your job differently between steps</para>
</listitem>
<listitem>
<para>Extensibility - adequately granular definition of steps allows the addition or subtraction of steps at a later time in the appropriate position within your job configuration</para>
</listitem>
</itemizedlist>
<para>
Step configurations are defined by instantiating implementations of the _StepConfiguration_ interface. Additionally, the utility class _StepConfigurationSupport_ provides a basic implementation of _StepConfiguration_ with default functionality that should be common to any concrete _StepConfiguration_ implementation. Generally, all step configuration implementations should extend from this class.
</para>
<para>
Two step configuration classes are available in the Spring Batch framework, and they are each discussed in detail in other sections of this guide. For most situations, the _SimpleStepConfiguration_ implementation is sufficient, but custom transaction management behavior can also be configured by using a _RepeatOperationsStepConfiguration_.
</para>
</section>
<section>
<title id="s.2.1.6">Step Instance</title>
<para>
A step instance, represented by the _StepInstance_ class, represents the business concept of a single step within a job invocation. That is to say, every job instance contains one or more step instances.
</para>
<para>
For example, suppose we have a job instance called "foo-01-01-2008" that is an instance of a job configuration containing three steps. Suppose these steps are named "step1", "step2" and "step3." There will be corresponding "foo-01-01-2008#step1", "foo-01-01-2008#step2" and "foo-01-01-2008#step3" step instances, which will be distinct from the step instances of any other job instance (e.g. those of "foo-01-02-2008").
{note}The step instance naming here is for clarity, this is not necessarily how the instance will be named internally within the framework.{note}
</para>
<para>
Each step instance will contain the current status of the batch execution, restart data, and a reference to its corresponding [Job Instance]. Additionally, a step instance keeps track of how many attempts are made to run the corresponding step. Each attempt to run a step will create a [Step Execution], so a single job instance might have several corresponding step executions.
</para>
</section>
<section>
<title id="s.2.1.7">Step Execution</title>
<para>
A step execution represents the technical concept of a single attempt to execute a step. It is a single attempt to execute the logic represented by a step instance.
</para>
<para>
For instance, if we have a step instance "foo-01-01-2008#step1" that fails to successfully complete its work the first time it is run, when we attempt to run it again, a new step execution will be created. Each of these step executions may represent a different invocation of the batch framework, but they will all correspond to the same step instance.
</para>
<para>
Step executions are represented by objects of the _StepExecution_ class. These step executions are created by an implementation of the _JobExecutor_ interface from a given _StepInstance_ and _JobExecution_. Each step execution contains a reference to its corresponding step instance and job execution, and transaction related data such as commit and rollback counts, start and end times and a _Properties_ instance containing statistics.
</para>
</section>
<section>
<title id="s.2.1.8">Tasklets</title>
<para>
A tasklet represents the execution of a logical unit of work, as defined by its implementation of the Spring Batch provided _Tasklet_ interface. Tasklets are used when defining step configurations to specify the work done by the step. Subsequently, the logic in a tasklet is atomic in terms of transactions. A transaction will never commit until an entire tasklet execution is complete (unless an exception occurs - a transaction might either commit or rollback if that behavior is specified in the step's exception management strategy).
</para>
</section>
</section>
<title id="s.2.3">Item-Oriented Processing Stereotypes</title>
<para>
A powerful batch processing paradigm implemented by the Spring Batch framework is the concept of item-oriented processing. That is, doing work by defining each unit of work as the operation of retrieving an item from input and then processing that item, including any side effects that processing might entail, such as file or database operations.
</para>
<para>
There are two basic stereotypes that represent the first-class participants in item-oriented processing, item providers and item processors. They are each represented by a simple interface provided by the Spring Batch framework, which allows free reign over their implementations and improves our ability to leverage the Spring framework's dependency injection capabilities.
</para>
<section>
<title id="s.2.3.1">Item Readers</title>
<para>
An item reader is an object that is used to retrieve the inputs for a step, one at a time. When the item reader has exhausted the items it can provide, it will indicate this in a meaningful way (generally by returning _null_). When coupled with an item processor, this forms a complete item-oriented process, as each item taken from the provider is then processed by the processor.
</para>
</section>
<section>
<title id="s.2.3.2">Item Writers/Processors</title>
<para>
An item processor is an object that is used to perform processing for a step, one item at a time. Generally, an item processor has no knowledge of the input it will receive next, only the item that that was passed in its current invocation. As a result, item processors will generally make no assumptions about the input they receive an treat every item the same way and keep track of its own state between invocations. When coupled with an item provider, this forms a complete item-oriented process, as each item taken from the provider is then processed by the processor.
</para>
</section>
</section>
<section>
<title id="s.2.4">Support Stereotypes</title>
<para>While item providers and processors serve as the main entry points for item-oriented processing, they are supplemented by a number of support classes that perform specific tasks within the provider / processor lifecycle. These support stereotypes are useful for dividing the work of item providers and processors into reusable pieces, as well as abstracting away the details of processing, such as interaction with external systems. Additionally, they give us another opportunity to leverage the powerful configuration features of the Spring framework, as we can switch between several beans implementing these support interfaces without changing the driving item provider or processor.
</para>
<title id="s.2">Job Stereotypes</title>
<para>This section describes stereotypes relating to the concept of a
batch job. A job is an entity that encapsulates an entire batch process.
The file containing the job may sometimes be referred to as the "job
configuration.</para>
<section>
<title id="s.2.4.1">Input Sources</title>
<para> An input source is a class that mediates interactions with an external source of input data, such as a file or a database. An input source often serves as a support for an item provider, typically abstracting away the details of interaction, such as the creation and maintenance of file handles, sockets or database connections.
</para>
<title id="s.2.1.1">Job</title>
<para>The job could be described as the heart of the Spring Batch
framework. It is represented by a Spring bean that implements the
<emphasis role="bold">Job</emphasis> interface and contains all of the
information necessary to define the operations performed by a job. A job
configuration is typically contained within a Spring XML configuration
file and the job's name is determined by the "id" attribute associated
with the job configuration bean. The job configuration contains:</para>
<itemizedlist>
<listitem>
<para>The simple name of the job</para>
</listitem>
<listitem>
<para>Definition and ordering of [Step Configurations|#Step
Configuration]</para>
</listitem>
<listitem>
<para>The limit of how many times this job may be started</para>
</listitem>
<listitem>
<para>Whether or not the job is restartable</para>
</listitem>
</itemizedlist>
A default simple implementation of the
<emphasis role="bold">Job</emphasis>
interface is provided by Spring Batch in the form of the JobSupport class. Jobs can be defined by creating beans from subclasses of
<emphasis role="bold">JobSupport</emphasis>
. The
<emphasis role="bold">JobSupport</emphasis>
class, however, does not provide much in the way of functionality. The provided
<emphasis role="bold">SimpleJob</emphasis>
class creates some standard functionality on top of
<emphasis role="bold">JobSupport</emphasis>
, namely a standard execution logic that all jobs should utilize. In general, all job configurations should be defined using a bean of type
<emphasis role="bold">SimpleJob</emphasis>
.
</section>
<section>
<title id="s.2.1.2">Job Instance</title>
<para>A job instance refers to the business concept of a single job
invocation. In other words, suppose you have a job called "foo" that is
run three times a day. There will be one "foo" configuration, and each
time "foo" is supposed to run would be an instance of the "foo" job.
Each instance would be uniquely identified as each one represents a
distinct batch need. Further, each instance might have attempted several
times to complete its work. Each attempt is represented by a [#Job
Execution], described below. A job instance is not considered to be
complete until an associated job execution completes successfully. As
such, a single job instance may have many executions. To keep track of
this, every job instance provides a reference to the last execution
attempt.</para>
<para>For example, a unique instance might be identified by just a job
name, or by the combination of a job name and a scheduled date. Using
this second type of identification, we might have two distinct
instances, "foo-01-01-2008" and "foo-01-02-2008." Although these two
instances would share the same configuration, they would each have their
own set of executions and the successful completion of one instance
would not affect the status of the other.</para>
<para>Job instances are represented by objects of the <emphasis
role="bold">JobInstance</emphasis> class, which are created when the job
is executed. Each job instance contains references to related [Step
Instances|#Step Instance] and a set of job parameters, represented by
the <emphasis role="bold">JobParameters</emphasis> that uniquely
identifies this job instance.</para>
</section>
<section>
<title id="s.2.1.3">Job Execution</title>
<para>A job execution refers to the technical concept of a single
attempt to run a job. It is a single attempt to execute the logic
represented by a job instance. A job execution may end in failure or
success, but the job instance corresponding to a given execution will
not be marked as complete unless the execution completes
successfully.</para>
<para>For instance, if we have a job instance "foo-01-01-2008" that
fails to successfully complete its work the first time it is run, when
we attempt to run it again, a new job execution will be created. If our
"foo" configuration is restartable, we may begin our second job
execution from a restart point. Otherwise, our job execution will start
from the beginning. In either case, we will see that our single job
instance has had two job executions.</para>
<para>Job executions are represented by objects of the <emphasis
role="bold">JobExecution</emphasis> class. These job executions are
created by an implementation of the <emphasis
role="bold">JobExecutorFacade</emphasis> interface from a given
<emphasis role="bold">JobInstance</emphasis> corresponding to a unique
<emphasis role="bold">JobParameters</emphasis> object. Each job
execution contains a reference to its corresponding job instance,
related Step Executions and step/chunk context data.</para>
</section>
</section>
<section>
<title id="s.2.1">Step Stereotypes</title>
<para>This section describes stereotypes relating to the concept of a
batch step. A step is an entity that encapsulates a single, independent
phase of a batch job. Therefore, every batch job is composed entirely of
one or more batch steps.</para>
<section>
<title id="s.2.1.1">Step Configuration</title>
<para>The step bean contains all of the information necessary to define
a discrete set of business logic within a job configuration. This is a
necessarily vague description because the contents of any given step
configuration are at the discretion of the developer writing jobs. A
step can be as narrowly defined as a single line of code or as broadly
defined as necessary to complete the entire work of your job. There are
several factors that will affect the breadth of your step
configurations.</para>
<itemizedlist>
<listitem>
<para>Re-usability - step definitions can be shared between
jobs</para>
</listitem>
<listitem>
<para>Transaction Management - depending on your desired transaction
strategy, you may divide the work of your job differently between
steps</para>
</listitem>
<listitem>
<para>Extensibility - adequately granular definition of steps allows
the addition or subtraction of steps at a later time in the
appropriate position within your job configuration</para>
</listitem>
</itemizedlist>
<para>Step configurations are defined by instantiating implementations
of the <emphasis role="bold">Step</emphasis> interface. Additionally,
the utility class <emphasis role="bold">StepSupport</emphasis> and its
abstract subclass, <emphasis role="bold">AbstractStep</emphasis> provide
a basic implementation of <emphasis role="bold">Step</emphasis> with
default functionality that should be common to any concrete Step
implementation. Generally, all step configuration implementations should
extend from these classes.</para>
<para>Two step implementation classes are available in the Spring Batch
framework, and they are each discussed in detail in other sections of
this guide. For most situations, the <emphasis
role="bold">SimpleStep</emphasis> implementation is sufficient, but
custom control flow behavior and transaction management behavior can
also be configured by using a <emphasis
role="bold">RepeatOperationsStep</emphasis>.</para>
</section>
<section>
<title id="s.2.1.2">Step Instance</title>
<para>A step instance, represented by the <emphasis
role="bold">StepInstance</emphasis> class, represents the business
concept of a single step within a job invocation. That is to say, every
job instance contains one or more step instances.</para>
<para>For example, suppose we have a job instance called
"foo-01-01-2008" that is an instance of a job configuration containing
three steps. Suppose these steps are named "step1", "step2" and "step3."
There will be corresponding "foo-01-01-2008#step1",
"foo-01-01-2008#step2" and "foo-01-01-2008#step3" step instances, which
will be distinct from the step instances of any other job instance (e.g.
those of "foo-01-02-2008").</para>
<note>
<title />
The step instance naming here is for clarity, this is not necessarily how the instance will be named internally within the framework.
</note>
<para>Each step instance will contain the current status of the batch
execution and a reference to its corresponding <emphasis
role="bold">JobInstance</emphasis>. Additionally, a step instance keeps
track of how many attempts are made to run the corresponding step. Each
attempt to run a step will create a <emphasis
role="bold">StepExecution</emphasis>, so a single job instance might
have several corresponding step executions.</para>
</section>
<section>
<title id="s.2.1.3">Step Execution</title>
<para>A step execution represents the technical concept of a single
attempt to execute a step. It is a single attempt to execute the logic
represented by a step instance.</para>
<para>For instance, if we have a step instance "foo-01-01-2008#step1"
that fails to successfully complete its work the first time it is run,
when we attempt to run it again, a new step execution will be created.
Each of these step executions may represent a different invocation of
the batch framework, but they will all correspond to the same step
instance.</para>
<para>Step executions are represented by objects of the <emphasis
role="bold">StepExecution</emphasis> class. These step executions are
created by an implementation of the <emphasis
role="bold">JobExecutor</emphasis> interface from a given <emphasis
role="bold">StepInstance</emphasis> and <emphasis
role="bold">JobExecution</emphasis>. Each step execution contains a
reference to its corresponding step instance and job execution, and
transaction related data such as commit and rollback count and start and
end times. Additionally, each step execution will contain a set of
execution attributes, which will contain statistics and restart
data.</para>
</section>
<section>
<title id="s.2.1.4">Chunk</title>
<para>
Each step execution is divided into one or more transactions. These
individual transactions are sometimes referred to as "chunks." Each
chunk may represent one or many individual operations within a step
execution, depending on configuration. Although chunk-based processing
is an important feature of Spring Batch, there is no domain object that
directly corresponds to a chunk, as chunks are implicitly created through
the opening and closing of transactions.
</para>
</section>
<section>
<title id="s.2.1.5">Step Contribution</title>
<para>
A step contribution represents the metadata for an uncommitted portion, or chunk,
of a step execution that is being "buffered" until a transaction boundary is reached
and the operations within the chunk are successfully committed. Each step contribution
is created by the enclosing step execution, and each contribution is later "applied"
by the enclosing step executor, allowing metadata to be applied only after each processed
chunk succeeds.
</para>
</section>
<section>
<title id="s.2.1.6">Tasklets</title>
<para>A tasklet represents the execution of a logical unit of work, as
defined by its implementation of the Spring Batch provided _Tasklet_
interface. Tasklets are used when defining step configurations to
specify the work done by the step. Subsequently, the logic in a tasklet
is atomic in terms of transactions. A transaction will never commit
until an entire tasklet execution is complete (unless an exception
occurs - a transaction might either commit or rollback if that behavior
is specified in the step's exception management strategy).</para>
</section>
</section>
<section>
<title id="s.5">Item-Oriented Processing Stereotypes</title>
<para>A powerful batch processing paradigm implemented by the Spring Batch
framework is the concept of item-oriented processing. That is, doing work
by defining each unit of work as the operation of retrieving an item from
input and then processing that item, including any side effects that
processing might entail, such as file or database operations.</para>
<section>
<title id="s.5.1">Reader and Writer Stereotypes</title>
<para>There are two basic stereotypes that represent the first-class
participants in item-oriented processing, item providers and item
processors. They are each represented by a simple interface provided by
the Spring Batch framework, which allows free reign over their
implementations and improves our ability to leverage the Spring
framework's dependency injection capabilities.</para>
<section>
<title id="s.5.1.1">Item Readers</title>
<para>An item reader is an object that is used to retrieve the inputs
for a step, one at a time. When the item reader has exhausted the items
it can provide, it will indicate this in a meaningful way (generally by
returning null). When coupled with an item processor, this forms a
complete item-oriented process, as each item taken from the provider is
then processed by the writer.</para>
</section>
<section>
<title id="s.5.1.2">Item Writers/Processors</title>
<para>An item writer is an object that is used to perform processing
for a step, one item at a time. Generally, an item writer has no
knowledge of the input it will receive next, only the item that that was
passed in its current invocation. As a result, item writers will
generally make no assumptions about the input they receive an treat
every item the same way and keep track of its own state between
invocations. When coupled with an item provider, this forms a complete
item-oriented process, as each item taken from the provider is then
processed by the processor.</para>
</section>
</section>
</section>
<section>
<title id="s.5.2">Support Stereotypes</title>
<para>While item readers and writers serve as the main entry points
for item-oriented processing, they might be supplemented by a number of support
classes that perform specific tasks within the reader / writer
lifecycle. These support stereotypes are useful for dividing the work of
item readers and writers into reusable pieces, as well as abstracting
away the details of processing, such as interaction with external systems.
Additionally, they give us another opportunity to leverage the powerful
configuration features of the Spring framework, as we can switch between
several beans implementing these support interfaces without changing the
driving item reader or writer.</para>
<section>
<title id="s.2.4.2">Item Transformers</title>
<para>An item transformer is a class that is capable of taking an object and changing it somehow before processing occurs. For instance, an item transformer my alter an object by changing its properties or by replacing it with another object entirely, such as a wrapper or derivative object. It can also be defined as an adaptor, allowing an object of one type to be converted for use as an object of a second type.
</para>
</section>
<section>
<title id="s.2.4.3">Item Writers</title>
<para>An item writer is a class that mediates interactions with an external target of output data, such as a file or database. An item writer often serves as a support for an item processor, typically abstracting away the details of interaction, such as the creation and maintenance of file handles, sockets, database connections and other output-related tasks such as buffering and stream flushing.
</para>
<para>An item transformer is a class that is capable of taking an object
and changing it somehow before processing occurs. For instance, an item
transformer my alter an object by changing its properties or by
replacing it with another object entirely, such as a wrapper or
derivative object. It can also be defined as an adaptor, allowing an
object of one type to be converted for use as an object of a second
type.</para>
</section>
</section>
<section>
<title id="s.3">High Level Processing Flow</title>

2
docs/src/site/docbook/reference/index.xml
View File

@@ -37,6 +37,6 @@
<xi:include href="samples.xml"/>
<xi:include href="batch-job-testing.xml"/>
<xi:include href="batch-performance-testing.xml"/>
<xi:include href="glossary.xml"/>
<!-- <xi:include href="glossary.xml"/> -->
</book>