Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

OPEN - issue BATCH-120: Documentation and samples for correct use of listeners and stream interfaces for stateful custom components

Browse Source
This commit is contained in:
dsyer
2008-03-25 09:40:45 +00:00
parent 37d35f42eb
commit f9e4181182
2 changed files with 136 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files

259
docs/src/site/docbook/reference/glossary.xml
View File

@@ -7,254 +7,211 @@
<glossentry>
<glossterm>Batch</glossterm>
<glossdef>
<para>An accumulation of business transactions over
time.
</para>
<para>An accumulation of business transactions over time.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Batch Application Style</glossterm>
<glossdef>
<para>
Term used to designate batch as an application style in its
own right similar to online, Web or SOA. It has standard
elements of input, validation, transformation of
information to business model, business processing and
output. In addition, it requires monitoring at a macro
level.
</para>
<para>Term used to designate batch as an application style in its own
right similar to online, Web or SOA. It has standard elements of
input, validation, transformation of information to business model,
business processing and output. In addition, it requires monitoring at
a macro level.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Batch Processing</glossterm>
<glossdef>
<para>
The handling of a batch of many business transactions that
have accumulated over a period of time (e.g. an hour, day,
week, month, or year). It is the application of a process,
or set of processes, to many data entities or objects in a
repetitive and predictable fashion with either no manual
element, or a separate manual element for error processing.
</para>
<para>The handling of a batch of many business transactions that have
accumulated over a period of time (e.g. an hour, day, week, month, or
year). It is the application of a process, or set of processes, to
many data entities or objects in a repetitive and predictable fashion
with either no manual element, or a separate manual element for error
processing.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Batch Window</glossterm>
<glossdef>
<para>
The time frame within which a batch job must complete. This
can be constrained by other systems coming online, other
dependent jobs needing to execute or other factors specific
to the batch environment.
</para>
<para>The time frame within which a batch job must complete. This can
be constrained by other systems coming online, other dependent jobs
needing to execute or other factors specific to the batch
environment.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Step Controller</glossterm>
<glossterm>Step</glossterm>
<glossdef>
<para>
It is the main batch task or Unit of Work controller. It
initializes the tasklet, and controls the transaction
environment based on commit interval setting, etc.
</para>
<para>It is the main batch task or unit of work controller. It
initializes the business logic, and controls the transaction
environment based on commit interval setting, etc.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Tasklet</glossterm>
<glossdef>
<para>
The main application program created by application developer to process
the business logic for each LUW.
</para>
<para>An application program created by application developer to
process the business logic for an entire Step.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Batch Job Type</glossterm>
<glossdef>
<para>
Job Types describe application of jobs for particular type
of processing. Common areas are interface processing
(typically flat files), forms processing (either for online
pdf generation or print formats), report processing.
</para>
<para>Job Types describe application of jobs for particular type of
processing. Common areas are interface processing (typically flat
files), forms processing (either for online pdf generation or print
formats), report processing.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Driving Query</glossterm>
<glossdef>
<para>A driving
query identifies the set of work for a job to do; the job then
breaks that work into individual units of work. For instance,
identify all financial transactions that have a status of
&quot;pending transmission&quot; and send them to our partner
system. The driving query returns a set of record IDs to process;
each record ID then becomes a unit of work. A driving query may
involve a join (if the criteria for selection falls across two or
more tables) or it may work with a single table.</para>
</glossdef>
<glossdef>
<para>A driving query identifies the set of work for a job to do; the
job then breaks that work into individual units of work. For instance,
identify all financial transactions that have a status of "pending
transmission" and send them to our partner system. The driving query
returns a set of record IDs to process; each record ID then becomes a
unit of work. A driving query may involve a join (if the criteria for
selection falls across two or more tables) or it may work with a
single table.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Item</glossterm>
<glossdef>
<para>
An item represents the smallest ammount of complete data for processing. In the most
simple terms this might mean a line in a file, a row in a database table, or a particular
element in an XML file.
</para>
<para>An item represents the smallest ammount of complete data for
processing. In the most simple terms this might mean a line in a file,
a row in a database table, or a particular element in an XML
file.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Logicial Unit of Work (LUW)</glossterm>
<glossterm>Logicial Unit of Work
(LUW)</glossterm>
<glossdef>
<para>A batch job iterates through a driving query
(or another input source such as a file) to perform the set of
work that the job must accomplish. Each iteration of work
performed is a unit of work.</para>
<para>A batch job iterates through a driving query (or another input
source such as a file) to perform the set of work that the job must
accomplish. Each iteration of work performed is a unit of work.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Commit Interval</glossterm>
<glossdef>
<para>A set of
LUWs constitute a commit interval.</para>
</glossdef>
<glossdef>
<para>A set of LUWs constitute a commit interval.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Partitioning</glossterm>
<glossdef>
<para>Splitting a
job into multiple threads where each thread is responsible for a
subset of the overall data to be processed. The threads of
execution may be within the same JVM or they may span JVMs in a
clustered environment that supports workload balancing.</para>
</glossdef>
<glossdef>
<para>Splitting a job into multiple threads where each thread is
responsible for a subset of the overall data to be processed. The
threads of execution may be within the same JVM or they may span JVMs
in a clustered environment that supports workload balancing.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Staging Table</glossterm>
<glossdef>
<para>A table
that holds temporary data while it is being processed.</para>
</glossdef>
<glossdef>
<para>A table that holds temporary data while it is being
processed.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Restartable</glossterm>
<glossdef>
<para>- a job that
can be executed again and will assume the same identity as when
run initially. In othewords, it is has the same job instance
id.</para>
</glossdef>
<glossdef>
<para>- a job that can be executed again and will assume the same
identity as when run initially. In othewords, it is has the same job
instance id.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Rerunnable</glossterm>
<glossdef>
<para>
a job that is restartable and manages it&apos;s own state in
terms of previous run&apos;s record processing. Note&gt;&gt;
Rerunnable is tied to the driving query. If the driving
query can be formed so that it will limit the processed rows
when the job is restarted than re-runnable = true. This is
managed by the application architecture. Often times a
condition is added to the where statement to limit the rows
returned by the driving query with something like &quot;and
processedFlag != true&quot;.
<note>
<title>Understanding Rerunnable</title>
<para>
If re-runnable = false the architecture assumes responsibility for
tracking which rows have been processed. There is a default strategy
for tracking the last record processed by partition. Most batch jobs
only have one partition. The option is only valid for a restartable
job. The reason being is that the architecture will have to persist
the restart data, which is only available on a restartable job as the
restart data is saved and made available from the execution of the
last run if restartable = true. In pseudo code it is the following:
</para>
<para>
StartOver ::= restartable = false. Restartable ::= true | false If
(Restartable) re-runnable ::= true | false We don't persist restart
information for a non-restartable job. As you can see, it doesn't
make sense. Rerunnable has always confused the best of us and we're
open to suggestions on how to better explain it.
</para>
</note>
</para>
<para>a job that is restartable and manages it's own state in terms of
previous run's record processing. An example of a rerunnable step is
one based on a driving query. If the driving query can be formed so
that it will limit the processed rows when the job is restarted than
it is re-runnable. This is managed by the application logic. Often
times a condition is added to the where statement to limit the rows
returned by the driving query with something like "and processedFlag
!= true". </para>
</glossdef>
</glossentry>
<glossentry>
<glossentry>
<glossterm>Repeat</glossterm>
<glossdef>
<para>One of the most basic units of batch processing, that
defines repeatability calling a portion of code until it is
finished, and while there is no error. Typically a batch
process would be repeatable as long as there is input.
</para>
<para>One of the most basic units of batch processing, that defines
repeatability calling a portion of code until it is finished, and
while there is no error. Typically a batch process would be repeatable
as long as there is input.</para>
</glossdef>
</glossentry>
<glossentry>
<glossentry>
<glossterm>Retry</glossterm>
<glossdef>
<para>Simplifies the execution of operations with retry semantics most frequently associated
with handling transactional output exceptions. Retry is slightly different from repeat, rather than continually calling a block of code,
retry is stateful, and continually calls the same block of code with the same input, until it either succeeds, or some type of retry limit
has been exceeded. It is only generally useful if the operation is non-deterministic meaning that a retry on a subsequent invocation might
succeed because something in the environment has improved.
</para>
</glossdef>
<glossdef>
<para>Simplifies the execution of operations with retry semantics most
frequently associated with handling transactional output exceptions.
Retry is slightly different from repeat, rather than continually
calling a block of code, retry is stateful, and continually calls the
same block of code with the same input, until it either succeeds, or
some type of retry limit has been exceeded. It is only generally
useful if the operation is non-deterministic meaning that a retry on a
subsequent invocation might succeed because something in the
environment has improved.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Recover</glossterm>
<glossdef>
<para>Recover operations handle an exception in such a way that a repeat process is able to
continue.
</para>
</glossdef>
<glossdef>
<para>Recover operations handle an exception in such a way that a
repeat process is able to continue.</para>
</glossdef>
</glossentry>
<glossentry>
<glossterm>Skip</glossterm>
<glossdef>
<para>Skip is a recovery strategy often used on file input sources as the strategy for ignoring
bad input records that failed validation.
</para>
</glossdef>
<glossdef>
<para>Skip is a recovery strategy often used on file input sources as
the strategy for ignoring bad input records that failed
validation.</para>
</glossdef>
</glossentry>
</glossdiv>
</glossary>
</glossary>

58
docs/src/site/docbook/reference/readersAndWriters.xml
View File

@@ -1090,60 +1090,58 @@
<classname>java.io.File</classname>. Both XML and Flat File resources can
be configured using standard Spring constructs:</para>
<programlisting> &lt;bean id="customerFileLocator"
class="org.springframework.core.io.FileSystemResource"&gt;
&lt;constructor-arg type="java.lang.String"
value="/outputs/20070122.testStream.CustomerReportStep.TEMP.txt" /&gt;
<programlisting> &lt;bean id="flatFileItemReader"
class="org.springframework.batch.item.file.FlatFileItemReader"&gt;
&lt;property name="resource"
value="file://outputs/20070122.testStream.CustomerReportStep.TEMP.txt" /&gt;
&lt;/bean&gt;</programlisting>
<para>The above <classname>Resource</classname> will load the file from
the file system, at the location specificied in the constructor. In most
spring applications, this solution is good enough. Resources are primarily
used to load spring application context files, and perhaps a properties
file. The names of these are known at compile time. However, in many batch
scenarios, the file name may need to be determined at runtime as a
parameter to the job. This could be solved using a
<classname>PropertyPlaceholderConfigurer</classname> and '-D' parameters.
(System arguments):</para>
the file system, at the location specificied. Note that absolute locations
have to start with a double slash ("//"). In most spring applications,
this solution is good enough because the names of these are known at
compile time. However, in batch scenarios, the file name may need to be
determined at runtime as a parameter to the job. This could be solved
using '-D' parameters, i.e. a system property:</para>
<programlisting>&lt;bean id="customerFileLocator"
class="org.springframework.core.io.FileSystemResource"&gt;
&lt;constructor-arg type="java.lang.String"
value="${file.name}" /&gt;
&lt;/bean&gt;</programlisting>
<programlisting>&lt;bean id="flatFileItemReader"
class="org.springframework.batch.item.file.FlatFileItemReader"&gt;
&lt;property name="resource" value="${input.file.name}" /&gt;
&lt;/bean&gt;</programlisting>
<para>All that would be required for this solution to work would be a
placeholder configurer in the <classname>ApplicationContext</classname>
and the system arguments provided. (-Dfile.name="/file.txt") This solution
will work, but it would be much preferred to place the file name in the
<classname>JobParameters</classname> of the job and access them that way.
To allow for this, Spring Batch provides the
system argument (-Dinput.file.name="file://file.txt"). (Note that although
a <classname>PropertyPlaceholderConfigurer</classname> can be used here,
it is not necessary if the system property is always set because the
<classname>ResourceEditor</classname> in Spring already filters and does
placeholder replacement on system properties.) </para>
<para>Often in a batch setting it is preferable to parameterise the file
name in the <classname>JobParameters</classname> of the job, instead of
through system properties, and access them that way. To allow for this,
Spring Batch provides the
<classname>StepExecutionResourceProxy</classname>. The proxy can use
either job name, step name, or any values from the
<classname>JobParameters</classname>, by surround them with %:</para>
<programlisting> &lt;bean id="inputFile"
class="org.springframework.batch.core.resource.StepExecutionResourceProxy" /&gt;
&lt;property name="filePattern" value="/%JOB_NAME%/%STEP_NAME%/%file.name%" /&gt;
&lt;property name="filePattern" value="//%JOB_NAME%/%STEP_NAME%/%file.name%" /&gt;
&lt;/bean&gt;</programlisting>
<para>Assuming a job name of 'fooJob', and a step name of 'fooStep', and
the key-value pair of 'file.name="fileName.txt' is in the
<classname>JobParameters</classname> the job is start with, the following
filename will be passed as the <classname>Resource</classname>:
"<filename>/fooJob/fooStep/fileName.txt</filename>". It should be noted
"<filename>//fooJob/fooStep/fileName.txt</filename>". It should be noted
that in order for the proxy to have access to the
<classname>StepExecution</classname>, it must be registered as a
<classname>StepListener</classname>: </para>
<classname>StepListener</classname>:</para>
<programlisting> &lt;bean id="fooStep" parent="abstractStep"
p:itemReader-ref="itemReader"
p:itemWriter-ref="itemWriter"&gt;
&lt;property name="listeners"&gt;
&lt;list&gt;
&lt;ref bean="inputFile" /&gt;
&lt;/list&gt;
&lt;/property&gt;
&lt;property name="listeners" ref="inputFile" /&gt;
&lt;/bean&gt;</programlisting>
<para>The <classname>StepListener</classname> interface will be discussed