579 lines
40 KiB
HTML
579 lines
40 KiB
HTML
<html><head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
|
<title>11. Common Batch Patterns</title><link rel="stylesheet" type="text/css" href="css/manual-multipage.css"><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"><link rel="home" href="index.html" title="Spring Batch - Reference Documentation"><link rel="up" href="index.html" title="Spring Batch - Reference Documentation"><link rel="prev" href="testing.html" title="10. Unit Testing"><link rel="next" href="jsr-352.html" title="12. JSR-352 Support"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">11. Common Batch Patterns</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="testing.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="jsr-352.html">Next</a></td></tr></table><hr></div><div class="chapter"><div class="titlepage"><div><div><h1 class="title"><a name="patterns" href="#patterns"></a>11. Common Batch Patterns</h1></div></div></div>
|
|
|
|
|
|
|
|
<p>Some batch jobs can be assembled purely from off-the-shelf components
|
|
in Spring Batch. For instance the <code class="classname">ItemReader</code> and
|
|
<code class="classname">ItemWriter</code> implementations can be configured to cover
|
|
a wide range of scenarios. However, for the majority of cases, custom code
|
|
will have to be written. The main API entry points for application
|
|
developers are the <code class="classname">Tasklet</code>,
|
|
<code class="classname">ItemReader</code>, <code class="classname">ItemWriter</code> and the
|
|
various listener interfaces. Most simple batch jobs will be able to use
|
|
off-the-shelf input from a Spring Batch <code class="classname">ItemReader</code>,
|
|
but it is often the case that there are custom concerns in the processing
|
|
and writing, which require developers to implement an
|
|
<code class="classname">ItemWriter</code> or
|
|
<code class="classname">ItemProcessor</code>.</p>
|
|
|
|
<p>Here, we provide a few examples of common patterns in custom business
|
|
logic. These examples primarily feature the listener interfaces. It should
|
|
be noted that an <code class="classname">ItemReader</code> or
|
|
<code class="classname">ItemWriter</code> can implement a listener interface as
|
|
well, if appropriate.</p>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="loggingItemProcessingAndFailures" href="#loggingItemProcessingAndFailures"></a>11.1 Logging Item Processing and Failures</h2></div></div></div>
|
|
|
|
|
|
<p>A common use case is the need for special handling of errors in a
|
|
step, item by item, perhaps logging to a special channel, or inserting a
|
|
record into a database. A chunk-oriented <code class="classname">Step</code>
|
|
(created from the step factory beans) allows users to implement this use
|
|
case with a simple <code class="classname">ItemReadListener</code>, for errors on
|
|
read, and an <code class="classname">ItemWriteListener</code>, for errors on
|
|
write. The below code snippets illustrate a listener that logs both read
|
|
and write failures:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> ItemFailureLoggerListener <span class="hl-keyword">extends</span> ItemListenerSupport {
|
|
|
|
<span class="hl-keyword">private</span> <span class="hl-keyword">static</span> Log logger = LogFactory.getLog(<span class="hl-string">"item.error"</span>);
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> onReadError(Exception ex) {
|
|
logger.error(<span class="hl-string">"Encountered error on read"</span>, e);
|
|
}
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> onWriteError(Exception ex, Object item) {
|
|
logger.error(<span class="hl-string">"Encountered error on write"</span>, ex);
|
|
}
|
|
|
|
}</pre>
|
|
|
|
<p>Having implemented this listener it must be registered with the
|
|
step:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><step</span> <span class="hl-attribute">id</span>=<span class="hl-value">"simpleStep"</span><span class="hl-tag">></span>
|
|
...
|
|
<span class="hl-tag"><listeners></span>
|
|
<span class="hl-tag"><listener></span>
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.example...ItemFailureLoggerListener"</span><span class="hl-tag">/></span>
|
|
<span class="hl-tag"></listener></span>
|
|
<span class="hl-tag"></listeners></span>
|
|
<span class="hl-tag"></step></span></pre>
|
|
|
|
<p>Remember that if your listener does anything in an
|
|
<code class="code">onError()</code> method, it will be inside a transaction that is
|
|
going to be rolled back. If you need to use a transactional resource such
|
|
as a database inside an <code class="code">onError()</code> method, consider adding a
|
|
declarative transaction to that method (see Spring Core Reference Guide
|
|
for details), and giving its propagation attribute the value
|
|
REQUIRES_NEW.</p>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="stoppingAJobManuallyForBusinessReasons" href="#stoppingAJobManuallyForBusinessReasons"></a>11.2 Stopping a Job Manually for Business Reasons</h2></div></div></div>
|
|
|
|
|
|
<p>Spring Batch provides a <code class="methodname">stop</code>() method
|
|
through the <code class="classname">JobLauncher</code> interface, but this is
|
|
really for use by the operator rather than the application programmer.
|
|
Sometimes it is more convenient or makes more sense to stop a job
|
|
execution from within the business logic.</p>
|
|
|
|
<p>The simplest thing to do is to throw a
|
|
<code class="classname">RuntimeException</code> (one that isn't retried
|
|
indefinitely or skipped). For example, a custom exception type could be
|
|
used, as in the example below:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> PoisonPillItemWriter <span class="hl-keyword">implements</span> ItemWriter<T> {
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> write(T item) <span class="hl-keyword">throws</span> Exception {
|
|
<span class="hl-keyword">if</span> (isPoisonPill(item)) {
|
|
<span class="hl-keyword">throw</span> <span class="hl-keyword">new</span> PoisonPillException(<span class="hl-string">"Posion pill detected: "</span> + item);
|
|
}
|
|
}
|
|
|
|
}</pre>
|
|
|
|
<p>Another simple way to stop a step from executing is to simply return
|
|
<code class="code">null</code> from the <code class="classname">ItemReader</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> EarlyCompletionItemReader <span class="hl-keyword">implements</span> ItemReader<T> {
|
|
|
|
<span class="hl-keyword">private</span> ItemReader<T> delegate;
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> setDelegate(ItemReader<T> delegate) { ... }
|
|
|
|
<span class="hl-keyword">public</span> T read() <span class="hl-keyword">throws</span> Exception {
|
|
T item = delegate.read();
|
|
<span class="hl-keyword">if</span> (isEndItem(item)) {
|
|
<span class="hl-keyword">return</span> null; <span class="hl-comment">// end the step here</span>
|
|
}
|
|
<span class="hl-keyword">return</span> item;
|
|
}
|
|
|
|
}</pre>
|
|
|
|
<p>The previous example actually relies on the fact that there is a
|
|
default implementation of the <code class="classname">CompletionPolicy</code>
|
|
strategy which signals a complete batch when the item to be processed is
|
|
null. A more sophisticated completion policy could be implemented and
|
|
injected into the <code class="classname">Step</code> through the
|
|
<code class="classname">SimpleStepFactoryBean</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><step</span> <span class="hl-attribute">id</span>=<span class="hl-value">"simpleStep"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><tasklet></span>
|
|
<span class="hl-tag"><chunk</span> <span class="hl-attribute">reader</span>=<span class="hl-value">"reader"</span> <span class="hl-attribute">writer</span>=<span class="hl-value">"writer"</span> <span class="hl-attribute">commit-interval</span>=<span class="hl-value">"10"</span>
|
|
<span class="bold"><strong>chunk-completion-policy="completionPolicy"</strong></span>/>
|
|
<span class="hl-tag"></tasklet></span>
|
|
<span class="hl-tag"></step></span>
|
|
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"completionPolicy"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.example...SpecialCompletionPolicy"</span><span class="hl-tag">/></span></pre>
|
|
|
|
<p>An alternative is to set a flag in the
|
|
<code class="classname">StepExecution</code>, which is checked by the
|
|
<code class="classname">Step</code> implementations in the framework in between
|
|
item processing. To implement this alternative, we need access to the
|
|
current <code class="classname">StepExecution</code>, and this can be achieved by
|
|
implementing a <code class="classname">StepListener</code> and registering it with
|
|
the <code class="classname">Step</code>. Here is an example of a listener that
|
|
sets the flag:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> CustomItemWriter <span class="hl-keyword">extends</span> ItemListenerSupport <span class="hl-keyword">implements</span> StepListener {
|
|
|
|
<span class="hl-keyword">private</span> StepExecution stepExecution;
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> beforeStep(StepExecution stepExecution) {
|
|
<span class="hl-keyword">this</span>.stepExecution = stepExecution;
|
|
}
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> afterRead(Object item) {
|
|
<span class="hl-keyword">if</span> (isPoisonPill(item)) {
|
|
stepExecution.setTerminateOnly(true);
|
|
}
|
|
}
|
|
|
|
}</pre>
|
|
|
|
<p>The default behavior here when the flag is set is for the step to
|
|
throw a <code class="classname">JobInterruptedException</code>. This can be
|
|
controlled through the <code class="classname">StepInterruptionPolicy</code>, but
|
|
the only choice is to throw or not throw an exception, so this is always
|
|
an abnormal ending to a job.</p>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="addingAFooterRecord" href="#addingAFooterRecord"></a>11.3 Adding a Footer Record</h2></div></div></div>
|
|
|
|
|
|
<p>Often when writing to flat files, a "footer" record must be appended
|
|
to the end of the file, after all processing has be completed. This can
|
|
also be achieved using the <code class="classname">FlatFileFooterCallback</code>
|
|
interface provided by Spring Batch. The
|
|
<code class="classname">FlatFileFooterCallback</code> (and its counterpart, the
|
|
<code class="classname">FlatFileHeaderCallback</code>) are optional properties of
|
|
the <code class="classname">FlatFileItemWriter</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"itemWriter"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...FlatFileItemWriter"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"resource"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"outputResource"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"lineAggregator"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"lineAggregator"</span><span class="hl-tag">/></span>
|
|
<span class="bold"><strong><property name="headerCallback" ref="headerCallback" /></strong></span>
|
|
<span class="bold"><strong><property name="footerCallback" ref="footerCallback" /></strong></span>
|
|
<span class="hl-tag"></bean></span></pre>
|
|
|
|
<p>The footer callback interface is very simple. It has just one method
|
|
that is called when the footer must be written:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">interface</span> FlatFileFooterCallback {
|
|
|
|
<span class="hl-keyword">void</span> writeFooter(Writer writer) <span class="hl-keyword">throws</span> IOException;
|
|
|
|
}</pre>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="writingASummaryFooter" href="#writingASummaryFooter"></a>11.3.1 Writing a Summary Footer</h3></div></div></div>
|
|
|
|
|
|
<p>A very common requirement involving footer records is to aggregate
|
|
information during the output process and to append this information to
|
|
the end of the file. This footer serves as a summarization of the file
|
|
or provides a checksum.</p>
|
|
|
|
<p>For example, if a batch job is writing
|
|
<code class="classname">Trade</code> records to a flat file, and there is a
|
|
requirement that the total amount from all the
|
|
<code class="classname">Trade</code>s is placed in a footer, then the following
|
|
<code class="classname">ItemWriter</code> implementation can be used:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> TradeItemWriter <span class="hl-keyword">implements</span> ItemWriter<Trade>,
|
|
FlatFileFooterCallback {
|
|
|
|
<span class="hl-keyword">private</span> ItemWriter<Trade> delegate;
|
|
|
|
<span class="hl-keyword">private</span> BigDecimal totalAmount = BigDecimal.ZERO;
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> write(List<? <span class="hl-keyword">extends</span> Trade> items) {
|
|
BigDecimal chunkTotal = BigDecimal.ZERO;
|
|
<span class="hl-keyword">for</span> (Trade trade : items) {
|
|
chunkTotal = chunkTotal.add(trade.getAmount());
|
|
}
|
|
|
|
delegate.write(items);
|
|
|
|
<span class="hl-comment">// After successfully writing all items</span>
|
|
totalAmount = totalAmount.add(chunkTotal);
|
|
}
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> writeFooter(Writer writer) <span class="hl-keyword">throws</span> IOException {
|
|
writer.write(<span class="hl-string">"Total Amount Processed: "</span> + totalAmount);
|
|
}
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> setDelegate(ItemWriter delegate) {...}
|
|
}</pre>
|
|
|
|
<p>This <code class="classname">TradeItemWriter</code> stores a
|
|
<code class="code">totalAmount</code> value that is increased with the
|
|
<code class="code">amount</code> from each <code class="classname">Trade</code> item written.
|
|
After the last <code class="classname">Trade</code> is processed, the framework
|
|
will call <code class="methodname">writeFooter</code>, which will put that
|
|
<code class="code">totalAmount</code> into the file. Note that the
|
|
<code class="methodname">write</code> method makes use of a temporary variable,
|
|
<code class="varname">chunkTotalAmount</code>, that stores the total of the trades
|
|
in the chunk. This is done to ensure that if a skip occurs in the
|
|
<code class="methodname">write</code> method, that the
|
|
<span class="property">totalAmount</span> will be left unchanged. It is only at
|
|
the end of the <code class="methodname">write</code> method, once we are
|
|
guaranteed that no exceptions will be thrown, that we update the
|
|
<code class="varname">totalAmount</code>.</p>
|
|
|
|
<p>In order for the <code class="methodname">writeFooter</code> method to be
|
|
called, the <code class="classname">TradeItemWriter</code> (which implements
|
|
<code class="classname">FlatFileFooterCallback</code>) must be wired into the
|
|
<code class="classname">FlatFileItemWriter</code> as the
|
|
<code class="code">footerCallback</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"tradeItemWriter"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"..TradeItemWriter"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"delegate"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"flatFileItemWriter"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"></bean></span>
|
|
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"flatFileItemWriter"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...FlatFileItemWriter"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"resource"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"outputResource"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"lineAggregator"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"lineAggregator"</span><span class="hl-tag">/></span>
|
|
<span class="bold"><strong> <property name="footerCallback" ref="tradeItemWriter" /></strong></span>
|
|
<span class="hl-tag"></bean></span></pre>
|
|
|
|
<p>The way that the <code class="classname">TradeItemWriter</code> has been
|
|
so far will only function correctly if the <code class="classname">Step</code>
|
|
is not restartable. This is because the class is stateful (since it
|
|
stores the <code class="code">totalAmount</code>), but the <code class="code">totalAmount</code>
|
|
is not persisted to the database, and therefore, it cannot be retrieved
|
|
in the event of a restart. In order to make this class restartable, the
|
|
<code class="classname">ItemStream</code> interface should be implemented along
|
|
with the methods <code class="methodname">open</code> and
|
|
<code class="methodname">update</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">void</span> open(ExecutionContext executionContext) {
|
|
<span class="hl-keyword">if</span> (executionContext.containsKey(<span class="hl-string">"total.amount"</span>) {
|
|
totalAmount = (BigDecimal) executionContext.get(<span class="hl-string">"total.amount"</span>);
|
|
}
|
|
}
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> update(ExecutionContext executionContext) {
|
|
executionContext.put(<span class="hl-string">"total.amount"</span>, totalAmount);
|
|
}</pre>
|
|
|
|
<p>The <code class="methodname">update</code> method will store the most
|
|
current version of <code class="code">totalAmount</code> to the
|
|
<code class="classname">ExecutionContext</code> just before that object is
|
|
persisted to the database. The <code class="methodname">open</code> method will
|
|
retrieve any existing <code class="code">totalAmount</code> from the
|
|
<code class="classname">ExecutionContext</code> and use it as the starting point
|
|
for processing, allowing the <code class="classname">TradeItemWriter</code> to
|
|
pick up on restart where it left off the previous time the
|
|
<code class="classname">Step</code> was executed.</p>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="drivingQueryBasedItemReaders" href="#drivingQueryBasedItemReaders"></a>11.4 Driving Query Based ItemReaders</h2></div></div></div>
|
|
|
|
|
|
<p>In the chapter on readers and writers, database input using paging
|
|
was discussed. Many database vendors, such as DB2, have extremely
|
|
pessimistic locking strategies that can cause issues if the table being
|
|
read also needs to be used by other portions of the online application.
|
|
Furthermore, opening cursors over extremely large datasets can cause
|
|
issues on certain vendors. Therefore, many projects prefer to use a
|
|
'Driving Query' approach to reading in data. This approach works by
|
|
iterating over keys, rather than the entire object that needs to be
|
|
returned, as the following example illustrates:</p>
|
|
|
|
<div class="mediaobject" align="center"><img src="images/drivingQueryExample.png" align="middle"></div>
|
|
|
|
<p>As you can see, this example uses the same 'FOO' table as was used
|
|
in the cursor based example. However, rather than selecting the entire
|
|
row, only the ID's were selected in the SQL statement. So, rather than a
|
|
FOO object being returned from <code class="classname">read</code>, an Integer
|
|
will be returned. This number can then be used to query for the 'details',
|
|
which is a complete Foo object:</p>
|
|
|
|
<div class="mediaobject" align="center"><img src="images/drivingQueryJob.png" align="middle"></div>
|
|
|
|
<p>An ItemProcessor should be used to transform the key obtained from
|
|
the driving query into a full 'Foo' object. An existing DAO can be used to
|
|
query for the full object based on the key.</p>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="multiLineRecords" href="#multiLineRecords"></a>11.5 Multi-Line Records</h2></div></div></div>
|
|
|
|
|
|
<p>While it is usually the case with flat files that one each record is
|
|
confined to a single line, it is common that a file might have records
|
|
spanning multiple lines with multiple formats. The following excerpt from
|
|
a file illustrates this:</p>
|
|
|
|
<pre class="programlisting">HEA;0013100345;2007-02-15
|
|
NCU;Smith;Peter;;T;20014539;F
|
|
BAD;;Oak Street 31/A;;Small Town;00235;IL;US
|
|
FOT;2;2;267.34</pre>
|
|
|
|
<p>Everything between the line starting with 'HEA' and the line
|
|
starting with 'FOT' is considered one record. There are a few
|
|
considerations that must be made in order to handle this situation
|
|
correctly:</p>
|
|
|
|
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">
|
|
<p>Instead of reading one record at a time, the
|
|
<code class="classname">ItemReader</code> must read every line of the
|
|
multi-line record as a group, so that it can be passed to the
|
|
<code class="classname">ItemWriter</code> intact.</p>
|
|
</li><li class="listitem">
|
|
<p>Each line type may need to be tokenized differently.</p>
|
|
</li></ul></div>
|
|
|
|
<p>Because a single record spans multiple lines, and we may not know
|
|
how many lines there are, the <code class="classname">ItemReader</code> must be
|
|
careful to always read an entire record. In order to do this, a custom
|
|
<code class="classname">ItemReader</code> should be implemented as a wrapper for
|
|
the <code class="classname">FlatFileItemReader</code>.</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"itemReader"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...MultiLineTradeItemReader"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"delegate"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.springframework.batch.item.file.FlatFileItemReader"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"resource"</span> <span class="hl-attribute">value</span>=<span class="hl-value">"data/iosample/input/multiLine.txt"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"lineMapper"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...DefaultLineMapper"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"lineTokenizer"</span> <span class="hl-attribute">ref</span>=<span class="hl-value">"orderFileTokenizer"</span><span class="hl-tag">/></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"fieldSetMapper"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><bean</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...PassThroughFieldSetMapper"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"></property></span>
|
|
<span class="hl-tag"></bean></span>
|
|
<span class="hl-tag"></property></span>
|
|
<span class="hl-tag"></bean></span>
|
|
<span class="hl-tag"></property></span>
|
|
<span class="hl-tag"></bean></span></pre>
|
|
|
|
<p>To ensure that each line is tokenized properly, which is especially
|
|
important for fixed length input, the
|
|
<code class="classname">PatternMatchingCompositeLineTokenizer</code> can be used
|
|
on the delegate <code class="classname">FlatFileItemReader</code>. See <a class="xref" href="readersAndWriters.html#prefixMatchingLineMapper" title="Multiple Record Types within a Single File">the section called “Multiple Record Types within a Single File”</a> for more details. The delegate
|
|
reader will then use a <code class="classname">PassThroughFieldSetMapper</code> to
|
|
deliver a <code class="classname">FieldSet</code> for each line back to the
|
|
wrapping <code class="classname">ItemReader</code>.</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><bean</span> <span class="hl-attribute">id</span>=<span class="hl-value">"orderFileTokenizer"</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.spr...PatternMatchingCompositeLineTokenizer"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"tokenizers"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><map></span>
|
|
<span class="hl-tag"><entry</span> <span class="hl-attribute">key</span>=<span class="hl-value">"HEA*"</span> <span class="hl-attribute">value-ref</span>=<span class="hl-value">"headerRecordTokenizer"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><entry</span> <span class="hl-attribute">key</span>=<span class="hl-value">"FOT*"</span> <span class="hl-attribute">value-ref</span>=<span class="hl-value">"footerRecordTokenizer"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><entry</span> <span class="hl-attribute">key</span>=<span class="hl-value">"NCU*"</span> <span class="hl-attribute">value-ref</span>=<span class="hl-value">"customerLineTokenizer"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"><entry</span> <span class="hl-attribute">key</span>=<span class="hl-value">"BAD*"</span> <span class="hl-attribute">value-ref</span>=<span class="hl-value">"billingAddressLineTokenizer"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"></map></span>
|
|
<span class="hl-tag"></property></span>
|
|
<span class="hl-tag"></bean></span></pre>
|
|
|
|
<p>This wrapper will have to be able recognize the end of a record so
|
|
that it can continually call <code class="methodname">read()</code> on its
|
|
delegate until the end is reached. For each line that is read, the wrapper
|
|
should build up the item to be returned. Once the footer is reached, the
|
|
item can be returned for delivery to the
|
|
<code class="classname">ItemProcessor</code> and
|
|
<code class="classname">ItemWriter</code>.</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">private</span> FlatFileItemReader<FieldSet> delegate;
|
|
|
|
<span class="hl-keyword">public</span> Trade read() <span class="hl-keyword">throws</span> Exception {
|
|
Trade t = null;
|
|
|
|
<span class="hl-keyword">for</span> (FieldSet line = null; (line = <span class="hl-keyword">this</span>.delegate.read()) != null;) {
|
|
String prefix = line.readString(<span class="hl-number">0</span>);
|
|
<span class="hl-keyword">if</span> (prefix.equals(<span class="hl-string">"HEA"</span>)) {
|
|
t = <span class="hl-keyword">new</span> Trade(); <span class="hl-comment">// Record must start with header</span>
|
|
}
|
|
<span class="hl-keyword">else</span> <span class="hl-keyword">if</span> (prefix.equals(<span class="hl-string">"NCU"</span>)) {
|
|
Assert.notNull(t, <span class="hl-string">"No header was found."</span>);
|
|
t.setLast(line.readString(<span class="hl-number">1</span>));
|
|
t.setFirst(line.readString(<span class="hl-number">2</span>));
|
|
...
|
|
}
|
|
<span class="hl-keyword">else</span> <span class="hl-keyword">if</span> (prefix.equals(<span class="hl-string">"BAD"</span>)) {
|
|
Assert.notNull(t, <span class="hl-string">"No header was found."</span>);
|
|
t.setCity(line.readString(<span class="hl-number">4</span>));
|
|
t.setState(line.readString(<span class="hl-number">6</span>));
|
|
...
|
|
}
|
|
<span class="hl-keyword">else</span> <span class="hl-keyword">if</span> (prefix.equals(<span class="hl-string">"FOT"</span>)) {
|
|
<span class="hl-keyword">return</span> t; <span class="hl-comment">// Record must end with footer</span>
|
|
}
|
|
}
|
|
Assert.isNull(t, <span class="hl-string">"No 'END' was found."</span>);
|
|
<span class="hl-keyword">return</span> null;
|
|
}</pre>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="executingSystemCommands" href="#executingSystemCommands"></a>11.6 Executing System Commands</h2></div></div></div>
|
|
|
|
|
|
<p>Many batch jobs may require that an external command be called from
|
|
within the batch job. Such a process could be kicked off separately by the
|
|
scheduler, but the advantage of common meta-data about the run would be
|
|
lost. Furthermore, a multi-step job would also need to be split up into
|
|
multiple jobs as well.</p>
|
|
|
|
<p>Because the need is so common, Spring Batch provides a
|
|
<code class="classname">Tasklet</code> implementation for calling system
|
|
commands:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><bean</span> <span class="hl-attribute">class</span>=<span class="hl-value">"org.springframework.batch.core.step.tasklet.SystemCommandTasklet"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"command"</span> <span class="hl-attribute">value</span>=<span class="hl-value">"echo hello"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-comment"><!-- 5 second timeout for the command to complete --></span>
|
|
<span class="hl-tag"><property</span> <span class="hl-attribute">name</span>=<span class="hl-value">"timeout"</span> <span class="hl-attribute">value</span>=<span class="hl-value">"5000"</span><span class="hl-tag"> /></span>
|
|
<span class="hl-tag"></bean></span></pre>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="handlingStepCompletionWhenNoInputIsFound" href="#handlingStepCompletionWhenNoInputIsFound"></a>11.7 Handling Step Completion When No Input is Found</h2></div></div></div>
|
|
|
|
|
|
<p>In many batch scenarios, finding no rows in a database or file to
|
|
process is not exceptional. The <code class="classname">Step</code> is simply
|
|
considered to have found no work and completes with 0 items read. All of
|
|
the <code class="classname">ItemReader</code> implementations provided out of the
|
|
box in Spring Batch default to this approach. This can lead to some
|
|
confusion if nothing is written out even when input is present. (which
|
|
usually happens if a file was misnamed, etc) For this reason, the meta
|
|
data itself should be inspected to determine how much work the framework
|
|
found to be processed. However, what if finding no input is considered
|
|
exceptional? In this case, programmatically checking the meta data for no
|
|
items processed and causing failure is the best solution. Because this is
|
|
a common use case, a listener is provided with just this
|
|
functionality:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> NoWorkFoundStepExecutionListener <span class="hl-keyword">extends</span> StepExecutionListenerSupport {
|
|
|
|
<span class="hl-keyword">public</span> ExitStatus afterStep(StepExecution stepExecution) {
|
|
<span class="hl-keyword">if</span> (stepExecution.getReadCount() == <span class="hl-number">0</span>) {
|
|
<span class="hl-keyword">return</span> ExitStatus.FAILED;
|
|
}
|
|
<span class="hl-keyword">return</span> null;
|
|
}
|
|
|
|
}</pre>
|
|
|
|
<p>The above <code class="classname">StepExecutionListener</code> inspects the
|
|
readCount property of the <code class="classname">StepExecution</code> during the
|
|
'afterStep' phase to determine if no items were read. If that is the case,
|
|
an exit code of FAILED is returned, indicating that the
|
|
<code class="classname">Step</code> should fail. Otherwise, null is returned,
|
|
which will not affect the status of the
|
|
<code class="classname">Step</code>.</p>
|
|
</div>
|
|
|
|
<div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="passingDataToFutureSteps" href="#passingDataToFutureSteps"></a>11.8 Passing Data to Future Steps</h2></div></div></div>
|
|
|
|
|
|
<p>It is often useful to pass information from one step to another.
|
|
This can be done using the <code class="classname">ExecutionContext</code>. The
|
|
catch is that there are two <code class="classname">ExecutionContext</code>s: one
|
|
at the <code class="classname">Step</code> level and one at the
|
|
<code class="classname">Job</code> level. The <code class="classname">Step</code>
|
|
<code class="classname">ExecutionContext</code> lives only as long as the step
|
|
while the <code class="classname">Job</code>
|
|
<code class="classname">ExecutionContext</code> lives through the whole
|
|
<code class="classname">Job</code>. On the other hand, the
|
|
<code class="classname">Step</code> <code class="classname">ExecutionContext</code> is
|
|
updated every time the <code class="classname">Step</code> commits a chunk while
|
|
the <code class="classname">Job</code> <code class="classname">ExecutionContext</code> is
|
|
updated only at the end of each <code class="classname">Step</code>.</p>
|
|
|
|
<p>The consequence of this separation is that all data must be placed
|
|
in the <code class="classname">Step</code> <code class="classname">ExecutionContext</code>
|
|
while the <code class="classname">Step</code> is executing. This will ensure that
|
|
the data will be stored properly while the <code class="classname">Step</code> is
|
|
on-going. If data is stored to the <code class="classname">Job</code>
|
|
<code class="classname">ExecutionContext</code>, then it will not be persisted
|
|
during <code class="classname">Step</code> execution and if the
|
|
<code class="classname">Step</code> fails, that data will be lost.</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> SavingItemWriter <span class="hl-keyword">implements</span> ItemWriter<Object> {
|
|
<span class="hl-keyword">private</span> StepExecution stepExecution;
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> write(List<? <span class="hl-keyword">extends</span> Object> items) <span class="hl-keyword">throws</span> Exception {
|
|
<span class="hl-comment">// ...</span>
|
|
|
|
ExecutionContext stepContext = <span class="hl-keyword">this</span>.stepExecution.getExecutionContext();
|
|
stepContext.put(<span class="hl-string">"someKey"</span>, someObject);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@BeforeStep</span></em>
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> saveStepExecution(StepExecution stepExecution) {
|
|
<span class="hl-keyword">this</span>.stepExecution = stepExecution;
|
|
}
|
|
}</pre>
|
|
|
|
<p>To make the data available to future <code class="classname">Step</code>s,
|
|
it will have to be "promoted" to the <code class="classname">Job</code>
|
|
<code class="classname">ExecutionContext</code> after the step has finished.
|
|
Spring Batch provides the
|
|
<code class="classname">ExecutionContextPromotionListener</code> for this purpose.
|
|
The listener must be configured with the keys related to the data in the
|
|
<code class="classname">ExecutionContext</code> that must be promoted. It can
|
|
also, optionally, be configured with a list of exit code patterns for
|
|
which the promotion should occur ("COMPLETED" is the default). As with all
|
|
listeners, it must be registered on the
|
|
<code class="classname">Step</code>.</p>
|
|
|
|
<pre class="programlisting"><span class="hl-tag"><job</span> <span class="hl-attribute">id</span>=<span class="hl-value">"job1"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><step</span> <span class="hl-attribute">id</span>=<span class="hl-value">"step1"</span><span class="hl-tag">></span>
|
|
<span class="hl-tag"><tasklet></span>
|
|
<span class="hl-tag"><chunk</span> <span class="hl-attribute">reader</span>=<span class="hl-value">"reader"</span> <span class="hl-attribute">writer</span>=<span class="hl-value">"savingWriter"</span> <span class="hl-attribute">commit-interval</span>=<span class="hl-value">"10"</span><span class="hl-tag">/></span>
|
|
<span class="hl-tag"></tasklet></span>
|
|
<span class="hl-tag"><listeners></span>
|
|
<span class="bold"><strong><listener ref="promotionListener"/></strong></span>
|
|
<span class="hl-tag"></listeners></span>
|
|
<span class="hl-tag"></step></span>
|
|
|
|
<span class="hl-tag"><step</span> <span class="hl-attribute">id</span>=<span class="hl-value">"step2"</span><span class="hl-tag">></span>
|
|
...
|
|
<span class="hl-tag"></step></span>
|
|
<span class="hl-tag"></job></span>
|
|
|
|
<span class="bold"><strong><beans:bean id="promotionListener" class="org.spr....ExecutionContextPromotionListener">
|
|
<beans:property name="keys" value="someKey"/>
|
|
</beans:bean></strong></span></pre>
|
|
|
|
<p>Finally, the saved values must be retrieved from the
|
|
<code class="classname">Job</code> <code class="classname">ExeuctionContext</code>:</p>
|
|
|
|
<pre class="programlisting"><span class="hl-keyword">public</span> <span class="hl-keyword">class</span> RetrievingItemWriter <span class="hl-keyword">implements</span> ItemWriter<Object> {
|
|
<span class="hl-keyword">private</span> Object someObject;
|
|
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> write(List<? <span class="hl-keyword">extends</span> Object> items) <span class="hl-keyword">throws</span> Exception {
|
|
<span class="hl-comment">// ...</span>
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@BeforeStep</span></em>
|
|
<span class="hl-keyword">public</span> <span class="hl-keyword">void</span> retrieveInterstepData(StepExecution stepExecution) {
|
|
JobExecution jobExecution = stepExecution.getJobExecution();
|
|
ExecutionContext jobContext = jobExecution.getExecutionContext();
|
|
<span class="hl-keyword">this</span>.someObject = jobContext.get(<span class="hl-string">"someKey"</span>);
|
|
}
|
|
}</pre>
|
|
</div>
|
|
</div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="testing.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="jsr-352.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">10. Unit Testing </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> 12. JSR-352 Support</td></tr></table></div></body></html> |