106 lines
5.9 KiB
HTML
106 lines
5.9 KiB
HTML
<?xml version="1.0" encoding="UTF-8" standalone="no"?><!DOCTYPE html><html xmlns="http://www.w3.org/1999/xhtml" xmlns:epub="http://www.idpf.org/2007/ops" xmlns:m="http://www.w3.org/1998/Math/MathML" xmlns:pls="http://www.w3.org/2005/01/pronunciation-lexicon" xmlns:ssml="http://www.w3.org/2001/10/synthesis" xmlns:svg="http://www.w3.org/2000/svg"><head><title>Multi-Line Records</title><link rel="stylesheet" type="text/css" href="docbook-epub.css"/><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"/><link rel="prev" href="ch11s04.xhtml" title="Driving Query Based ItemReaders"/><link rel="next" href="ch11s06.xhtml" title="Executing System Commands"/></head><body><header/><section class="section" title="Multi-Line Records" epub:type="subchapter" id="multiLineRecords"><div class="titlepage"><div><div><h2 class="title" style="clear: both">Multi-Line Records</h2></div></div></div>
|
|
|
|
|
|
<p>While it is usually the case with flat files that one each record is
|
|
confined to a single line, it is common that a file might have records
|
|
spanning multiple lines with multiple formats. The following excerpt from
|
|
a file illustrates this:</p>
|
|
|
|
<pre class="programlisting">HEA;0013100345;2007-02-15
|
|
NCU;Smith;Peter;;T;20014539;F
|
|
BAD;;Oak Street 31/A;;Small Town;00235;IL;US
|
|
FOT;2;2;267.34</pre>
|
|
|
|
<p>Everything between the line starting with 'HEA' and the line
|
|
starting with 'FOT' is considered one record. There are a few
|
|
considerations that must be made in order to handle this situation
|
|
correctly:</p>
|
|
|
|
<div class="itemizedlist" epub:type="list"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem" epub:type="list-item">
|
|
<p>Instead of reading one record at a time, the
|
|
<code class="classname">ItemReader</code> must read every line of the
|
|
multi-line record as a group, so that it can be passed to the
|
|
<code class="classname">ItemWriter</code> intact.</p>
|
|
</li><li class="listitem" epub:type="list-item">
|
|
<p>Each line type may need to be tokenized differently.</p>
|
|
</li></ul></div>
|
|
|
|
<p>Because a single record spans multiple lines, and we may not know
|
|
how many lines there are, the <code class="classname">ItemReader</code> must be
|
|
careful to always read an entire record. In order to do this, a custom
|
|
<code class="classname">ItemReader</code> should be implemented as a wrapper for
|
|
the <code class="classname">FlatFileItemReader</code>.</p>
|
|
|
|
<pre class="programlisting"><bean id="itemReader" class="org.spr...MultiLineTradeItemReader">
|
|
<property name="delegate">
|
|
<bean class="org.springframework.batch.item.file.FlatFileItemReader">
|
|
<property name="resource" value="data/iosample/input/multiLine.txt" />
|
|
<property name="lineMapper">
|
|
<bean class="org.spr...DefaultLineMapper">
|
|
<property name="lineTokenizer" ref="orderFileTokenizer"/>
|
|
<property name="fieldSetMapper">
|
|
<bean class="org.spr...PassThroughFieldSetMapper" />
|
|
</property>
|
|
</bean>
|
|
</property>
|
|
</bean>
|
|
</property>
|
|
</bean></pre>
|
|
|
|
<p>To ensure that each line is tokenized properly, which is especially
|
|
important for fixed length input, the
|
|
<code class="classname">PatternMatchingCompositeLineTokenizer</code> can be used
|
|
on the delegate <code class="classname">FlatFileItemReader</code>. See <a class="xref" href="ch06s06.xhtml#prefixMatchingLineMapper" title="Multiple Record Types within a Single File">the section called “Multiple Record Types within a Single File”</a> for more details. The delegate
|
|
reader will then use a <code class="classname">PassThroughFieldSetMapper</code> to
|
|
deliver a <code class="classname">FieldSet</code> for each line back to the
|
|
wrapping <code class="classname">ItemReader</code>.</p>
|
|
|
|
<pre class="programlisting"><bean id="orderFileTokenizer" class="org.spr...PatternMatchingCompositeLineTokenizer">
|
|
<property name="tokenizers">
|
|
<map>
|
|
<entry key="HEA*" value-ref="headerRecordTokenizer" />
|
|
<entry key="FOT*" value-ref="footerRecordTokenizer" />
|
|
<entry key="NCU*" value-ref="customerLineTokenizer" />
|
|
<entry key="BAD*" value-ref="billingAddressLineTokenizer" />
|
|
</map>
|
|
</property>
|
|
</bean></pre>
|
|
|
|
<p>This wrapper will have to be able recognize the end of a record so
|
|
that it can continually call <code class="methodname">read()</code> on its
|
|
delegate until the end is reached. For each line that is read, the wrapper
|
|
should build up the item to be returned. Once the footer is reached, the
|
|
item can be returned for delivery to the
|
|
<code class="classname">ItemProcessor</code> and
|
|
<code class="classname">ItemWriter</code>.</p>
|
|
|
|
<pre class="programlisting">private FlatFileItemReader<FieldSet> delegate;
|
|
|
|
public Trade read() throws Exception {
|
|
Trade t = null;
|
|
|
|
for (FieldSet line = null; (line = this.delegate.read()) != null;) {
|
|
String prefix = line.readString(0);
|
|
if (prefix.equals("HEA")) {
|
|
t = new Trade(); // Record must start with header
|
|
}
|
|
else if (prefix.equals("NCU")) {
|
|
Assert.notNull(t, "No header was found.");
|
|
t.setLast(line.readString(1));
|
|
t.setFirst(line.readString(2));
|
|
...
|
|
}
|
|
else if (prefix.equals("BAD")) {
|
|
Assert.notNull(t, "No header was found.");
|
|
t.setCity(line.readString(4));
|
|
t.setState(line.readString(6));
|
|
...
|
|
}
|
|
else if (prefix.equals("FOT")) {
|
|
return t; // Record must end with footer
|
|
}
|
|
}
|
|
Assert.isNull(t, "No 'END' was found.");
|
|
return null;
|
|
}</pre>
|
|
</section><footer/></body></html> |