From a3779fe8dc1daca40989abce06b50b99d2b774ca Mon Sep 17 00:00:00 2001 From: dhgarrette Date: Tue, 3 Feb 2009 02:46:17 +0000 Subject: [PATCH] BATCH-1056: Fixed numerous errors --- .../docbook/reference/readersAndWriters.xml | 683 +++++++++--------- 1 file changed, 352 insertions(+), 331 deletions(-) diff --git a/docs/src/site/docbook/reference/readersAndWriters.xml b/docs/src/site/docbook/reference/readersAndWriters.xml index 10b8f90e9..471990e57 100644 --- a/docs/src/site/docbook/reference/readersAndWriters.xml +++ b/docs/src/site/docbook/reference/readersAndWriters.xml @@ -27,17 +27,17 @@ XML - XML ItemReaders process XML independently of technologies used for parsing, mapping and validating objects. Input - data allows for the validation of and XML file against an XSD + data allows for the validation of an XML file against an XSD schema. - Database - A database resource is accessed that returns + Database - A database resource is accessed to return resultsets which can be mapped to objects for processing. The default SQL ItemReaders invoke a RowMapper to return objects, keep track of the current row if restart is - required, basic statistics, and some transaction enhancements that - will be explained later. + required, store basic statistics, and provide some transaction + enhancements that will be explained later. There are many more possibilities, but we'll focus on the basic ones for this chapter. A complete list of all available ItemReaders @@ -54,12 +54,11 @@ The read method defines the most essential - contract of the ItemReader, calling it returns one - Item, returning null if no more items are left. An item might represent a - line in a file, a row in a database, or an element in an XML file. It is - generally expected that these will be mapped to a usable domain object - (i.e. Trade, Foo, etc) but there is no requirement in the contract to do - so. + contract of the ItemReader; calling it returns one + Item or null if no more items are left. An item might represent a line in + a file, a row in a database, or an element in an XML file. It is generally + expected that these will be mapped to a usable domain object (i.e. Trade, + Foo, etc) but there is no requirement in the contract to do so. It is expected that implementations of the ItemReader interface will be forward only. However, @@ -79,8 +78,8 @@ ItemReader, but with inverse operations. Resources still need to be located, opened and closed but they differ in that an ItemWriter writes out, rather than reading in. In - the case of databases or queues these may be inserts, updates or sends. - The format of the serialization of the output is specific for every batch + the case of databases or queues these may be inserts, updates, or sends. + The format of the serialization of the output is specific to each batch job. As with ItemReader, @@ -95,15 +94,15 @@ As with read on ItemReader, write provides - the basic contract of ItemWriter, it will attempt + the basic contract of ItemWriter; it will attempt to write out the list of items passed in as long as it is open. Because it is generally expected that items will be 'batched' together into a chunk - and then output, the interface accepts a list, rather than an item by - itself. After writing out the list, any flushing that may be necessary can - be performed before returning from the write method. For example, if - writing to a Hibernate DAO, multiple calls to write can be made, one for - each item. The writer can then call close on the hibernate Session before - returning. + and then output, the interface accepts a list of items, rather than an + item by itself. After writing out the list, any flushing that may be + necessary can be performed before returning from the write method. For + example, if writing to a Hibernate DAO, multiple calls to write can be + made, one for each item. The writer can then call close on the hibernate + Session before returning.
@@ -139,14 +138,14 @@ } The class above contains another ItemWriter - that it delgates to after having provided some business logic. This + to which it delgates after having provided some business logic. This pattern could easily be used for an ItemReader as well, perhaps to obtain more reference data based upon the input that was provided by the main ItemReader. It is also useful if you need to control the call to write yourself. However, if you only want to 'transform' the item passed in for writing - before it is actual written, there isn't much need to call - write yourself, you just want to modify the item. + before it is actually written, there isn't much need to call + write yourself: you just want to modify the item. For this scenario, Spring Batch provides the ItemProcessor interface: @@ -155,14 +154,14 @@ O process(I item) throws Exception; } - An ItemProcessor is very simple, given one - object, transform it and return another. The object provided may or may + An ItemProcessor is very simple; given one + object, transform it and return another. The provided object may or may not be of the same type. The point is that business logic may be applied within process, and is completely up to the developer to create. An ItemProcessor can be wired directly into a step, For example, assuming an ItemReader provides a class of type Foo, and it needs to be converted to type Bar before being - written out. An ItemTransformer can be written that + written out. An ItemProcessor can be written that performs the conversion: public class Foo {} @@ -174,15 +173,15 @@ public class FooProcessor implements ItemProcessor<Foo,Bar>{ //Perform simple transformation, convert a Foo to a Bar - public Bar transform(Foo foo) throws Exception { + public Bar process(Foo foo) throws Exception { return new Bar(foo); } } public class BarWriter implements ItemWriter<Bar>{ - public void write(Bar bar) throws Exception { - //write bar + public void write(List<? extends Bar> bars) throws Exception { + //write bars } //rest of class ommitted for clarity @@ -193,12 +192,12 @@ class FooProcessor that adheres to the ItemProcessor interface. The transformation is simple, but any type of transformation could be done here. The - BarWriter will be used to write out 'Bars', - throwing an exception if any other type is provided. Similarly, the - FooProcessor will throw an exception if anything but a - Foo is provided. The - FooProcessor can then be injected into a - Step: + BarWriter will be used to write out + Bar objects, throwing an exception if any other + type is provided. Similarly, the FooProcessor will + throw an exception if anything but a Foo is + provided. The FooProcessor can then be injected + into a Step: <job id="ioSampleJob"> @@ -213,11 +212,11 @@ Chaining ItemProcessors Performing a single transformation is useful in many scenarios, - but what if you want to 'chain' together multiple ItemProcessors? This - can be accomplished using the composite pattern mentioned previously. To - update the previous, single transformation, example, - Foo will be Transformed to - Bar, which will be transformed to + but what if you want to 'chain' together multiple + ItemProcessors? This can be accomplished using + the composite pattern mentioned previously. To update the previous, + single transformation, example, Foo will be + transformed to Bar, which will be transformed to Foobar and written out: public class Foo {} @@ -233,22 +232,22 @@ public class FooProcessor implements ItemProcessor<Foo,Bar>{ //Perform simple transformation, convert a Foo to a Bar - public Bar transform(Foo foo) throws Exception { + public Bar process(Foo foo) throws Exception { return new Bar(foo); } } public class BarProcessor implements ItemProcessor<Bar,FooBar>{ - public FooBar transform(Bar bar) throws Exception { + public FooBar process(Bar bar) throws Exception { return new Foobar(bar); } } public class FoobarWriter implements ItemWriter<FooBar>{ - public void write(Object item) throws Exception { - //write Foobar + public void write(List<? extends FooBar> items) throws Exception { + //write items } //rest of class ommitted for clarity @@ -293,8 +292,8 @@ One typical use for an item processor is to filter out records before they are passed to the ItemWriter. Filtering is an action distinct from skpping; skipping indicates that a record is invalid - whereas filtering simply indicates that a record should not be written. - + whereas filtering simply indicates that a record should not be + written. For example, consider a batch job that reads a file containing three different types of records: records to insert, records to update, @@ -303,7 +302,7 @@ ItemWriter. But, since these records are not actually bad records, we would want to filter them out, rather than skip. As a result, the ItemWriter would receive only "insert" and - "update" records. + "update" records. To filter a record, one simply returns "null" from the ItemProcessor. The framework will detect that the @@ -317,36 +316,37 @@
ItemStream - Both ItemReaders and ItemWriters serve their individual purposes - well, but there is a common concern among both of them that necessitates - another interface. In general, as part of the scope of a batch job, - readers and writers need to be opened, closed, and require a mechanism for - persisting state: + Both ItemReaders and + ItemWriters serve their individual purposes well, + but there is a common concern among both of them that necessitates another + interface. In general, as part of the scope of a batch job, readers and + writers need to be opened, closed, and require a mechanism for persisting + state: public interface ItemStream { - void open(ExecutionContext executionContext) throws StreamException; + void open(ExecutionContext executionContext) throws ItemStreamException; void update(ExecutionContext executionContext) throws ItemStreamException; - void close(ExecutionContext executionContext) throws StreamException; + void close() throws ItemStreamException; } - Before describing each method, its worth briefly mentioning the + Before describing each method, we should mention the ExecutionContext. Clients of an - ItemReader that also implements + ItemReader that also implement ItemStream should call open before any calls to - read, to open any resources such as files or - obtain connections. A similar restriction applies to an - ItemWriter that also implements + read in order to open any resources such as files + or to obtain connections. A similar restriction applies to an + ItemWriter that implements ItemStream. As mentioned in Chapter 2, if expected data is found in the ExecutionContext, it may be used to start the ItemReader or ItemWriter at a location other than its initial state. Conversely, close will be called to ensure - any resources allocated during open will be + that any resources allocated during open will be released safely. update is called primarily to ensure that any state currently being held is loaded into the provided ExecutionContext. This method will be called before @@ -373,9 +373,11 @@ If they do, and they are being used in conjunction with Spring Batch Core as part of a Step in a Job, then they almost certainly need to be registered manually with the - Step. Registration is automatic when the reader, - writer, or processor is directly wired into the Step. The delegates are - not known to the Step, so they need to be injected + Step. A reader, writer, or processor that is + directly wired into the Step will be registered automatically if it + implements ItemStream or a + StepListener interface. But because the delegates + are not known to the Step, they need to be injected as listeners or streams (or both if appropriate): @@ -406,7 +408,9 @@ always been the flat file. Unlike XML, which has an agreed upon standard for defining how it is structured (XSD), anyone reading a flat file must understand ahead of time exactly how the file is structured. In general, - all flat files fall into two types: Delimited and Fixed Length. + all flat files fall into two types: Delimited and Fixed Length. Delimited + files are those in which fields are separated by a delimiter, such as a + comma. Fixed Length files have fields that are a set length.
The FieldSet @@ -565,9 +569,10 @@ LineMapper As with RowMapper, which takes a low - level construct such as ResultSet and returns an Object, - flat file procesing requires the same construct to convert a String - line into an Object: + level construct such as ResultSet and returns + an Object, flat file processing requires the + same construct to convert a String line into an + Object: public interface LineMapper<T> { T mapLine(String line, int lineNumber) throws Exception; @@ -575,29 +580,29 @@ - The basic contract is that, given the current line, and the line - number its associated with, return a resulting domain object. This is - similar to RowMapper in that each line is - associated with it's line number, just as each row in a - ResultSet is tied to the row number it belongs - to. This allows for tying the line number to the resulting domain - object for identity comparison, or for more informative logging. - However, unlike RowMapper, the + The basic contract is that, given the current line and the line + number with which it is associated, the mapper should return a + resulting domain object. This is similar to + RowMapper in that each line is associated with + its line number, just as each row in a + ResultSet is tied to its row number. This + allows the line number to be tied to the resulting domain object for + identity comparison or for more informative logging. However, unlike + RowMapper, the LineMapper is given a raw line which, as - discussed above, only gets you halfway there. What is needed is - tokenization of the line into a FieldSet, which - can then be mapped to an object, as described below. + discussed above, only gets you halfway there. The line must be + tokenized into a FieldSet, which can then be + mapped to an object, as described below.
LineTokenizer - Because there can be many formats of flat file data, which all - need to be converted to a FieldSet so that a - useful domain object can be created from them, an abstraction for - turning a line of input into a FieldSet is - necessary. In Spring Batch, this is called a - LineTokenizer: + An abstraction for turning a line of input into a line into a + FieldSet is necessary because there can be many + formats of flat file data that need to be converted to a + FieldSet. In Spring Batch, this interface is + the LineTokenizer: public interface LineTokenizer { @@ -610,30 +615,31 @@ The contract of a LineTokenizer is such that, given a line of input (in theory the - String could encompass more than one line) a + String could encompass more than one line), a FieldSet representing the line will be - returned. This can then be passed to a + returned. This FieldSet can then be passed to a FieldSetMapper. Spring Batch contains the - following LineTokenizer implementations: + following LineTokenizer implementations: DelmitedLineTokenizer - Used for - files that separate records by a delimiter. The most common is a - comma, but pipes or semicolons are often used as well + files where fields in a record are separated by a delimiter. The + most common delimiter is a comma, but pipes or semicolons are + often used as well. - FixedLengthTokenizer - Used for - tokenizing files where each record is separated by a 'fixed width' - that must be defined per record. + FixedLengthTokenizer - Used for files + where fields in a record are each a 'fixed width'. The width of + each field must be defined for each record type. PrefixMatchingCompositeLineTokenizer - - Tokenizer that determines which among a list of Tokenizers - should be used on a particular line by checking against a - prefix. + - Determines which among a list of + LineTokenizers should be used on a + particular line by checking against a prefix.
@@ -644,8 +650,8 @@ The FieldSetMapper interface defines a single method, mapLine, which takes a FieldSet object and maps its contents to an - object. This object may be a custom DTO or domain object, or it could - be as simple as an array, depending on your needs. The + object. This object may be a custom DTO, a domain object, or a simple + array, depending on the needs of the job. The FieldSetMapper is used in conjunction with the LineTokenizer to translate a line of data from a resource into an object of the desired type: @@ -659,8 +665,9 @@ - The pattern used is the same as RowMapper - used by JdbcTemplate. + The pattern used is the same as the + RowMapper used by + JdbcTemplate.
@@ -674,28 +681,30 @@ - Pass the string line into the LineTokenizer#tokenize() - method, in order to retrieve a - FieldSet + Pass the string line into the + LineTokenizer#tokenize() method, in + order to retrieve a FieldSet. - Pass the FieldSet returned from tokenizing to a - FieldSetMapper, returning the result from the ItemReader#read() - method + Pass the FieldSet returned from + tokenizing to a FieldSetMapper, returning + the result from the ItemReader#read() + method. The two interfaces described above represent two separate tasks: converting a line into a FieldSet, and mapping - a FieldSet to a domain object. Becaue the input - of a LineTokenizer matches the input of the - LineMapper (a line), and the output of a - FieldSetMapper matches the output of the - LineMapper, and this is the deafult behavior - most users will need, a default implementation that uses both a - LineTokenizer and - FieldSetMapper is provided: + a FieldSet to a domain object. Because the + input of a LineTokenizer matches the input of + the LineMapper (a line), and the output of a + FieldSetMapper matches the output of the + LineMapper, a default implementation that uses + both a LineTokenizer and + FieldSetMapper is provided. The + DefaultLineMapper represents the behavior most + users will need: public class DefaultLineMapper<T> implements LineMapper<T>, InitializingBean { @@ -739,8 +748,8 @@ "AdamBo00,Adams,Bob,te,1946,1969", "AdamCh00,Adams,Charlie,wr,1979,2003" - The contents of this file will be mapped to the following Player - domain object: + The contents of this file will be mapped to the following + Player domain object: public class Player implements Serializable { private String ID; @@ -762,13 +771,14 @@ } - In order to map a FieldSet into a Player - object, a FieldSetMapper that returns players - needs to be defined: + In order to map a FieldSet into a + Player object, a + FieldSetMapper that returns players needs to be + defined: protected static class PlayerFieldSetMapper implements FieldSetMapper<Player> { - public Object mapLine(FieldSet fieldSet) { + public Player mapFieldSet(FieldSet fieldSet) { Player player = new Player(); player.setID(fieldSet.readString(0)); @@ -808,13 +818,14 @@
Mapping fields by name - There is one additional functionality a - LineTokenizer that is similar in function to a - Jdbc ResultSet. The names of the fields can be - injected into the LineTokenizer to increase the + There is one additional piece of functionality that is allowed + by both DelimitedLineTokenizer and + FixedLengthTokenizer that is similar in + function to a Jdbc ResultSet. The names of the + fields can be injected into either of these + LineTokenizer implementations to increase the readability of the mapping function. First, the column names of all - fields in the flat file are injected into the - LineTokenizer: + fields in the flat file are injected into the tokenizer: tokenizer.setNames(new String[] {"ID", "lastName","firstName","position","birthYear","debutYear"}); @@ -825,7 +836,7 @@ public class PlayerMapper implements FieldSetMapper<Player> { - public Object mapLine(FieldSet fs) { + public Player mapFieldSet(FieldSet fs) { if(fs == null){ return null; @@ -841,7 +852,6 @@ return player; } - }
@@ -856,8 +866,8 @@ providing a FieldSetMapper that automatically maps fields by matching a field name with a setter on the object using the JavaBean specification. Again using the football example, the - FieldSetMapper configuration looks like the - following: + BeanWrapperFieldSetMapper configuration looks + like the following: <bean id="fieldSetMapper" @@ -932,11 +942,12 @@ - This LineTokenizer will return the same - FieldSet as if a delimiter had been used, - allowing the same approach above to be used such as the - BeanWrapperFieldSetMapper, in a way that is - ignorant of how the actual line was parsed. + Because the FixedLengthLineTokenizer uses + the same LineTokenizer interface as discussed + above, it will return the same FieldSet as if a + delimiter had been used. This allows the same approaches to be used in + handling its output, such as using the + BeanWrapperFieldSetMapper. It should be noted that supporting the above ranges requires a specialized property editor be configured anywhere in the @@ -1041,13 +1052,13 @@
Exception Handling in flat files - There are many scenarios when tokenizing a line that cause + There are many scenarios when tokenizing a line may cause exceptions to be thrown. Many flat files are imperfect and contain - records that aren't formatted correctly. Many users choose to skip the - lines causing these errors, logging out the issue, original line, and - line number, for manual inspection later. (or by another batch job) - For this reason, Spring Batch provides a hierarchy of exceptions for - handling parse exceptions: + records that aren't formatted correctly. Many users choose to skip + these erroneous lines, logging out the issue, original line, and line + number. These logs can later be inspected manually or or by another + batch job. For this reason, Spring Batch provides a hierarchy of + exceptions for handling parse exceptions: FlatFileParseException and FlatFileFormatException. FlatFileParseException is thrown by the @@ -1067,8 +1078,9 @@ FieldSet. However, if the number of column names doesn't match the number of columns found while tokenizing a line the FieldSet can't be created, and a - IncorrectTokenCountException is thrown, which contains the number of - tokens encountered, and the number expected: + IncorrectTokenCountException is thrown, which + contains the number of tokens encountered, and the number + expected: tokenizer.setNames(new String[] {"A", "B", "C", "D"}); @@ -1083,8 +1095,9 @@ - Because the tokenizer was configured with 4 columns, but only - 3 tokens were found in the file, an IncorrectTokenCountException was + Because the tokenizer was configured with 4 column names, but + only 3 tokens were found in the file, an + IncorrectTokenCountException was thrown.
@@ -1092,10 +1105,10 @@ IncorrectLineLengthException Files formatted in a fixed length format have additional - requirements when parsing because unlike a delimited format, each - column must strictly adhere to the width defined for it. If the - total line length doesn't add up to the widest value of this column, - an exception is thrown: + requirements when parsing because, unlike a delimited format, each + column must strictly adhere to its predefined width. If the total + line length doesn't add up to the widest value of this column, an + exception is thrown: tokenizer.setColumns(new Range[] { new Range(1, 5), new Range(6, 10), new Range(11, 15) }); @@ -1132,9 +1145,9 @@ The above example is almost identical to the one before it, - except the tokenizer.setStrict(false) was called. This setting tells - the tokenizer to not enforce line lengths when tokenizing the line. - A FieldSet is now correctly created and + except that tokenizer.setStrict(false) was called. This setting + tells the tokenizer to not enforce line lengths when tokenizing the + line. A FieldSet is now correctly created and returned. However, it will only contain empty tokens for the remaining values.
@@ -1145,17 +1158,18 @@ FlatFileItemWriter Writing out to flat files has the same problems and issues that - reading in from a file must overcome. It must be able to write out in - either delimited or fixed length formats in a transactional + reading in from a file must overcome. A step must be able to write out + in either delimited or fixed length formats in a transactional manner.
LineAggregator Just as the LineTokenizer interface is - necessary to take an item and turn it into a string, file writing must - have a way to aggregate multiple fields into a single string for - writing to a file. In Spring Batch this is the + necessary to take an item and turn it into a + String, file writing must have a way to + aggregate multiple fields into a single string for writing to a file. + In Spring Batch this is the LineAggregator: @@ -1180,9 +1194,9 @@ PassThroughLineAggregator The most basic implementation of the LineAggregator interface - is the PassThroughLineAggregator, which simply assumes that the - object is already a string, or that it's string representation is - acceptable for writing: + is the PassThroughLineAggregator, which + simply assumes that the object is already a string, or that its + string representation is acceptable for writing: public class PassThroughLineAggregator<T> implements LineAggregator<T> { @@ -1196,8 +1210,8 @@ The above implementation is useful if direct control of creating the string is required, but the advantages of a - FlatFileItemWriter, such as transaction and restart support, are - necessary. + FlatFileItemWriter, such as transaction and + restart support, are necessary.
@@ -1205,8 +1219,9 @@ Simplified File Writing Example Now that the LineAggregator interface and - it's most basic implementation, PassThroughLineAggregator, has been - defined the basic flow of writing can be explained: + it's most basic implementation, + PassThroughLineAggregator, have been defined, + the basic flow of writing can be explained: @@ -1251,23 +1266,24 @@ FieldExtractor The above example may be useful for the most basic uses of a - writing to a file. However, most users of the FlatFileItemWriter will - will have a domain object that needs to be written out, and thus must - be converted into a line. In file reading, the following was - required: + writing to a file. However, most users of the + FlatFileItemWriter will will have a domain + object that needs to be written out, and thus must be converted into a + line. In file reading, the following was required: Read one line from the file. - Pass the string line into the LineTokenizer#tokenize() - method, in order to retrieve a - FieldSet + Pass the string line into the + LineTokenizer#tokenize() method, in + order to retrieve a FieldSet - Pass the FieldSet returned from tokenizing to a - FieldSetMapper, returning the result from the ItemReader#read() + Pass the FieldSet returned from + tokenizing to a FieldSetMapper, returning + the result from the ItemReader#read() method @@ -1289,8 +1305,9 @@ Because there is no way for the framework to know which fields - from the object need to be written out, a FieldExtractor must be - written to accomplish the task: + from the object need to be written out, a + FieldExtractor must be written to accomplish + the task of turning the item into an array: public interface FieldExtractor<T> { @@ -1303,7 +1320,7 @@ Implementations of the FieldExtractor interface should create an array from the fields of the provided - object, which can then be written out with a delimited between the + object, which can then be written out with a delimiter between the elements, or as part of a field-width line.
@@ -1313,18 +1330,21 @@ converted to an array, such as a Collection, needs to be written out. For example, a List could be passed through, in which case it only needs to be converted - to an Object array to be written out. For this type of scenario the - PassThroughFieldExtractor can be used. It should be noted, that if - the object passed in is not an array, and not a Collection, then an - Object array containing solely the item will be returned. + to an Object array to be written out. For + this type of scenario the + PassThroughFieldExtractor can be used. It + should be noted, that if the object passed in is not an array, and + not a Collection, then an + Object array containing solely the item will + be returned.
BeanWrapperFieldExtractor As with the BeanWrapperFieldSetMapper - described in the file reading section, it is much preferrable to - configure how to convert an domain object to an object array, rather + described in the file reading section, it is often preferrable to + configure how to convert a domain object to an object array, rather than writing the conversion yourself. The BeanWrapperFieldExtractor provides just this type of functionality: @@ -1349,9 +1369,10 @@ This extractor implementation has only one required property, the names of the fields to map. Just as the BeanWrapperFieldSetMapper needs field names - to map fields on the FieldSet to setters on the provided object, the + to map fields on the FieldSet to setters on + the provided object, the BeanWrapperFieldExtractor needs names to map - to getters for creating an object array. It's worth noting that the + to getters for creating an object array. It is worth noting that the order of the names determines the order of the fields within the array.
@@ -1362,24 +1383,17 @@ The most basic flat file format is one in which all fields are separated by a delimiter. This can be accomplished using a - DelimitedLineAggregator. The example below writes out a simple domain - object that represents a credit to a customer account: + DelimitedLineAggregator. The example below + writes out a simple domain object that represents a credit to a + customer account: public class CustomerCredit { private int id; - private String name; - private BigDecimal credit; - public CustomerCredit(int id, String name, BigDecimal credit) { - this.id = id; - this.name = name; - this.credit = credit; - } - //getters and setters removed for clarity } @@ -1409,19 +1423,19 @@ In this case, the BeanWrapperFieldExtractor described earlier in this chapter is used to turn the name and credit fields within - CustomerCredit into an object array, which is then written out with - commas between each field. + CustomerCredit into an object array, which is + then written out with commas between each field.
Fixed Width File Writing Example - Delimited is not the only type of flat file format, many prefer + Delimited is not the only type of flat file format. Many prefer to use a set width for each column to delineate between fields, which is usually referred to as 'fixed width'. Spring Batch supports this in - file writing via the FormatterLineAggregator. Using the same - CustomerCredit domain object described above, it can be configured as - follows: + file writing via the FormatterLineAggregator. + Using the same CustomerCredit domain object + described above, it can be configured as follows: <bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> @@ -1448,9 +1462,12 @@ - The underlying implementation is built using the same Formatter - added as part of Java 5. Most details on how to configure a formatter - can be found in the javadoc of The underlying implementation is built using the same + Formatter added as part of Java 5. The Java + Formatter is based on the + printf functionality of the C programming + language. Most details on how to configure a formatter can be found in + the javadoc of Formatter.
@@ -1462,18 +1479,18 @@ opens the file if it exists, and throws an exception if it does not. File writing isn't quite so simple. At first glance it seems like a similar straight forward contract should exist for - FlatFileItemWriter, if the file already exists, - throw an exception, if it does not, create it and start writing. + FlatFileItemWriter: if the file already exists, + throw an exception, and if it does not, create it and start writing. However, potentially restarting a Job can cause - issues. In normal restart scenarios, the contract is reversed, if the - file exists start writing to it from the last known good position, if - it does not, throw an exception. However, what happens if the file - name for this job is always the same? In this case, you would want to - delete the file if it exists, unless it's a restart. Because of this - possibility, the FlatFileItemWriter contains - the property, shouldDeleteIfExists. Setting - this property to true will cause an existing file with the same name - to be deleted when the writer is opened. + issues. In normal restart scenarios, the contract is reversed: if the + file exists, start writing to it from the last known good position, + and if it does not, throw an exception. However, what happens if the + file name for this job is always the same? In this case, you would + want to delete the file if it exists, unless it's a restart. Because + of this possibility, the FlatFileItemWriter + contains the property, shouldDeleteIfExists. + Setting this property to true will cause an existing file with the + same name to be deleted when the writer is opened. @@ -1593,7 +1610,7 @@ - FragmentDeserializer - UnMarshalling + FragmentDeserializer - Unmarshalling facility provided by Spring OXM for mapping the XML fragment to an object. @@ -1624,7 +1641,7 @@ (i.e. root element) and the object type to bind. Then, similar to a FieldSet, the names of the other elements that map to fields within the object type are described as key/value pairs in - the map. In the configuration file we can use a spring configuration + the map. In the configuration file we can use a Spring configuration utility to describe the required alias as follows: @@ -1646,17 +1663,17 @@ </bean> - On input the reader reads the XML resource until it recognizes a - new fragment is about to start (by matching the tag name by default). - The reader creates a standalone XML document from the fragment (or at - least makes it appear so) and passes the document to a deserializer - (typically a wrapper around a Spring OXM + On input the reader reads the XML resource until it recognizes + that a new fragment is about to start (by matching the tag name by + default). The reader creates a standalone XML document from the fragment + (or at least makes it appear so) and passes the document to a + deserializer (typically a wrapper around a Spring OXM Unmarshaller) to map the XML to a Java object. - In summary, if you were to see this in scripted code like Java the - injection provided by the spring configuration would look something like - the following: + In summary, this procedure is analogous to the following scripted + Java code which uses the injection provided by the Spring + configuration: StaxEventItemReader xmlStaxEventItemReader = new StaxEventItemReader() @@ -1678,12 +1695,12 @@ CustomerCredit credit = null; while (hasNext) { - credit = xmlStaxEventItemReader.read(); - if (credit == null) { - hasNext = false; - } else { - println trade; - } + credit = xmlStaxEventItemReader.read(); + if (credit == null) { + hasNext = false; + } else { + println trade; + } } @@ -1735,7 +1752,7 @@ To summarize with a Java example, the following code illustrates all of the points discussed, demonstrating the programmatic setup of the - required properties. + required properties: StaxEventItemWriter staxItemWriter = new StaxEventItemWriter() FileSystemResource resource = new FileSystemResource(File.createTempFile("StaxEventWriterOutputSourceTests", "xml")) @@ -1766,9 +1783,9 @@ Multi-File Input It is a common requirement to process multiple files within a single - Step. Assuming the files are all formatted the - same, the MultiResourceItemReader supports this - type of input for both XML and flat file processing. Consider the + Step. Assuming the files all have the same + formatting, the MultiResourceItemReader supports + this type of input for both XML and flat file processing. Consider the following files in a directory: file-1.txt file-2.txt ignored.txt @@ -1779,7 +1796,7 @@ files by using wildcards: - <bean id="multiResourceReader" class="org.springframework.batch.item.SortedMultiResourceItemReader"> + <bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiResourceItemReader"> <property name="resources" value="classpath:data/multiResourceJob/input/file-*.txt" /> <property name="delegate" ref="flatFileItemReader" /> </bean> @@ -1800,9 +1817,9 @@ Like most enterprise application styles, a database is the central storage mechanism for batch. However, batch differs from other application - styles due to the sheer size of the datasets that must be worked with. The - Spring Core JdbcTemplate illustrates this problem - well. If you use JdbcTemplate with a + styles due to the sheer size of the datasets with which the system must + work. The Spring Core JdbcTemplate illustrates this + problem well. If you use JdbcTemplate with a RowMapper, the RowMapper will be called once for every result returned from the provided query. This causes few issues in scenarios where the dataset is small, but the @@ -1851,15 +1868,16 @@ - The example illustrates the basic pattern. Given a 'FOO' table, + This example illustrates the basic pattern. Given a 'FOO' table, which has three columns: ID, NAME, and BAR, select all rows with an ID - greater than one but less than 7. This puts the beginning of the cursor + greater than 1 but less than 7. This puts the beginning of the cursor (row 1) on ID 2. The result of this row should be a completely mapped - Foo object, calling read() again, moves the cursor to the next row, - which is the Foo with an ID of 3. The results of these reads will be - written out after each read, thus allowing the - objects to be garbage collected. (Assuming no instance variables are - maintaining references to them) + Foo object. Calling read() again moves the + cursor to the next row, which is the Foo with an ID of 3. The results of + these reads will be written out after each + read, thus allowing the objects to be garbage + collected (assuming no instance variables are maintaining references to + them).
JdbcCursorItemReader @@ -1871,24 +1889,24 @@ DataSource. The following database schema will be used as an example: - CREATE TABLE CUSTOMER ( - ID BIGINT IDENTITY PRIMARY KEY, - NAME VARCHAR(45), - CREDIT FLOAT -); + CREATE TABLE CUSTOMER ( + ID BIGINT IDENTITY PRIMARY KEY, + NAME VARCHAR(45), + CREDIT FLOAT + ); Many people prefer to use a domain object for each row, so we'll use an implementation of the RowMapper interface to map a CustomerCredit object: - public class CustomerCreditRowMapper implements RowMapper { + public class CustomerCreditRowMapper implements RowMapper { - public static final String ID_COLUMN = "id"; - public static final String NAME_COLUMN = "name"; - public static final String CREDIT_COLUMN = "credit"; + public static final String ID_COLUMN = "id"; + public static final String NAME_COLUMN = "name"; + public static final String CREDIT_COLUMN = "credit"; - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Object mapRow(ResultSet rs, int rowNum) throws SQLException { CustomerCredit customerCredit = new CustomerCredit(); customerCredit.setId(rs.getInt(ID_COLUMN)); @@ -1896,16 +1914,15 @@ customerCredit.setCredit(rs.getBigDecimal(CREDIT_COLUMN)); return customerCredit; - } - + } } Because JdbcTemplate is so familiar to users of Spring, and the JdbcCursorItemReader - shares key interfaces with it, it's useful to see an example of how to - read in this data with JdbcTemplate, in order - to contrast it with the ItemReader. For the - purposes of this example, let's assume there are 1,000 rows in the + shares key interfaces with it, it is useful to see an example of how + to read in this data with JdbcTemplate, in + order to contrast it with the ItemReader. For + the purposes of this example, let's assume there are 1,000 rows in the CUSTOMER database. The first example will be using JdbcTemplate: @@ -1992,7 +2009,8 @@ Gives the Jdbc driver a hint as to the number of rows that should be fetched from the database when more rows are needed by the ResultSet object used - by the ItemReader. By default, no hint is given. + by the ItemReader. By default, no + hint is given. @@ -2007,10 +2025,10 @@ queryTimeout Sets the number of seconds the driver will wait for a - Statement object to execute to the given number of seconds. - If the limit is exceeded, a + Statement object to execute to the + given number of seconds. If the limit is exceeded, a DataAccessEception is thrown. - (consult your driver vendor documentation for + (Consult your driver vendor documentation for details). @@ -2018,23 +2036,24 @@ verifyCursorPosition Because the same ResultSet - held by the ItemReader is passed to the - RowMapper, it's possible for users to - call ResultSet.next() themselves, which could cause issues - with the reader's internal count. Settings this value to - true will cause an exception to be thrown if the cursor - position is not the same after the - RowMapper call as it was - before. + held by the ItemReader is passed to + the RowMapper, it is possible for + users to call ResultSet.next() + themselves, which could cause issues with the reader's + internal count. Setting this value to true will cause an + exception to be thrown if the cursor position is not the + same after the RowMapper call as it + was before. saveState Indicates whether or not the reader's state should be - saved in the ExecutionContext provided by - ItemStream#update(ExecutionContext) The default value is - false. + saved in the ExecutionContext + provided by + ItemStream#update(ExecutionContext) + The default value is false. @@ -2044,9 +2063,9 @@ supports setting the absolute row on a ResultSet. It is recommended that this is set to true for Jdbc drivers that supports - ResultSet.absolute() as it may improve performance, - especially if a step fails while working with a large data - set. + ResultSet.absolute() as it may + improve performance, especially if a step fails while + working with a large data set. @@ -2058,8 +2077,8 @@ which is the default, then the cursor will be opened using its own connection and will not participate in any transactions started for the rest of the step processing. If - you set this flag to true then you must wrap the DataSource - in an + you set this flag to true then you must wrap the + DataSource in an ExtendedConnectionDataSourceProxy to prevent the connection from being closed and released after each commit. When you set this option to true then the @@ -2080,26 +2099,25 @@ HibernateCursorItemReader Just as normal Spring users make important decisions about - whether or not to use ORM solutions, which affects whether or not they + whether or not to use ORM solutions, which affect whether or not they use a JdbcTemplate or a HibernateTemplate, Spring Batch users have the same options. HibernateCursorItemReader is the Hibernate implementation of the cursor technique. Hibernate's usage in batch has been fairly controversial. This has largely been because - hibernate was originally developed to support online application + Hibernate was originally developed to support online application styles. However, that doesn't mean it can't be used for batch processing. The easiest approach for solving this problem is to use a StatelessSession rather than a standard session. This removes all of the caching and dirty checking hibernate - employs that can cause issues when using it in a batch scenario. For - more information on the differences between stateless and normal - hibernate sessions, refer to the documentation of your specific - hibernate release. The - HibernateCursorItemReader allows you to declare - an HQL statement and pass in a SessionFactory, - which will pass back one item per call to - read in the same basic fashion as the - JdbcCursorItemReader. Below is an example + employs that can cause issues in a batch scenario. For more + information on the differences between stateless and normal hibernate + sessions, refer to the documentation of your specific hibernate + release. The HibernateCursorItemReader allows + you to declare an HQL statement and pass in a + SessionFactory, which will pass back one item + per call to read in the same basic fashion as + the JdbcCursorItemReader. Below is an example configuration using the same 'customer credit' example as the JDBC reader: @@ -2226,7 +2244,7 @@ similar to the Hibernate StatelessSession so we have to use other features provided by the JPA specification. Since JPA supports paging, this is a natural choice when it comes to using - JPA for batch processing. After each page is read the entities will + JPA for batch processing. After each page is read, the entities will become detached and the persistence context will be cleared in order to allow the entities to be garbage collected once the page is processed. @@ -2504,8 +2522,8 @@ input data to indicate whether or not it has been processed. When a particular record is being read (or written out) the processed flag is flipped from false to true. The SQL statement can then contain an extra - statement in the where clause, such as: "where PROCESSED_IND = false", - thereby insuring that only unprocessed records will be returned in the + statement in the where clause, such as "where PROCESSED_IND = false", + thereby ensuring that only unprocessed records will be returned in the case of a restart. In this scenario, it is preferable to not store any state, such as the current row number, since it will be irrelevant upon restart. For this reason, all readers and writers include the 'saveState' @@ -2521,12 +2539,12 @@ <property name="saveState" value="false" /> <property name="sql"> <value> - SELECT games.player_id, games.year_no, SUM(COMPLETES), - SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), - SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), - SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) - from games, players where players.player_id = - games.player_id group by games.player_id, games.year_no + SELECT games.player_id, games.year_no, SUM(COMPLETES), + SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), + SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), + SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) + from games, players where players.player_id = + games.player_id group by games.player_id, games.year_no </value> </property> </bean> @@ -2535,7 +2553,7 @@ The ItemReader configured above will not make any entries in the ExecutionContext for any - executions it participates in. + executions in which it participates.
@@ -2583,9 +2601,9 @@ - This very simple class takes a list of items, and returns one at a - time, removing it from the list. When the list empty, it returns null, - thus satisfying the most basic requirements of an + This very simple class takes a list of items, and returns them one + at a time, removing each from the list. When the list is empty, it + returns null, thus satisfying the most basic requirements of an ItemReader, as illustrated below: List<String> items = new ArrayList<String>(); @@ -2608,16 +2626,15 @@ goes out, and processing begins again, the ItemReader must start at the beginning. This is actually valid in many scenarios, but it is sometimes preferable that - a batch job starts off at where it left off. The key discriminant is - often whether the reader is stateful or stateless. A stateless reader - does not need to worry about restartability, but a stateful one has to - try and reconstitute its last known state on restart. For this reason, - we recommend that you keep custom readers stateless as far as - possible, so you don't have to worry about restartability. + a batch job starts where it left off. The key discriminant is often + whether the reader is stateful or stateless. A stateless reader does + not need to worry about restartability, but a stateful one has to try + and reconstitute its last known state on restart. For this reason, we + recommend that you keep custom readers stateless if possible, so you + don't have to worry about restartability. - If you do need to store state, then in Spring Batch, this is - implemented with the ItemStream - interface: + If you do need to store state, then the + ItemStream interface should be used: public class CustomItemReader<T> implements ItemReader<T>, ItemStream { @@ -2648,22 +2665,23 @@ } } - public void close(ExecutionContext executionContext) throws ItemStreamException {} - public void update(ExecutionContext executionContext) throws ItemStreamException { executionContext.putLong(CURRENT_INDEX, new Long(currentIndex).longValue()); }; + + public void close() throws ItemStreamException {} } - On each call to ItemStream + On each call to the ItemStream update method, the current index of the ItemReader will be stored in the provided ExecutionContext with a key of 'current.index'. When the ItemStream open method is called, the ExecutionContext is - checked to see if it contains an entry with that key, and if so the - current index is moved to that location. This is a fairly trivial - example, but it still meets the general contract: + checked to see if it contains an entry with that key. If the key is + found, then the current index is moved to that location. This is a + fairly trivial example, but it still meets the general + contract: ExecutionContext executionContext = new ExecutionContext(); ((ItemStream)itemReader).open(executionContext); @@ -2686,14 +2704,17 @@ It is also worth noting that the key used within the ExecutionContext should not be trivial. That is because the same ExecutionContext is used for - all ItemStreams within a Step. In most cases, - simply prepending the key with the class name should be enough to - guarantee uniqueness. However, in the rare cases where two of the same - type of ItemStream are used in the same step - (which can happen if two files are need for output) then a more unique - name will be needed. For this reason, many of the Spring Batch - ItemReader and ItemWriters have a setName() property that allows this - key name to be overridden. + all ItemStreams within a + Step. In most cases, simply prepending the key + with the class name should be enough to guarantee uniqueness. However, + in the rare cases where two of the same type of + ItemStream are used in the same step (which can + happen if two files are need for output) then a more unique name will + be needed. For this reason, many of the Spring Batch + ItemReader and + ItemWriter implementations have a + setName() property that allows this key name + to be overridden.
@@ -2704,8 +2725,9 @@ in many ways to the ItemReader example above, but differs in enough ways as to warrant its own example. However, adding restartability is essentially the same, so it won't be covered in this - example. As with the ItemReader example, a List - will be used in order to keep the example as simple as possible: + example. As with the ItemReader example, a + List will be used in order to keep the example as + simple as possible: public class CustomItemWriter<T> implements ItemWriter<T> { @@ -2743,9 +2765,8 @@ ItemStream as well as ItemWriter. Remember also that the client of the writer needs to be aware of the ItemStream, - so you may need to register it with a factory bean (e.g. one of the - StepFactoryBean implementations in Spring Batch - Core). + so you may need to register it as a stream in the configuration + xml.