diff --git a/docs/.project b/docs/.project index 74ee6df8e..e0d2794ab 100644 --- a/docs/.project +++ b/docs/.project @@ -1,17 +1,29 @@ - - - docs - - - - - - org.eclipse.jdt.core.javabuilder - - - - - - org.eclipse.jdt.core.javanature - - + + + docs + @key 32303037303533312D313020646F63732F77786C756E64 + + + + + org.eclipse.jdt.core.javabuilder + + + + + com.soyatec.additional.Builder + + + + + org.codehaus.groovy.eclipse.groovyBuilder + + + + + + org.eclipse.jdt.core.javanature + com.soyatec.additional.Nature + org.codehaus.groovy.eclipse.groovyNature + + diff --git a/docs/src/site/docbook/reference/infrastructure.xml b/docs/src/site/docbook/reference/infrastructure.xml index 5f4500874..fb8caa5e8 100644 --- a/docs/src/site/docbook/reference/infrastructure.xml +++ b/docs/src/site/docbook/reference/infrastructure.xml @@ -409,9 +409,16 @@ } - Now by adding an PlayerFieldSetMapper that creates a Player for - each FieldSet Item and accessing the data within a record, we have - mapping logic that looks like the following: + We can now inject a fieldset mapper into the ListPlayerReader, + for example, that can take advantage of a PlayerFieldSetMapper for + transforming a line that consists of one item separated by delimiters + into a domain object - Player in this + case. We inject programmatically by invoking the following: + + itemReader.setFieldSetMapper(fieldSetMapper); + + and define the fieldSetMapper class in the following + declaration: protected static class PlayerFieldSetMapper implements FieldSetMapper { @@ -429,23 +436,77 @@ } } + + There is one additional preference that can be used that is + similar in function to the jdbc fieldset. The names of the fields can + be injected into the Tokenizer to increase the readability of the + mapping function. We can expose this behavior by adding the following. + First, we tell the tokenizer what the names of the fields in the + fieldset are: + + + tokenizer.setNames(new String[] {"ID", "lastName","firstName","position","birthYear","debutYear"}); + + + and provide a mapper that uses this information as + follows: + + + public class PlayerMapper implements FieldSetMapper { + public Object mapLine(FieldSet fs) { + + if(fs == null){ + return null; + } + + Player player = new Player(); + player.setID(fs.readString("ID")); + player.setLastName(fs.readString("lastName")); + player.setFirstName(fs.readString("firstName")); + player.setPosition(fs.readString("position")); + player.setDebutYear(fs.readInt("debutYear")); + player.setBirthYear(fs.readInt("birthYear")); + + return player; + } + + } +
- - Configuring and Using LineTokenizers - - - To separate the structure of the raw records, LineTokenizer, or - one of it subclasses, is used to parse data obtained from the an + We have already mentioned tokenizers in the preceding section. + In order to separate the structure of the raw records, LineTokenizer, + or one of it subclasses, is used to parse data obtained from the an ItemReader, most typically a file. Flat File Item Readers, as mentioned above, typically process records in two forms, fixed and delimited. A fixed length input record is where the fields are - assigned fixed locations within a line of a file. An example would be: - + assigned fixed locations within a line of a file. In the preceding + section we gave an example of a comma delimited record layout as + follows: + + + AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996 + AbduRa00,Abdullah,Rabih,rb,1975,1999 + AberWa00,Abercrombie,Walter,rb,1959,1982 + AbraDa00,Abramowicz,Danny,wr,1945,1967 + AdamBo00,Adams,Bob,te,1946,1969 + AdamCh00,Adams,Charlie,wr,1979,2003 + + + Here, rather than fix fields to positions within a line or + record, fields are simply separated by some predefined symbol. We + illustrate with commas in this example as it is a very familiar format + to developers who have experience with csv file formats exported from + spreadsheets. + + On the other hand a fixed record format might look like the + following: + + 12345678901234567890123456789012345678901234567890 AbduKa00Abdul-Jabbar Karim rb19741996 AbduRa00Abdullah Rabih rb19751999 @@ -455,36 +516,11 @@ AdamCh00Adams Charlie wr19792003 - - One can see that each field in the record starts at the same position in the record. It's fixed from a starting position to an end of position for each field, which may include a user defined end position such as EOL. - On the other hand a delimited record format might look like the following: - - - - AbduKa00,Abdul-Jabbar,Karim,rb,1974,1996 - AbduRa00,Abdullah,Rabih,rb,1975,1999 - AberWa00,Abercrombie,Walter,rb,1959,1982 - AbraDa00,Abramowicz,Danny,wr,1945,1967 - AdamBo00,Adams,Bob,te,1946,1969 - AdamCh00,Adams,Charlie,wr,1979,2003 - - - - - - Here, rather than fix fields to positions within a line or - record, fields are simply separated by some predefined symbol. We - illustrate with commas in this example as it is a very familiar format - to developers who have experience with to exported csv files from - spreadsheets. - - - Neither of these formats are particularly self describing but are still very much in use in flat file exchanges between system interfaces. Both formats share in common the requirement to read in a @@ -495,51 +531,22 @@ second dependency is a LineTokenizer, which will be discused below. - - The interface for a LineTokenizer is very simple, given a string; it will return a FieldSet that wraps the results from tokenizing the provided string. The tokens are created through a LineTokenizer and a FieldSetMapper is used to map the - FieldSet to an object. The framework provides a - few convenience classes, the FieldSetInputSource - and the SimpleFlatFileInputSource. They provide a - convenient way to read the FieldSet. The FieldSet - is configured as a property for an Item Provider, which wraps an Input - Source. You can see this in the following example: - - + FieldSet to an object. The + FieldSet is configured as a property for an Item + Provider, which wraps an Input Source. Programatically you can inject + the tokenizer as demonstrated previously on the ItemReader via the + method call toYou can see this in the following example: And, as you can see, the field names will get passed in the the mapper. The actual mapping provided by the developer would then look as simple as: - - - - -public class PlayerMapper implements FieldSetMapper { - public Object mapLine(FieldSet fs) { - if(fs == null){ - return null; - } - - Player player = new player(); - player.setID(fs.readString("ID")); - player.setLastName(fs.readString("lastName")); - player.setFirstName(fs.readString("firstName")); - player.setPosition(fs.readString("position")); - player.setDebutYear(fs.readInt("debutYear")); - player.setBirthYear(fs.readInt("birthYear")); - - return player; - } -} - - - - + itemReader.setTokenizer(tokenizer);
@@ -563,10 +570,11 @@ public class PlayerMapper implements FieldSetMapper { - Reading flat files in the Spring Batch framework is facilitated by - the class FlatFileItemReader, which - provides basic functionality for reading and parsing flat files. In - addition, there are default implementations of the (XML data files will be discussed separately). Reading flat files + in the Spring Batch framework is facilitated by the class FlatFileItemReader, which provides basic + functionality for reading and parsing flat files. In addition, there are + default implementations of the Skippable and ItemStream interfaces that solve the majority of file processing needs. @@ -584,20 +592,19 @@ public class PlayerMapper implements FieldSetMapper { converted to distinct fields. We explored fieldSetMapper and tokenizer while reviewing how to create a - custom ItemReader. We'll revisit these properties in light of how we - use them with the FlatFileItemReader. - In addition, we'll explore integration with the file system via the - resource property. The resource - property represents a Spring Core Resource. Documentation explaining how to - create beans of this type can be found in ItemReader. We'll revisit + these properties in light of how we use them with the FlatFileItemReader. In addition, we'll explore + integration with the file system via the resource property. The + resource property represents a Spring + Core Resource. Documentation + explaining how to create beans of this type can be found in Spring Framework, Chapter 4.Resources. Therefore, this guide will not go into the details of creating Resource objects except to make a couple of points on the locating files to process within a batch environment. - Tokenizers and field set mappers will be discussed a bit - later.> + Tokenizers and field set mappers will be discussed a bit later. As mentioned, the location of the file is defined by the resource property. There are only a few methods exposed through a @@ -707,62 +714,62 @@ public class PlayerMapper implements FieldSetMapper { --> -
- XML Item Readers and Writers - - Spring Batch provides transactional infrastructure for both reading - XML records and mapping them to Java objects as well as writing Java - objects as XML records. - - StAX API is used for I/O as other standard XML APIs do not fit batch - processing requirements (DOM loads the whole input into memory at once and - SAX controls the parsing process allowing the user only to provide - callbacks). - - Spring Batch is not tied to any particular OXM technology. Typical - use is to delegate OXM to Spring WS which provides uniform abstraction for - the most popular OXM technologies. However dependency on Spring WS is - optional and you can choose to implement Spring Batch specific interfaces - if desired. - - Lets take a closer look how XML input and output work in batch. It - is assumed the XML resource is a collection of 'fragments' corresponding - to individual records. Note that OXM tools are designed to work with - standalone XML documents rather than XML fragments cut out of an XML - document, therefore the Spring Batch infrastructure needs to work around - this fact (as described below). - - On input the reader reads the XML resource until it recognizes a new - fragment is about to start (by matching the tag name by default). The - reader creates a standalone XML document from the fragment (or at least - makes it appear so) and passes the document to a deserializer (typically a - wrapper around Spring WS Unmarshaller) to map the XML to a Java - object. - - Output works symetrically to input. Java object is passed to a - serializer (typically a wrapper around Spring WS Marshaller) which writes - to output using a custom event writer that filters the StartDocument and - EndDocument events produced for each fragment by the OXM tools. - - For example configuration of XML input and output see the sample - xmlStaxJob. //TODO inline the example once it is not subject to change + - show sample input file
- -
- Item Writers + XML Item Readers and Writers - The Item Writers are similar in functionality to the input source - with the exception that the operations are reversed. They still need to - be located, opened and closed but they differ in the case that we write - to output sources. In the case of databases or queues these may be - inserts, updates or sends. The format of the serialization of the output - source is specific for every batch job. + Spring Batch provides transactional infrastructure for both + reading XML records and mapping them to Java objects as well as writing + Java objects as XML records. + + StAX API is used for I/O as other standard XML APIs do not fit + batch processing requirements (DOM loads the whole input into memory at + once and SAX controls the parsing process allowing the user only to + provide callbacks). + + Spring Batch is not tied to any particular OXM technology. Typical + use is to delegate OXM to Spring WS which provides uniform abstraction + for the most popular OXM technologies. However dependency on Spring WS + is optional and you can choose to implement Spring Batch specific + interfaces if desired. + + Lets take a closer look how XML input and output work in batch. It + is assumed the XML resource is a collection of 'fragments' corresponding + to individual records. Note that OXM tools are designed to work with + standalone XML documents rather than XML fragments cut out of an XML + document, therefore the Spring Batch infrastructure needs to work around + this fact (as described below). + + On input the reader reads the XML resource until it recognizes a + new fragment is about to start (by matching the tag name by default). + The reader creates a standalone XML document from the fragment (or at + least makes it appear so) and passes the document to a deserializer + (typically a wrapper around Spring WS Unmarshaller) to map the XML to a + Java object. + + Output works symetrically to input. Java object is passed to a + serializer (typically a wrapper around Spring WS Marshaller) which + writes to output using a custom event writer that filters the + StartDocument and EndDocument events produced for each fragment by the + OXM tools. + + For example configuration of XML input and output see the sample + xmlStaxJob. //TODO inline the example once it is not subject to change + + show sample input file
- +
+ Item Writers + + The Item Writers are similar in functionality to the input source + with the exception that the operations are reversed. They still need to be + located, opened and closed but they differ in the case that we write to + output sources. In the case of databases or queues these may be inserts, + updates or sends. The format of the serialization of the output source is + specific for every batch job. +
+
SQL Sources