diff --git a/docs/models/Batch Presentation Diagrams.vsd b/docs/models/Batch Presentation Diagrams.vsd index 2a8224235..d408a1bd5 100644 Binary files a/docs/models/Batch Presentation Diagrams.vsd and b/docs/models/Batch Presentation Diagrams.vsd differ diff --git a/docs/src/site/docbook/reference/appendix.xml b/docs/src/site/docbook/reference/appendix.xml index 26a0def33..3b0d8a271 100644 --- a/docs/src/site/docbook/reference/appendix.xml +++ b/docs/src/site/docbook/reference/appendix.xml @@ -25,12 +25,14 @@ - AbstractBufferedItemReaderItemStream + AbstractItemCountingItemStreamItemReader java.lang.Object - Abstract base class that provides bufferring - functionality for any subclass. + Abstract base class that provides basic + restart capabilities by counting the number of items returned from + an ItemReader. @@ -42,15 +44,6 @@ time - - ValidatingItemReader - - java.lang.Object - - A simple extension of DelegatingItemReader - that provides for validation before returning input. - - ItemReaderAdapter @@ -75,18 +68,6 @@ role="bold">END_RECORD - - DelegatingItemReader - - java.lang.Object - - Extends AbstractMethodInvokingDelegator, which - enables dynamically calling of a custom method of the injected - object. Provides a convenient API for dynamic method invocation - shielding subclasses from the low-level details and exception - handling. - - FlatFileItemReader @@ -153,6 +134,26 @@ received through the injected JmsOperations receive() method + + + JpaPagingItemReader + + java.lang.Object + + Given a JPQL statement, pages through the + rows, such that large datasets can be read without running out of + memory + + + + JdbcPagingItemReader + + java.lang.Object + + Given a SQL statement, pages through the rows, + such that large datasets can be read without running out of + memory + @@ -220,16 +221,6 @@ role="bold">ItemWriter objects - - DelegatingItemWriter - - java.lang.Object - - Wraps ItemWriter and is BeanAware allowing it - to respond to Spring Bean events like - afterPropertiesSet(). - - ItemWriterAdapter @@ -251,17 +242,6 @@ name - - ItemTransformerItemWriter - - java.lang.Object - - Extends DelegatingItemWriter by defining its - doProcess method to call an - injected ItemTransformer - - FlatFileItemWriter @@ -270,8 +250,8 @@ Attempts to convert the item to a String, Collection or array using an injected - Coverter and then recurses. See - [HOWTO - Write to a File] + Coverter and then + recurses. @@ -282,8 +262,19 @@ This item writer is hibernate session aware and handles some transaction-related work that a non-"hibernate aware" item writer would not need to know about and then delegates - to another item writer to do the actual writing. See [HOWTO - - Write to a Database] + to another item writer to do the actual writing. + + + + JpaAwareItemWriter + + java.lang.Object + + This item writer is JPA EntityManager aware + and handles some transaction-related work that a non-"jpa aware" + ItemWriter would not need to know about and + then delegates to another writer to do the actual writing. + @@ -294,7 +285,7 @@ Uses an ObjectToXmlSerializer implementation to convert each item to XML and then writes it to an XML file using - StAX. See [HOWTO - Write to a File] + StAX. diff --git a/docs/src/site/docbook/reference/core.xml b/docs/src/site/docbook/reference/core.xml index e63affabb..1abc19220 100644 --- a/docs/src/site/docbook/reference/core.xml +++ b/docs/src/site/docbook/reference/core.xml @@ -605,9 +605,9 @@ Step interface. Two step implementation classes are available in the Spring Batch framework, and they are each discussed in detail in Chatper 4 of this guide. For most situations, the - StepHandlerStep implementation is sufficient, - but for situations where only one call is needed, such as a stored - procedure call or a wrapper around existing script, a + StepHandlerStep implementation is sufficient, but + for situations where only one call is needed, such as a stored procedure + call or a wrapper around existing script, a TaskletStep may be a better option. @@ -1040,6 +1040,19 @@ various implementations can be found in Chapter 3. +
+ Item Processor + + ItemProcessor is an abstraction that + represents the business processing of an item. While the + ItemReader reads one item, and the + ItemWriter writes them, the + ItemProcessor provides access to transform or apply + other business processing. If while processing the item it's determined + that it's not valid, returning null indicates that it should not be + written out. +
+
Tasklet @@ -1050,4 +1063,4 @@ logic that is not natural to split into read-(transform)-write phases, such as invoking a system command or a stored procedure.
- + \ No newline at end of file diff --git a/docs/src/site/docbook/reference/readersAndWriters.xml b/docs/src/site/docbook/reference/readersAndWriters.xml index 989e30e7a..b88f6bce1 100644 --- a/docs/src/site/docbook/reference/readersAndWriters.xml +++ b/docs/src/site/docbook/reference/readersAndWriters.xml @@ -52,11 +52,8 @@ public interface ItemReader<T> { - T read() throws Exception, UnexpectedInputException, NoWorkFoundException, ParseException; + T read() throws Exception, UnexpectedInputException, ParseException; - void mark() throws MarkFailedException; - - void reset() throws ResetFailedException; } @@ -68,13 +65,11 @@ (i.e. Trade, Foo, etc) but there is no requirement in the contract to do so. - The mark and reset - methods are important due to the transactional nature of batch processing. - Mark() will be called before reading begins. Calling - reset at anytime will position the - ItemReader to its position when - mark was last called. The semantics are very - similar to java.io.Reader. + It is expected that implementations of the + ItemReader interface will be forward only. However, + if the underlying resource is transactional (such as a JMS queue) thehn + calling read may return the same logical item on subsequent calls in a + rollback scenario. It is also worth noting that a lack of items to process by an ItemReader will not cause an exception to be @@ -99,31 +94,22 @@ public interface ItemWriter<T> { - void write(T item) throws Exception; + void write(List<? extends T> items) throws Exception; - void flush() throws FlushFailedException; - - void clear() throws ClearFailedException; } As with read on ItemReader, write provides the basic contract of ItemWriter, it will attempt - to write out the item passed in as long as it is open. As with - mark and reset, - flush and clear are - necessary due to the transactional nature of batch processing. Because it - is generally expected that items will be 'batched' together into a chunk, - and then output, it is expected that an ItemWriter - will perform some type of buffering. flush will - empty the buffer by writing the items out, whereas - clear will simply throw the contents of the - buffer away. In most cases, a Step implementation - will call flush before a commit and - clear in case of rollback. It is expected that - implementations of the Step interface will call - these methods. + to write out the list of items passed in as long as it is open. Because it + is generally expected that items will be 'batched' together into a chunk + and then output, the interface accepts a list, rather than an item by + itself. After writing out the list, any flushing that may be necessary can + be performed before returning from the write method. For example, if + writing to a Hibernate DAO, multiple calls to write can be made, one for + each item. The writer can then call close on the hibernate Session before + returning.
@@ -324,6 +310,18 @@ +
+ LineMapper + + As with RowMapper, which takes a low + level construct such as ResultSet and returns an Object, + flat file procesing requires the same construct to convert a String + line into an Object:public interface LineMapper<T> { + + T mapLine(String line, int lineNumber) throws Exception; +} +
+
FieldSetMapper @@ -2315,10 +2313,6 @@ } return null; } - - public void mark() throws MarkFailedException { }; - - public void reset() throws ResetFailedException { }; } This very simple class takes a list of items, and returns one at a @@ -2337,83 +2331,6 @@ assertEquals("3", itemReader.read()); assertNull(itemReader.read()); -
- Making the <classname>ItemReader</classname> - transactional - - This most basic ItemReader will work, but - what happens if the transaction needs to be rolled back? This will - usually caused by an error in the ItemWriter, since the ItmReader - generally won't do anything that invalidates the transaction, but - without supporting it, there would be erroneous results. ItemReaders - are notified about rollbacks via the mark and - reset methods. In the example above they're - empty, but we'll need to add code to them in order to support the - rollback scenario: - - public class CustomItemReader<T> implements ItemReader<T>{ - - List<T> items; - int currentIndex = 0; - int lastMarkedIndex = 0; - - public CustomItemReader(List<T> items) { - this.items = items; - } - - public T read() throws Exception, UnexpectedInputException, - NoWorkFoundException, ParseException { - - if (currentIndex < items.size()) { - return items.get(currentIndex++); - } - return null; - } - - public void mark() throws MarkFailedException { - lastMarkedIndex = currentIndex; - }; - - public void reset() throws ResetFailedException { - currentIndex = lastMarkedIndex; - }; - } - - The CustomItemReader has now been - modified to keep track of where it is currently, and where it was when - mark() was last called. This allows the new - ItemReader to fulfill the basic contract that - calling reset returns the - ItemReader to the state it was in when - mark was last called: - - //Assume same setup as last example, a list with "1", "2", and "3" - itemReader.mark(); - assertEquals("1", itemReader.read()); - assertEquals("2", itemReader.read()); - itemReader.reset(); - assertEquals("1", itemReader.read()); - - In most real world scenarios, there will likely be some kind of - underlying resource that will require tracking. In the case of a file, - mark will hold the current location within - the file, and reset will move it back. The - JdbcCursorItemReader, for example, holds on to - the current row number, and on reset moves the cursor back by calling - the ResultSet - absolute method, which moves the current - cursor to the row number supplied. The - CustomItemReader now completely adheres to the - entire ItemReader contract. - read will return the appropriates items, - returning null when empty, and reset returns - the ItemReader back to it's state as of the - last call to mark, allowing for correct - support of a rollback. (It's assumed a Step - implementation will call mark and - reset). -
-
Making the <classname>ItemReader</classname> restartable @@ -2434,11 +2351,10 @@ implemented with the ItemStream interface: - public class CustomItemReader<T> implements ItemReader<T>, ItemStream{ + public class CustomItemReader<T> implements ItemReader<T>, ItemStream { List<T> items; int currentIndex = 0; - int lastMarkedIndex = 0; private static final String CURRENT_INDEX = "current.index"; public CustomItemReader(List<T> items) { @@ -2446,32 +2362,22 @@ } public T read() throws Exception, UnexpectedInputException, - NoWorkFoundException, ParseException { + ParseException { if (currentIndex < items.size()) { return items.get(currentIndex++); } - + return null; } - public void mark() throws MarkFailedException { - lastMarkedIndex = currentIndex; - } - - public void reset() throws ResetFailedException { - currentIndex = lastMarkedIndex; - } - public void open(ExecutionContext executionContext) throws ItemStreamException { - if(executionContext.containsKey(CURRENT_INDEX)){ currentIndex = new Long(executionContext.getLong(CURRENT_INDEX)).intValue(); } else{ currentIndex = 0; - lastMarkedIndex = 0; - } + } } public void close(ExecutionContext executionContext) throws ItemStreamException {} @@ -2534,65 +2440,19 @@ example. As with the ItemReader example, a List will be used in order to keep the example as simple as possible: - public class CustomItemWriter<T> implements ItemWriter<T>{ + public class CustomItemWriter<T> implements ItemWriter<T> { - List<T> output = new ArrayList<T>(); + List<T> output = TransactionAwareProxyFactory.createTransactionalList(); - public void write(T item) throws Exception { - output.add(item); + public void write(List<? extends T> items) throws Exception { + output.addAll(items); } - public void clear() throws ClearFailedException { } - - public void flush() throws FlushFailedException { } - } - -
- Making the <classname>ItemReader</classname> - transactional - - The example is extremely simple, but it's worth showing to - illustrate an ItemWriter that doesn't respond - to rollbacks and commits (i.e. clear and - flush). If your potential writer is such that - it doesn't need to care about rollback or commit, likely because it's - writing to a database, then there is little value to the - ItemWriter interface in that scenario other - than using it to meet another class's requirement for an - implementation of the ItemWriter interface. In - that case, the ItemWriterAdapter would be a - better solution. However, if it does need to be transactional, then - flush and clear - should be implemented to allow for a buffering solution: - - public class CustomItemWriter<T> implements ItemWriter<T>{ - - List<T> output = new ArrayList<T>(); - List<T> buffer = new ArrayList<T>(); - - public void write(T item) throws Exception { - buffer.add(item); - } - - public void clear() throws ClearFailedException { - buffer.clear(); - } - - public void flush() throws FlushFailedException { - for(T t:buffer){ - output.add(t); - } + public List<T> getOutput() { + return output; } } - The ItemWriter buffers all output, only - writing to the actual output (in this case by added to a list) when - the ItemWriter flush() - method is called. The contents of the buffer are thrown away when - ItemWriter clear() is - called. -
-
Making the <classname>ItemWriter</classname> restartable