diff --git a/docs/src/site/docbook/reference/common-patterns.xml b/docs/src/site/docbook/reference/common-patterns.xml index b834dfbc7..390df2bef 100644 --- a/docs/src/site/docbook/reference/common-patterns.xml +++ b/docs/src/site/docbook/reference/common-patterns.xml @@ -258,7 +258,7 @@ in the event of a restart. In order to make this class restartable, the ItemStream interface should be implemented along with the methods open and - update: + update: private static final String TOTAL_AMOUNT_KEY = "total.amount"; @@ -306,12 +306,11 @@ + scale="80" width="" /> - @@ -326,12 +325,11 @@ + scale="80" width="66%" /> - diff --git a/docs/src/site/docbook/reference/domain.xml b/docs/src/site/docbook/reference/domain.xml index b57cbd98a..47bb0bc52 100644 --- a/docs/src/site/docbook/reference/domain.xml +++ b/docs/src/site/docbook/reference/domain.xml @@ -21,8 +21,8 @@ - simple and default implementations that allow for quick adoption - and ease of use out-of-the-box + simple and default implementations that allowed for quick + adoption and ease of use out-of-the-box @@ -47,13 +47,13 @@ + format="PNG" scale="70" /> + format="PNG" scale="85" /> Figure 2.1: Batch Stereotypes @@ -63,7 +63,7 @@ language of batch. A Job has one to many steps, which has exactly one ItemReader, ItemProcessor, and ItemWriter. A job needs to be launched (JobLauncher), and meta data about the currently running process needs to be - stored (JobRepository). + stored (JobRepository)
Job @@ -78,12 +78,12 @@ + scale="90" /> - + @@ -133,11 +133,11 @@ case of this job, there will be one logical JobInstance per day. For example, there will be a January 1st run, and a January 2nd run. If the January 1st run fails the - first time and is run again the next day, it is still the January 1st - run. (Usually this corresponds with the data it is processing as well, - meaning the January 1st run processes data for January 1st, etc). - Therefore, each JobInstance can have multiple - executions (JobExecution is discussed in more + first time and is run again the next day, it's still the January 1st + run. (Usually this corresponds with the data its processing as well, + meaning the January 1st run processes data for January 1st, etc) That is + to say, each JobInstance can have multiple + executions. (JobExecution is discussed in more detail below) and only one JobInstance corresponding to a particular Job can be running at a given time. The definition of a JobInstance @@ -151,11 +151,11 @@ likely be a business decision, it is left up to the ItemReader to decide. What using the same JobInstance will determine, however, is whether - or not the 'state' (i.e. the ExecutionContext, - which is discussed below) from previous executions will be used. Using a - new JobInstance will mean 'start from the - beginning' and using an existing instance will generally mean 'start - from where you left off'. + or not the 'state' (i.e. The ExecutionContext, which is discussed below) + from previous executions will be used. Using a new + JobInstance will mean 'start from the beginning' + and using an existing instance will generally mean 'start from where you + left off'.
@@ -165,20 +165,21 @@ differs from Job, the natural question to ask is: "how is one JobInstance distinguished from another?" The answer is: JobParameters. - JobParameters is a set of parameters used to - start a batch job. They can be used for identification or even as + JobParameters are any set of parameters used to + start a batch job, which can be used for identification or even as reference data during the run: + scale="90" /> + fileref="images/job-stereotypes-parameters.png" + scale="80" /> @@ -203,10 +204,10 @@ will not be considered complete unless the execution completes successfully. Using the EndOfDay Job described above as an example, consider a JobInstance for - 01-01-2008 that failed the first time it was run. If it is run again + 01-01-2008 that failed the first time it was run. If it is ran again, with the same job parameters as the first run (01-01-2008), a new - JobExecution will be created. However, there will - still be only one JobInstance. + JobExecution will be created. However, there will still be only one + JobInstance. A Job defines what a job is and how it is to be executed, and JobInstance is a purely @@ -225,9 +226,9 @@ status A BatchStatus object that - indicates the status of the execution. While running, it's - BatchStatus.STARTED, if it fails, it's BatchStatus.FAILED, and - if it finishes successfully, it's BatchStatus.COMPLETED + indicates the status of the execution. While it's running, it's + BatchStatus.STARTED, if it fails it's BatchStatus.FAILED, and if + it finishes successfully it's BatchStatus.COMPLETED @@ -258,19 +259,18 @@ createTime A java.util.Date representing the - current system time when the JobExecution - was first persisted. The job may not have been started yet (and - thus has no start time), but it will always have a createTime, - which is required by the framework for managing job level - ExecutionContexts. + current system time when the JobExecution was first persisted. + The job may not have been started yet (and thus has no start + time), but it will always have a createTime, which is required + by the framework for managing job level + ExecutionContexts. lastUpdated A java.util.Date representing the - last time a JobExecution was - persisted. + last time a JobExecution was persisted. @@ -389,17 +389,17 @@ will be kicked off again for 01-01, starting where it left off and completing successfully at 9:30. Because it's now the next day, the 01-02 job must be run as well, which is kicked off just afterwards at - 9:31, and completes in its normal one hour time at 10:30. There is no + 9:31, and completes in it's normal one hour time at 10:30. There is no requirement that one JobInstance be kicked off after another, unless there is potential for the two jobs to attempt to access the same data, causing issues with locking at the database level. It is entirely up to the scheduler to determine when a Job should be run. Since they're separate - JobInstances, Spring Batch will make no attempt - to stop them from being run concurrently. (Attempting to run the same + JobInstances, Spring Batch will make no attempt to stop them from being + run concurrently. (Attempting to run the same JobInstance while another is already running will result in a JobExecutionAlreadyRunningException - being thrown). There should now be an extra entry in both the + being thrown) There should now be an extra entry in both the JobInstance and JobParameters tables, and two extra entries in the JobExecution table: @@ -554,12 +554,12 @@ + scale="90" /> - + @@ -568,12 +568,12 @@ A StepExecution represents a single attempt to execute a Step. A new - StepExecution will be created each time a - Step is run, similar to - JobExecution. However, if a step fails to execute - because the step before it fails, there will be no execution persisted - for it. A StepExecution will only be created when - its Step is actually started. + StepExecution will be created each time a Step is + run, similar to JobExecution. However, if a step + fails to execute because the step before it fails, there will be no + execution persisted for it. A StepExecution will + only be created when it's Step is actually + started. Step executions are represented by objects of the StepExecution class. Each execution contains a @@ -596,8 +596,8 @@ A BatchStatus object that indicates the status of the execution. While it's running, the - status is BatchStatus.STARTED, if it fails, the status is - BatchStatus.FAILED, and if it finishes successfully, the status + status is BatchStatus.STARTED, if it fails the status is + BatchStatus.FAILED, and if it finishes successfully the status is BatchStatus.COMPLETED @@ -670,15 +670,15 @@ processSkipCount - The number of times process has - failed, resulting in a skipped item. + The number of times process has failed, resulting in a + skipped item. filterCount The number of items that have been 'filtered' by the - ItemProcessor. + ItemProcessor @@ -848,14 +848,14 @@ this scenario. This value will be updated just before each commit by the framework, and can contain multiple rows corresponding to entries within the ExecutionContext. Being notified before a - commit requires one of the various StepListeners, - or an ItemStream, which are discussed in more - detail later in this guide. As with the previous example, it is assumed - that the Job is restarted the next day. When it is - restarted, the values from the ExecutionContext of - the last run are reconstituted from the database, and when the - ItemReader is opened, it can check to see if it has - any stored state in the context, and initialize itself from there: + commit requires one of the various StepListeners, or an + ItemStream, which are discussed in more detail + later in this guide. As with the previous example, it is assumed that the + Job is restarted the next day. When it is restarted, the values from the + ExecutionContext of the last run are reconstituted + from the database, and when the ItemReader is + opened, it can check to see if it has any stored state in the context, and + initialize itself from there: if (executionContext.containsKey(getKey(LINES_READ_COUNT))) { log.debug("Initializing for restart. Restart data is: " + executionContext); @@ -898,7 +898,7 @@ exists per StepExecution at any given time. Clients of the ExecutionContext should be careful because this creates a shared keyspace, so care should be taken when putting - values in to ensure no data is overwritten. However, the + values in to ensure no data is overwritten, however, the Step stores absolutely no data in the context, so there is no way to adversely affect the framework. @@ -915,7 +915,7 @@ - As noted in the comment, ecStep will not equal ecJob; they are two + As noted in the comment, ecStep will not equal ecJob, they are two different ExecutionContexts. The one scoped to the Step will be saved at every commit point in the Step, whereas the one scoped to the @@ -981,7 +981,7 @@ the output of a Step, one item at a time. Generally, an item writer has no knowledge of the input it will receive next, only the item that was passed in its current invocation. More - details about the ItemWriter interface and its + details about the ItemWriter interface and it's various implementations can be found in
@@ -994,9 +994,9 @@ ItemReader reads one item, and the ItemWriter writes them, the ItemProcessor provides access to transform or apply - other business processing. If, while processing the item, it is determined - that the item is not valid, returning null indicates that the item should - not be written out. More details about the ItemProcessor interface can be - found in . + other business processing. If while processing the item it's determined + that it's not valid, returning null indicates that it should not be + written out. More details about the ItemProcessor interface can be found + in . diff --git a/docs/src/site/docbook/reference/job.xml b/docs/src/site/docbook/reference/job.xml index 10ff2cf5b..00e41a540 100644 --- a/docs/src/site/docbook/reference/job.xml +++ b/docs/src/site/docbook/reference/job.xml @@ -10,31 +10,30 @@ + fileref="images/spring-batch-reference-model.png" scale="90" /> - While the Job object may seem like a simple - container for steps, there are many configuration options of which a - developers must be aware . Furthermore, there are many considerations for - how a Job will be run and how its meta-data will be - stored during that run. This chapter will explain the various configuration - options and runtime concerns of a Job. + While the Job object may seem like a simple container for steps, there + are many configuration options that developers should be aware of. + Furthermore, there are many considerations for how a + Job will be run and how its meta data will be stored + during that run. This chapter will explain the various configuration options + and runtime concerns of a Job.
Configuring a Job There are multiple implementations of the Job interface, however, the - namespace abstracts away the differences in configuration. It has only - three required dependencies: a name, JobRepository, - and a list of Steps. + linkend="job">Job interface, however, the namespace abstracts away + the differences in configuration. It has only three required dependencies: + a name, JobRepository, and a list of Steps. <job id="footballJob"> @@ -58,26 +57,20 @@ - Note that if the job repository's default id is overridden, it must - be explicitly specified on every - Step as well as on - the Job. -
Restartability - One key issue when execution a batch job concerns the behavior of - a Job when it is restarted? The launching of a - Job is considered to be a 'restart' if a - JobExecution already exists for the particular - JobInstance. Ideally, all jobs should be able to - start up where they left off, but there are scenarios where this is not - possible. It is entirely up to the developer to - ensure that a new JobInstance is created in this scenario. - However, Spring Batch does provide some help. If a - Job should never be restarted, but should always - be run as part of a new JobInstance, then the - restartable property may be set to 'false': + One key concern when execution a batch job, is what happens when a + failed job is restarted? A Job is considered to have been 'restarted' if + the same JobInstance has more than one + JobExecution. Ideally, all jobs should be able to start up where they + left off, but there are scenarios where this is not possible. It is entirely up to the developer to ensure that a new + instance is always created in this scenario. However, Spring + Batch does provide some help. If a Job should never be restarted, but + should always be run as part of a new + JobInstance, then the restartable property may be + set to 'false': <job id="footballJob" restartable="false"> @@ -138,9 +131,8 @@ - JobListeners can be added to a - SimpleJob via the listeners element on the - job: + Listeners can be added to a SimpleJob via + the setJobListeners property: <job id="footballJob"> @@ -154,17 +146,17 @@ - It should be noted that afterJob will be - called regardless of the success or failure of the - Job. If success or failure needs to be determined - it can be obtained from the JobExecution: + It should be noted that afterJob will be called regardless of the + success or failure of the Job. If success or + failure needs to be determined it can be obtained from the + JobExecution: void afterJob(JobExecution jobExecution){ - if( jobExecution.getStatus() == BatchStatus.COMPLETED ){ + if( jobExecution.getStatus = BatchStatus.COMPLETED ){ //job success } - else if(jobExecution.getStatus() == BatchStatus.FAILED){ + else if(jobExecution.getStatus = BatchStatus.FAILED){ //job failure } } @@ -176,14 +168,14 @@ JobFactory and Stateful Components in Steps Unlike many traditional Spring applications, many of the - components of a batch application are stateful; the file readers and + components of a batch application are stateful, the file readers and writers are obvious examples. The recommended way to deal with this is to create a fresh ApplicationContext for each job execution. If the Job is launched from the - command line with CommandLineJobRunner, this is - trivial. For more complex launching scenarios where jobs are executed in - parallel or serially from the same process, some extra steps have to be - taken to ensure that the ApplicationContext is + command line with CommandLineJobRunner this is + trivial. For more complex launching scenarios, where jobs are executed + in parallel or serially from the same process, some extra steps have to + be taken to ensure that the ApplicationContext is refreshed. This is preferable to using prototype scope for the stateful beans because then they would not receive lifecycle callbacks from the container at the end of use. (e.g. through destroy-method in XML) @@ -204,13 +196,12 @@ Configuring a JobRepository As described in earlier, the JobRepository is - used for basic CRUD operations of the various persisted domain objects - within Spring Batch, such as JobExecution and - StepExecution. It is required by many of the major - framework features, such as the JobLauncher, + linkend="jobRepository">JobRepository is used for basic CRUD + operations of the various persisted domain objects within Spring Batch, + such as JobExecution and StepExecution. It is required by many of the + major framework features, such as the JobLauncher, Job, and Step. The batch - namespace abstracts away many of the implementation details of the + namespace abstract much of the implementation details of the JobRepository implementations and their collaborators. However, there are still a few configuration options available: @@ -235,7 +226,7 @@ If the namespace is used, transactional advice will be automatically created around the repository. This is to ensure that the batch meta data, including state that is necessary for restarts after a - failure, is persisted correctly. The behavior of the framework is not + failure, is persisted correctly. The behaviour of the framework is not well defined if the repository methods are not transactional. The isolation level in the create* method attributes is specified separately to ensure that when jobs are launched, if two @@ -255,7 +246,7 @@ If the namespace or factory beans aren't used then it is also - essential to configure the transactional behavior of the repository + essential to configure the transactional behaviour of the repository using AOP: @@ -275,7 +266,7 @@ This fragment can be used as is, with almost no changes. Remember also to include the appropriate namespace declarations and to make sure - spring-tx and spring-aop (or the whole of spring) are on the + spring-tx and spring-aop (or the whole of spring) is on the classpath.
@@ -303,7 +294,7 @@ as SYSTEM.TEST_JOB_EXECUTION. - Only the table prefix is configurable. The table and column + Only the table prefix is configurable, the table and column names are not.
@@ -312,11 +303,10 @@ In-Memory Repository There are scenarios in which you may not want to persist your - domain objects to the database. One reason may be speed; storing domain + domain objects to the database. One reason may be speed, storing domain objects at each commit point takes extra time. Another reason may be - that you just don't need to persist status for a particular job. For - this reason, Spring batch provides an in-memory Map version of the job - respository: + that you just don't need to persist status for a particular job. Spring + batch provides a solution: <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" /> @@ -328,7 +318,7 @@ The most basic implementation of the JobLauncher interface is the - SimpleJobLauncher. Its only required dependency is + SimpleJobLauncher. It's only required dependency is a JobRepository, in order to obtain an execution: @@ -337,45 +327,44 @@ <property name="jobRepository" ref="jobRepository" /> </bean> - Once a JobExecution is - obtained, it is passed to the execute method of - Job, ultimately returning the - JobExecution to the caller: + Once a JobExecution is obtained, + it is passed to the execute method of Job, + ultimately returning the JobExecution to the + caller: + fileref="images/job-launcher-sequence-sync.png" scale="70" + width="66%" /> - The sequence is straightforward and works well when launched from a - scheduler. However, issues arise when trying to launch from an HTTP - request. In this scenario, the launching needs to be done asynchronously - so that the SimpleJobLauncher returns immediately - to its caller. This is because it is not good practice to keep an HTTP - request open for the amount of time needed by long running processes such - as batch. An example sequence is below: + The sequence is straightforward, and works well when launched from a + scheduler, but causes issues when trying to launch from an HTTP request. + In this scenario, the launching needs to be done asynchronously, so that + the SimpleJobLauncher returns immediately to it's + caller. This is because it is not good practice to keep an HTTP request + open for the amount of time needed by long running processes such as + batch. An example sequence is below: + fileref="images/job-launcher-sequence-async.png" scale="70" + width="66%" /> @@ -399,16 +388,16 @@
Running a Job - At a minimum, launching a batch job requires two things: the - Job to be launched and a - JobLauncher. Both can be contained within the same - context or different contexts. For example, if launching a job from the - command line, a new JVM will be instantiated for each Job, and thus every - job will have its own JobLauncher. However, if - running from within a web container within the scope of an - HttpRequest, there will usually be one - JobLauncher, configured for asynchronous job - launching, that multiple requests will invoke to launch their jobs. + At a minimum, launching a batch job requires two things: the Job to + be launched and a JobLauncher. Both can be + contained within the same context or different contexts. For example, if + launching a job from the command line, a new JVM will be instantiated for + each Job, and thus every job will have it's own + JobLauncher. However, if running from within a web + container within the scope of an HttpRequest, there + will usually be one JobLauncher, configured for + asynchronous job launching, that multiple requests will invoke to launch + their jobs.
Running Jobs from the Command Line @@ -432,32 +421,29 @@ CommandLineJobRunner. It's important to note that this is just one way to bootstrap your application, but there are many ways to launch a Java process, and this class should in no way be - viewed as definitive. The CommandLineJobRunner - performs four tasks: + viewed as definitive. It performs four tasks: - Load the appropriate - ApplicationContext + Loads the appropriate Application Context - Parse command line arguments into - JobParameters + Parses command line arguments into JobParameters - Locate the appropriate job based on arguments + Locates the appropriate job based on arguments - Use the JobLauncher provided in the - application context to launch the job. + Uses the JobLauncher provided in the application context to + launch the job. - All of these tasks are accomplished using only the arguments - passed in. The following are required arguments: + All of these tasks are accomplished based completely upon the + arguments passed in. The following are required arguments: CommandLineJobRunner arguments @@ -493,21 +479,22 @@ This example is using the same 'EndOfDay' example from . The first argument is 'endOfDayJob.xml', which is the Spring ApplicationContext containing the - Job. The second argument, 'endOfDay' represents - the job name. The final argument, 'schedule.date(date)=2008/01/01' - will be converted into JobParameters. An - example of the XML configuration is below: + Job. The second argument, 'endOfDay' represents the job name. The + final argument, 'schedule.date=01-01-2008' will be converted into + JobParameters. An example of the XML + configuration is below: - <job id="endOfDay"> - <steps> - <step name="step1" parent="simpleStep" /> + <bean id="endOfDay" + class="org.springframework.batch.core.job.SimpleJob"> + <property name="steps"> + <bean id="step1" parent="simpleStep" /> <!-- Step details removed for clarity --> - </steps> - </job> + </property> + </bean> <!-- Launcher details removed for clarity --> - <beans: bean id="jobLauncher" - class="org.springframework.batch.core.launch.support.SimpleJobLauncher" /> + <bean id="jobLauncher" + class="org.springframework.batch.core.launch.support.SimpleJobLauncher" /> This example is overly simplistic, since there are many more requirements to a run a batch job in Spring Batch in general, but it @@ -520,23 +507,22 @@
ExitCodes - When launching a batch job from the command-line, an enterprise - scheduler is often used. Most schedulers are fairly dumb and work only - at the process level. This means that they only know about some + When launching a batch job from the command-line, it is often + from an enterprise scheduler. Most schedulers are fairly dumb, and + work only at the process level. Meaning, they only know about some operating system process such as a shell script that they're invoking. In this scenario, the only way to communicate back to the scheduler about the success or failure of a job is through return codes. A - return code is a number that is returned to a scheduler by the process - that indicates the result of the run. In the simplest case: 0 is - success and 1 is failure. However, there may be more complex - scenarios: If job A returns 4 kick off job B, and if it returns 5 kick - off job C. This type of behavior is configured at the scheduler level, - but it is important that a processing framework such as Spring Batch - provide a way to return a numeric representation of the 'Exit Code' - for a particular batch job. In Spring Batch this is encapsulated - within an ExitStatus, which is covered in more - detail in Chapter 5. For the purposes of discussing exit codes, the - only important thing to know is that an + number is returned to a scheduler that is told how to interpret the + result. In the simple case: 0 is success and 1 is failure. However, + there may be scenarios such as: If job A returns 4 kick off job B, if + it returns 5 kick off job C. This type of behavior is configured at + the scheduler level, but it is important that a processing framework + such as Spring Batch provide a way to return a numeric representation + of of the 'Exit Code' for a particular batch job. In Spring Batch this + is encapsulated within an ExitStatus, which is + covered in more detail in Chapter 5. For the purposes of discussing + exit codes, the only important thing to know is that an ExitStatus has an exit code property that is set by the framework (or the developer) and is returned as part of the JobExecution returned from the @@ -568,7 +554,7 @@ it will be injected into the runner after the context is created. All that needs to be done to provide your own ExitCodeMapper is to declare the implementation - as a root level bean and ensure that it is part of the + as a root level bean, and ensure it's part of the ApplicationContext that is loaded by the runner.
@@ -595,20 +581,19 @@ - A JobLauncher uses the JobRepository to create new - JobExecution objects and run them. + JobExecution objects, and run them. Job and Step implementations later use the same JobRepository for basic updates of the same executions during the running of a Job. - The basic operations suffice for simple scenarios, but in a large batch - environment with hundreds of batch jobs and complex scheduling + The basic operations suffice for simple scenarios. However, in a large + batch environment with hundreds of batch jobs and complex scheduling requirements, more advanced access of the meta data is required: @@ -618,9 +603,8 @@ - + @@ -668,10 +652,9 @@ Earlier in this chapter, it was mentioned that the table prefix of the - JobRepository can be modified to allow for - different versions or schemas. Because the - JobExplorer is working with the same tables, it - too needs the ability to set a prefix: + JobRepository can be modified to allow for different versions or + schemas. Because the JobExplorer is working with the same tables, it too + needs the ability to set a prefix: <bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean" @@ -756,43 +739,35 @@ self-explanatory, and more detailed explanations can be found on the javadoc - of the interface. However, the - startNextInstance method is worth noting. This - method will always start a new instance of a Job. + of the interface. However, the 'startNextInstance' method is + worth noting. This method will always start a new instance of a Job. This can be extremely useful if there are serious issues in a - JobExecution and the Job + JobExecution, and the Job needs to be started over again from the beginning. Unlike JobLauncher though, which requires a new - JobParameters object that will trigger a new - JobInstance if the parameters are different from - any previous set of parameters, the - startNextInstance method will use the - JobParametersIncrementer tied to the - Job to force the Job to a - new instance: + JobParameters that will trigger a new JobInstance + if they are different than any previous one, the startNextInstance + method will use the JobParametersIncrementer tied to the Job to force + the Job to a new instance: public interface JobParametersIncrementer { JobParameters getNext(JobParameters parameters); - } The contract of JobParametersIncrementer is - that, given a JobParameters - object, it will return the 'next' JobParameters - object by incrementing any necessary values it may contain. This - strategy is useful because the framework has no way of knowing what - changes to the JobParameters make it the 'next' - instance. For example, if the only value in - JobParameters is a date, and the next instance - should be created, should that value be incremented by one day? Or one - week (if the job is weekly for instance)? The same can be said for any - numerical values that help to identify the Job, - as shown below: + that, given a JobParameters, it + will return the 'next' parameter by incrementing any values it may + contain. This strategy is useful because the framework has no way of + knowing what changes to the JobParameters make it the 'next' instance. + For example, if the only value in JobParameters is a date, and the next + instance should be created, should that value be incremented by one day? + Or one week? (if the job is weekly for instance) The same can be said + for any numerical values that help to identify the Job, as shown + below: public class SampleIncrementer implements JobParametersIncrementer { @@ -809,13 +784,12 @@ In this example, the value with a key of 'run.id' is used to - discriminate between JobInstances. If the - JobParameters passed in is null, it can be - assumed that the Job has never been run before - and thus its initial state can be returned. However, if not, the old - value is obtained, incremented by one, and returned. An incrementer can - be associated with Job via the 'incrementer' - attribute in the namespace: + discriminate between JobInstances. If the JobParameters passed in is + null, it can be assumed that the Job has never been run before and thus + it's initial state can be returned. However, if not, the old value is + obtained, incremented by one, and returned. An incrementer can be + associated with Job via the 'incrementer' attribute in the + namespace: <job id="footballJob" incrementer="sampleIncrementer"> @@ -843,8 +817,8 @@ The shutdown is not immediate, since there is no way to force immediate shutdown, especially if the execution is currently in developer code that the framework has no control over, such as a - business service. However, as soon as control is returned back to the - framework, it will set the status of the current + business service. What it does mean, is that as soon as control is + returned back to the framework, it will set the status of the current StepExecution to BatchStatus.STOPPED, save it, then do the same for the JobExecution before finishing. diff --git a/docs/src/site/docbook/reference/readersAndWriters.xml b/docs/src/site/docbook/reference/readersAndWriters.xml index 1f8f87aa6..b268f11b5 100644 --- a/docs/src/site/docbook/reference/readersAndWriters.xml +++ b/docs/src/site/docbook/reference/readersAndWriters.xml @@ -27,17 +27,17 @@ XML - XML ItemReaders process XML independently of technologies used for parsing, mapping and validating objects. Input - data allows for the validation of an XML file against an XSD + data allows for the validation of and XML file against an XSD schema. - Database - A database resource is accessed to return + Database - A database resource is accessed that returns resultsets which can be mapped to objects for processing. The default SQL ItemReaders invoke a RowMapper to return objects, keep track of the current row if restart is - required, store basic statistics, and provide some transaction - enhancements that will be explained later. + required, basic statistics, and some transaction enhancements that + will be explained later. There are many more possibilities, but we'll focus on the basic ones for this chapter. A complete list of all available ItemReaders @@ -54,11 +54,12 @@ The read method defines the most essential - contract of the ItemReader; calling it returns one - Item or null if no more items are left. An item might represent a line in - a file, a row in a database, or an element in an XML file. It is generally - expected that these will be mapped to a usable domain object (i.e. Trade, - Foo, etc) but there is no requirement in the contract to do so. + contract of the ItemReader, calling it returns one + Item, returning null if no more items are left. An item might represent a + line in a file, a row in a database, or an element in an XML file. It is + generally expected that these will be mapped to a usable domain object + (i.e. Trade, Foo, etc) but there is no requirement in the contract to do + so. It is expected that implementations of the ItemReader interface will be forward only. However, @@ -78,8 +79,8 @@ ItemReader, but with inverse operations. Resources still need to be located, opened and closed but they differ in that an ItemWriter writes out, rather than reading in. In - the case of databases or queues these may be inserts, updates, or sends. - The format of the serialization of the output is specific to each batch + the case of databases or queues these may be inserts, updates or sends. + The format of the serialization of the output is specific for every batch job. As with ItemReader, @@ -94,15 +95,15 @@ As with read on ItemReader, write provides - the basic contract of ItemWriter; it will attempt + the basic contract of ItemWriter, it will attempt to write out the list of items passed in as long as it is open. Because it is generally expected that items will be 'batched' together into a chunk - and then output, the interface accepts a list of items, rather than an - item by itself. After writing out the list, any flushing that may be - necessary can be performed before returning from the write method. For - example, if writing to a Hibernate DAO, multiple calls to write can be - made, one for each item. The writer can then call close on the hibernate - Session before returning. + and then output, the interface accepts a list, rather than an item by + itself. After writing out the list, any flushing that may be necessary can + be performed before returning from the write method. For example, if + writing to a Hibernate DAO, multiple calls to write can be made, one for + each item. The writer can then call close on the hibernate Session before + returning.
@@ -138,14 +139,14 @@ } The class above contains another ItemWriter - to which it delgates after having provided some business logic. This + that it delgates to after having provided some business logic. This pattern could easily be used for an ItemReader as well, perhaps to obtain more reference data based upon the input that was provided by the main ItemReader. It is also useful if you need to control the call to write yourself. However, if you only want to 'transform' the item passed in for writing - before it is actually written, there isn't much need to call - write yourself: you just want to modify the item. + before it is actual written, there isn't much need to call + write yourself, you just want to modify the item. For this scenario, Spring Batch provides the ItemProcessor interface: @@ -154,14 +155,14 @@ O process(I item) throws Exception; } - An ItemProcessor is very simple; given one - object, transform it and return another. The provided object may or may + An ItemProcessor is very simple, given one + object, transform it and return another. The object provided may or may not be of the same type. The point is that business logic may be applied within process, and is completely up to the developer to create. An ItemProcessor can be wired directly into a step, For example, assuming an ItemReader provides a class of type Foo, and it needs to be converted to type Bar before being - written out. An ItemProcessor can be written that + written out. An ItemTransformer can be written that performs the conversion: public class Foo {} @@ -173,15 +174,15 @@ public class FooProcessor implements ItemProcessor<Foo,Bar>{ //Perform simple transformation, convert a Foo to a Bar - public Bar process(Foo foo) throws Exception { + public Bar transform(Foo foo) throws Exception { return new Bar(foo); } } public class BarWriter implements ItemWriter<Bar>{ - public void write(List<? extends Bar> bars) throws Exception { - //write bars + public void write(Bar bar) throws Exception { + //write bar } //rest of class ommitted for clarity @@ -192,12 +193,12 @@ class FooProcessor that adheres to the ItemProcessor interface. The transformation is simple, but any type of transformation could be done here. The - BarWriter will be used to write out - Bar objects, throwing an exception if any other - type is provided. Similarly, the FooProcessor will - throw an exception if anything but a Foo is - provided. The FooProcessor can then be injected - into a Step: + BarWriter will be used to write out 'Bars', + throwing an exception if any other type is provided. Similarly, the + FooProcessor will throw an exception if anything but a + Foo is provided. The + FooProcessor can then be injected into a + Step: <job id="ioSampleJob"> @@ -212,11 +213,11 @@ Chaining ItemProcessors Performing a single transformation is useful in many scenarios, - but what if you want to 'chain' together multiple - ItemProcessors? This can be accomplished using - the composite pattern mentioned previously. To update the previous, - single transformation, example, Foo will be - transformed to Bar, which will be transformed to + but what if you want to 'chain' together multiple ItemProcessors? This + can be accomplished using the composite pattern mentioned previously. To + update the previous, single transformation, example, + Foo will be Transformed to + Bar, which will be transformed to Foobar and written out: public class Foo {} @@ -232,22 +233,22 @@ public class FooProcessor implements ItemProcessor<Foo,Bar>{ //Perform simple transformation, convert a Foo to a Bar - public Bar process(Foo foo) throws Exception { + public Bar transform(Foo foo) throws Exception { return new Bar(foo); } } public class BarProcessor implements ItemProcessor<Bar,FooBar>{ - public FooBar process(Bar bar) throws Exception { + public FooBar transform(Bar bar) throws Exception { return new Foobar(bar); } } public class FoobarWriter implements ItemWriter<FooBar>{ - public void write(List<? extends FooBar> items) throws Exception { - //write items + public void write(Object item) throws Exception { + //write Foobar } //rest of class ommitted for clarity @@ -285,68 +286,41 @@
- -
- Filtering Records - - One typical use for an item processor is to filter out records - before they are passed to the ItemWriter. Filtering is an action - distinct from skpping; skipping indicates that a record is invalid - whereas filtering simply indicates that a record should not be - written. - - For example, consider a batch job that reads a file containing - three different types of records: records to insert, records to update, - and records to delete. If record deletion is not supported by the - system, then we would not want to send any "delete" records to the - ItemWriter. But, since these records are not - actually bad records, we would want to filter them out, rather than - skip. As a result, the ItemWriter would receive only "insert" and - "update" records. - - To filter a record, one simply returns "null" from the - ItemProcessor. The framework will detect that the - result is "null" and avoid adding that item to the list of records - delivered to the ItemWriter. As usual, an - exception thrown from the ItemProcessor will - result in a skip. -
ItemStream - Both ItemReaders and - ItemWriters serve their individual purposes well, - but there is a common concern among both of them that necessitates another - interface. In general, as part of the scope of a batch job, readers and - writers need to be opened, closed, and require a mechanism for persisting - state: + Both ItemReaders and ItemWriters serve their individual purposes + well, but there is a common concern among both of them that necessitates + another interface. In general, as part of the scope of a batch job, + readers and writers need to be opened, closed, and require a mechanism for + persisting state: public interface ItemStream { - void open(ExecutionContext executionContext) throws ItemStreamException; + void open(ExecutionContext executionContext) throws StreamException; void update(ExecutionContext executionContext) throws ItemStreamException; - void close() throws ItemStreamException; + void close(ExecutionContext executionContext) throws StreamException; } - Before describing each method, we should mention the + Before describing each method, its worth briefly mentioning the ExecutionContext. Clients of an - ItemReader that also implement + ItemReader that also implements ItemStream should call open before any calls to - read in order to open any resources such as files - or to obtain connections. A similar restriction applies to an - ItemWriter that implements + read, to open any resources such as files or + obtain connections. A similar restriction applies to an + ItemWriter that also implements ItemStream. As mentioned in Chapter 2, if expected data is found in the ExecutionContext, it may be used to start the ItemReader or ItemWriter at a location other than its initial state. Conversely, close will be called to ensure - that any resources allocated during open will be + any resources allocated during open will be released safely. update is called primarily to ensure that any state currently being held is loaded into the provided ExecutionContext. This method will be called before @@ -373,11 +347,9 @@ If they do, and they are being used in conjunction with Spring Batch Core as part of a Step in a Job, then they almost certainly need to be registered manually with the - Step. A reader, writer, or processor that is - directly wired into the Step will be registered automatically if it - implements ItemStream or a - StepListener interface. But because the delegates - are not known to the Step, they need to be injected + Step. Registration is automatic when the reader, + writer, or processor is directly wired into the Step. The delegates are + not known to the Step, so they need to be injected as listeners or streams (or both if appropriate): @@ -408,9 +380,7 @@ always been the flat file. Unlike XML, which has an agreed upon standard for defining how it is structured (XSD), anyone reading a flat file must understand ahead of time exactly how the file is structured. In general, - all flat files fall into two types: Delimited and Fixed Length. Delimited - files are those in which fields are separated by a delimiter, such as a - comma. Fixed Length files have fields that are a set length. + all flat files fall into two types: Delimited and Fixed Length.
The FieldSet @@ -477,7 +447,7 @@ processing locations and vice versa. File moving utilities are beyond the scope of the spring batch architecture but it is not unusual for batch job streams to include file moving utilities as steps in the job - stream. It is sufficient that the batch architecture only needs to know + stream. Its sufficient that the batch architecture only needs to know how to locate the files to be processed. Spring Batch begins the process of feeding the data into the pipe from this starting point. However, LineMapper As with RowMapper, which takes a low - level construct such as ResultSet and returns - an Object, flat file processing requires the - same construct to convert a String line into an - Object: + level construct such as ResultSet and returns an Object, + flat file procesing requires the same construct to convert a String + line into an Object: public interface LineMapper<T> { T mapLine(String line, int lineNumber) throws Exception; @@ -580,29 +549,29 @@ - The basic contract is that, given the current line and the line - number with which it is associated, the mapper should return a - resulting domain object. This is similar to - RowMapper in that each line is associated with - its line number, just as each row in a - ResultSet is tied to its row number. This - allows the line number to be tied to the resulting domain object for - identity comparison or for more informative logging. However, unlike - RowMapper, the + The basic contract is that, given the current line, and the line + number its associated with, return a resulting domain object. This is + similar to RowMapper in that each line is + associated with it's line number, just as each row in a + ResultSet is tied to the row number it belongs + to. This allows for tying the line number to the resulting domain + object for identity comparison, or for more informative logging. + However, unlike RowMapper, the LineMapper is given a raw line which, as - discussed above, only gets you halfway there. The line must be - tokenized into a FieldSet, which can then be - mapped to an object, as described below. + discussed above, only gets you halfway there. What is needed is + tokenization of the line into a FieldSet, which + can then be mapped to an object, as described below.
LineTokenizer - An abstraction for turning a line of input into a line into a - FieldSet is necessary because there can be many - formats of flat file data that need to be converted to a - FieldSet. In Spring Batch, this interface is - the LineTokenizer: + Because there can be many formats of flat file data, which all + need to be converted to a FieldSet so that a + useful domain object can be created from them, an abstraction for + turning a line of input into a FieldSet is + necessary. In Spring Batch, this is called a + LineTokenizer: public interface LineTokenizer { @@ -615,31 +584,30 @@ The contract of a LineTokenizer is such that, given a line of input (in theory the - String could encompass more than one line), a + String could encompass more than one line) a FieldSet representing the line will be - returned. This FieldSet can then be passed to a + returned. This can then be passed to a FieldSetMapper. Spring Batch contains the - following LineTokenizer implementations: + following LineTokenizer implementations: DelmitedLineTokenizer - Used for - files where fields in a record are separated by a delimiter. The - most common delimiter is a comma, but pipes or semicolons are - often used as well. + files that separate records by a delimiter. The most common is a + comma, but pipes or semicolons are often used as well - FixedLengthTokenizer - Used for files - where fields in a record are each a 'fixed width'. The width of - each field must be defined for each record type. + FixedLengthTokenizer - Used for + tokenizing files where each record is separated by a 'fixed width' + that must be defined per record. PrefixMatchingCompositeLineTokenizer - - Determines which among a list of - LineTokenizers should be used on a - particular line by checking against a prefix. + - Tokenizer that determines which among a list of Tokenizers + should be used on a particular line by checking against a + prefix.
@@ -650,8 +618,8 @@ The FieldSetMapper interface defines a single method, mapLine, which takes a FieldSet object and maps its contents to an - object. This object may be a custom DTO, a domain object, or a simple - array, depending on the needs of the job. The + object. This object may be a custom DTO or domain object, or it could + be as simple as an array, depending on your needs. The FieldSetMapper is used in conjunction with the LineTokenizer to translate a line of data from a resource into an object of the desired type: @@ -665,9 +633,8 @@ - The pattern used is the same as the - RowMapper used by - JdbcTemplate. + The pattern used is the same as RowMapper + used by JdbcTemplate.
@@ -681,30 +648,28 @@ - Pass the string line into the - LineTokenizer#tokenize() method, in - order to retrieve a FieldSet. + Pass the string line into the LineTokenizer#tokenize() + method, in order to retrieve a + FieldSet - Pass the FieldSet returned from - tokenizing to a FieldSetMapper, returning - the result from the ItemReader#read() - method. + Pass the FieldSet returned from tokenizing to a + FieldSetMapper, returning the result from the ItemReader#read() + method The two interfaces described above represent two separate tasks: converting a line into a FieldSet, and mapping - a FieldSet to a domain object. Because the - input of a LineTokenizer matches the input of - the LineMapper (a line), and the output of a - FieldSetMapper matches the output of the - LineMapper, a default implementation that uses - both a LineTokenizer and - FieldSetMapper is provided. The - DefaultLineMapper represents the behavior most - users will need: + a FieldSet to a domain object. Becaue the input + of a LineTokenizer matches the input of the + LineMapper (a line), and the output of a + FieldSetMapper matches the output of the + LineMapper, and this is the deafult behavior + most users will need, a default implementation that uses both a + LineTokenizer and + FieldSetMapper is provided: public class DefaultLineMapper<T> implements LineMapper<T>, InitializingBean { @@ -748,8 +713,8 @@ "AdamBo00,Adams,Bob,te,1946,1969", "AdamCh00,Adams,Charlie,wr,1979,2003" - The contents of this file will be mapped to the following - Player domain object: + The contents of this file will be mapped to the following Player + domain object: public class Player implements Serializable { private String ID; @@ -771,14 +736,13 @@ } - In order to map a FieldSet into a - Player object, a - FieldSetMapper that returns players needs to be - defined: + In order to map a FieldSet into a Player + object, a FieldSetMapper that returns players + needs to be defined: protected static class PlayerFieldSetMapper implements FieldSetMapper<Player> { - public Player mapFieldSet(FieldSet fieldSet) { + public Object mapLine(FieldSet fieldSet) { Player player = new Player(); player.setID(fieldSet.readString(0)); @@ -800,7 +764,7 @@ FlatFileItemReader<Player> itemReader = new FlatFileItemReader<Player>(); itemReader.setResource(new FileSystemResource("resources/players.csv")); - //DelimitedLineTokenizer defaults to comma as its delimiter + //DelimitedLineTokenizer defaults to comma as it's delimiter LineMapper<Player> lineMapper = new DefaultLineMapper<Player>(); lineMapper.setLineTokenizer(new DelimitedLineTokenizer()); lineMapper.setFieldSetMapper(new PlayerFieldSetMapper()); @@ -818,14 +782,13 @@
Mapping fields by name - There is one additional piece of functionality that is allowed - by both DelimitedLineTokenizer and - FixedLengthTokenizer that is similar in - function to a Jdbc ResultSet. The names of the - fields can be injected into either of these - LineTokenizer implementations to increase the + There is one additional functionality a + LineTokenizer that is similar in function to a + Jdbc ResultSet. The names of the fields can be + injected into the LineTokenizer to increase the readability of the mapping function. First, the column names of all - fields in the flat file are injected into the tokenizer: + fields in the flat file are injected into the + LineTokenizer: tokenizer.setNames(new String[] {"ID", "lastName","firstName","position","birthYear","debutYear"}); @@ -836,7 +799,7 @@ public class PlayerMapper implements FieldSetMapper<Player> { - public Player mapFieldSet(FieldSet fs) { + public Object mapLine(FieldSet fs) { if(fs == null){ return null; @@ -852,6 +815,7 @@ return player; } + }
@@ -866,8 +830,8 @@ providing a FieldSetMapper that automatically maps fields by matching a field name with a setter on the object using the JavaBean specification. Again using the football example, the - BeanWrapperFieldSetMapper configuration looks - like the following: + FieldSetMapper configuration looks like the + following: <bean id="fieldSetMapper" @@ -942,12 +906,11 @@ - Because the FixedLengthLineTokenizer uses - the same LineTokenizer interface as discussed - above, it will return the same FieldSet as if a - delimiter had been used. This allows the same approaches to be used in - handling its output, such as using the - BeanWrapperFieldSetMapper. + This LineTokenizer will return the same + FieldSet as if a delimiter had been used, + allowing the same approach above to be used such as the + BeanWrapperFieldSetMapper, in a way that is + ignorant of how the actual line was parsed. It should be noted that supporting the above ranges requires a specialized property editor be configured anywhere in the @@ -968,97 +931,69 @@
-
+
Multiple record types within a single file All of the file reading examples up to this point have all made - a key assumption for simplicity's sake: all of the records in a file - have the same format. However, this may not always be the case. It is - very common that a file might have records with different formats that - need to be tokenized differently and mapped to different objects. The + a key assumption for simplicity's sake: one record equals one line. + However, this may not always be the case. Its very common that a file + might have records spanning multiple lines with multiple formats. The following excerpt from a file illustrates this: - - USER;Smith;Peter;;T;20014539;F - LINEA;1044391041ABC037.49G201XX1383.12H - LINEB;2134776319DEF422.99M005LI + HEA;0013100345;2007-02-15 + NCU;Smith;Peter;;T;20014539;F + BAD;;Oak Street 31/A;;Small Town;00235;IL;US + SAD;Smith, Elizabeth;Elm Street 17;;Some City;30011;FL;United States + BIN;VISA;VISA-12345678903 + LIT;1044391041;37.49;0;0;4.99;2.99;1;45.47 + LIT;2134776319;221.99;5;0;7.99;2.99;1;221.87 + SIN;UPS;EXP;DELIVER ONLY ON WEEKDAYS + FOT;2;2;267.34 - - - In this file we have three types of records, "USER", "LINEA", - and "LINEB". A "USER" line corresponds to a User object. "LINEA" and - "LINEB" both correspond to Line objects, though a "LINEA" has more - information than a "LINEB". - - The ItemReader will read each line - individually, but we must specify different - LineTokenizer and - FieldSetMapper objects so that the - ItemWriter will recieve the correct items. The - PrefixMatchingCompositeLineMapper makes this - easy by allowing maps of prefixes to - LineTokenizers and prefixes to - FieldSetMappers to be configured: + Everything between the line starting with 'HEA' and the line + starting with 'FOT' is considered one record. The + PrefixMatchingCompositeLineTokenizer makes this easier by matching the + prefix in a line with a particular tokenizer: - <bean id="orderFileLineMapper" - class="org.springframework.batch.item.file.mapping.PrefixMatchingCompositeLineMapper"> + <bean id="orderFileDescriptor" + class="org.springframework.batch.io.file.transform.PrefixMatchingCompositeLineTokenizer"> <property name="tokenizers"> - <map> - <entry key="USER" value-ref="userTokenizer" /> - <entry key="LINEA" value-ref="lineATokenizer" /> - <entry key="LINEB" value-ref="lineBTokenizer" /> - </map> - </property> - <property name="fieldSetMappers"> - <map> - <entry key="USER" value-ref="userFieldSetMapper" /> - <entry key="LINE" value-ref="lineFieldSetMapper" /> - </map> + <map> + <entry key="HEA" value-ref="headerRecordDescriptor" /> + <entry key="FOT" value-ref="footerRecordDescriptor" /> + <entry key="BCU" value-ref="businessCustomerLineDescriptor" /> + <entry key="NCU" value-ref="customerLineDescriptor" /> + <entry key="BAD" value-ref="billingAddressLineDescriptor" /> + <entry key="SAD" value-ref="shippingAddressLineDescriptor" /> + <entry key="BIN" value-ref="billingLineDescriptor" /> + <entry key="SIN" value-ref="shippingLineDescriptor" /> + <entry key="LIT" value-ref="itemLineDescriptor" /> + <entry key="" value-ref="defaultLineDescriptor" /> + </map> </property> </bean> - In this example, "LINEA" and "LINEB" have separate - LineTokenizers but they both use the same - FieldSetMapper. - - The PrefixMatchingCompositeLineMapper - makes use of the PatternMatcher's - matchPattern method in order to select the - correct delegate for each line. The pattern will always match the most - specific pattern possible, regardless of the order in the - configuration. So if "LINE" and "LINEA" were both listed as prefixes, - "LINEA" would match prefix "LINEA", while "LINEB" would match prefix - "LINE". Additionally, the empty string ("") can serve as a default - prefix by matching any line not matched by any other prefix. - - - <entry key="" value-ref="defaultLineTokenizer" /> - - - - There is also a - PrefixMatchingCompositeLineTokenizer that can - be used for tokenization alone. - - It is also common for a flat file to contain records that each - span multiple lines. To handle this situation, a more complex strategy - is required. A demonstration of this common patter can be found in - . + This ensures that the line will be parsed correctly, which is + especially important for fixed length input. Any users of the + FlatFileItemReader in this scenario must + continue calling read until the footer for + the record is returned, allowing them to return a complete order as + one 'item'.
Exception Handling in flat files - There are many scenarios when tokenizing a line may cause + There are many scenarios when tokenizing a line that cause exceptions to be thrown. Many flat files are imperfect and contain - records that aren't formatted correctly. Many users choose to skip - these erroneous lines, logging out the issue, original line, and line - number. These logs can later be inspected manually or or by another - batch job. For this reason, Spring Batch provides a hierarchy of - exceptions for handling parse exceptions: + records that aren't formatted correctly. Many users choose to skip the + lines causing these errors, logging out the issue, original line, and + line number, for manual inspection later. (or by another batch job) + For this reason, Spring Batch provides a hierarchy of exceptions for + handling parse exceptions: FlatFileParseException and FlatFileFormatException. FlatFileParseException is thrown by the @@ -1078,9 +1013,8 @@ FieldSet. However, if the number of column names doesn't match the number of columns found while tokenizing a line the FieldSet can't be created, and a - IncorrectTokenCountException is thrown, which - contains the number of tokens encountered, and the number - expected: + IncorrectTokenCountException is thrown, which contains the number of + tokens encountered, and the number expected: tokenizer.setNames(new String[] {"A", "B", "C", "D"}); @@ -1095,9 +1029,8 @@ - Because the tokenizer was configured with 4 column names, but - only 3 tokens were found in the file, an - IncorrectTokenCountException was + Because the tokenizer was configured with 4 columns, but only + 3 tokens were found in the file, an IncorrectTokenCountException was thrown.
@@ -1105,10 +1038,10 @@ IncorrectLineLengthException Files formatted in a fixed length format have additional - requirements when parsing because, unlike a delimited format, each - column must strictly adhere to its predefined width. If the total - line length doesn't add up to the widest value of this column, an - exception is thrown: + requirements when parsing because unlike a delimited format, each + column must strictly adhere to the width defined for it. If the + total line length doesn't add up to the widest value of this column, + an exception is thrown: tokenizer.setColumns(new Range[] { new Range(1, 5), new Range(6, 10), new Range(11, 15) }); @@ -1145,9 +1078,9 @@ The above example is almost identical to the one before it, - except that tokenizer.setStrict(false) was called. This setting - tells the tokenizer to not enforce line lengths when tokenizing the - line. A FieldSet is now correctly created and + except the tokenizer.setStrict(false) was called. This setting tells + the tokenizer to not enforce line lengths when tokenizing the line. + A FieldSet is now correctly created and returned. However, it will only contain empty tokens for the remaining values.
@@ -1158,18 +1091,17 @@ FlatFileItemWriter Writing out to flat files has the same problems and issues that - reading in from a file must overcome. A step must be able to write out - in either delimited or fixed length formats in a transactional + reading in from a file must overcome. It must be able to write out in + either delimited or fixed length formats in a transactional manner.
LineAggregator Just as the LineTokenizer interface is - necessary to take an item and turn it into a - String, file writing must have a way to - aggregate multiple fields into a single string for writing to a file. - In Spring Batch this is the + necessary to take an item and turn it into a string, file writing must + have a way to aggregate multiple fields into a single string for + writing to a file. In Spring Batch this is the LineAggregator: @@ -1194,9 +1126,9 @@ PassThroughLineAggregator The most basic implementation of the LineAggregator interface - is the PassThroughLineAggregator, which - simply assumes that the object is already a string, or that its - string representation is acceptable for writing: + is the PassThroughLineAggregator, which simply assumes that the + object is already a string, or that it's string representation is + acceptable for writing: public class PassThroughLineAggregator<T> implements LineAggregator<T> { @@ -1210,8 +1142,8 @@ The above implementation is useful if direct control of creating the string is required, but the advantages of a - FlatFileItemWriter, such as transaction and - restart support, are necessary. + FlatFileItemWriter, such as transaction and restart support, are + necessary.
@@ -1219,9 +1151,8 @@ Simplified File Writing Example Now that the LineAggregator interface and - its most basic implementation, - PassThroughLineAggregator, have been defined, - the basic flow of writing can be explained: + it's most basic implementation, PassThroughLineAggregator, has been + defined the basic flow of writing can be explained: @@ -1266,24 +1197,23 @@ FieldExtractor The above example may be useful for the most basic uses of a - writing to a file. However, most users of the - FlatFileItemWriter will will have a domain - object that needs to be written out, and thus must be converted into a - line. In file reading, the following was required: + writing to a file. However, most users of the FlatFileItemWriter will + will have a domain object that needs to be written out, and thus must + be converted into a line. In file reading, the following was + required: Read one line from the file. - Pass the string line into the - LineTokenizer#tokenize() method, in - order to retrieve a FieldSet + Pass the string line into the LineTokenizer#tokenize() + method, in order to retrieve a + FieldSet - Pass the FieldSet returned from - tokenizing to a FieldSetMapper, returning - the result from the ItemReader#read() + Pass the FieldSet returned from tokenizing to a + FieldSetMapper, returning the result from the ItemReader#read() method @@ -1305,9 +1235,8 @@ Because there is no way for the framework to know which fields - from the object need to be written out, a - FieldExtractor must be written to accomplish - the task of turning the item into an array: + from the object need to be written out, a FieldExtractor must be + written to accomplish the task: public interface FieldExtractor<T> { @@ -1320,7 +1249,7 @@ Implementations of the FieldExtractor interface should create an array from the fields of the provided - object, which can then be written out with a delimiter between the + object, which can then be written out with a delimited between the elements, or as part of a field-width line.
@@ -1330,21 +1259,18 @@ converted to an array, such as a Collection, needs to be written out. For example, a List could be passed through, in which case it only needs to be converted - to an Object array to be written out. For - this type of scenario the - PassThroughFieldExtractor can be used. It - should be noted, that if the object passed in is not an array, and - not a Collection, then an - Object array containing solely the item will - be returned. + to an Object array to be written out. For this type of scenario the + PassThroughFieldExtractor can be used. It should be noted, that if + the object passed in is not an array, and not a Collection, then an + Object array containing solely the item will be returned.
BeanWrapperFieldExtractor As with the BeanWrapperFieldSetMapper - described in the file reading section, it is often preferrable to - configure how to convert a domain object to an object array, rather + described in the file reading section, it is much preferrable to + configure how to convert an domain object to an object array, rather than writing the conversion yourself. The BeanWrapperFieldExtractor provides just this type of functionality: @@ -1369,10 +1295,9 @@ This extractor implementation has only one required property, the names of the fields to map. Just as the BeanWrapperFieldSetMapper needs field names - to map fields on the FieldSet to setters on - the provided object, the + to map fields on the FieldSet to setters on the provided object, the BeanWrapperFieldExtractor needs names to map - to getters for creating an object array. It is worth noting that the + to getters for creating an object array. It's worth noting that the order of the names determines the order of the fields within the array.
@@ -1383,17 +1308,24 @@ The most basic flat file format is one in which all fields are separated by a delimiter. This can be accomplished using a - DelimitedLineAggregator. The example below - writes out a simple domain object that represents a credit to a - customer account: + DelimitedLineAggregator. The example below writes out a simple domain + object that represents a credit to a customer account: public class CustomerCredit { private int id; + private String name; + private BigDecimal credit; + public CustomerCredit(int id, String name, BigDecimal credit) { + this.id = id; + this.name = name; + this.credit = credit; + } + //getters and setters removed for clarity } @@ -1423,19 +1355,19 @@ In this case, the BeanWrapperFieldExtractor described earlier in this chapter is used to turn the name and credit fields within - CustomerCredit into an object array, which is - then written out with commas between each field. + CustomerCredit into an object array, which is then written out with + commas between each field.
Fixed Width File Writing Example - Delimited is not the only type of flat file format. Many prefer + Delimited is not the only type of flat file format, many prefer to use a set width for each column to delineate between fields, which is usually referred to as 'fixed width'. Spring Batch supports this in - file writing via the FormatterLineAggregator. - Using the same CustomerCredit domain object - described above, it can be configured as follows: + file writing via the FormatterLineAggregator. Using the same + CustomerCredit domain object described above, it can be configured as + follows: <bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"> @@ -1462,12 +1394,9 @@ - The underlying implementation is built using the same - Formatter added as part of Java 5. The Java - Formatter is based on the - printf functionality of the C programming - language. Most details on how to configure a formatter can be found in - the javadoc of The underlying implementation is built using the same Formatter + added as part of Java 5. Most details on how to configure a formatter + can be found in the javadoc of Formatter.
@@ -1479,18 +1408,18 @@ opens the file if it exists, and throws an exception if it does not. File writing isn't quite so simple. At first glance it seems like a similar straight forward contract should exist for - FlatFileItemWriter: if the file already exists, - throw an exception, and if it does not, create it and start writing. + FlatFileItemWriter, if the file already exists, + throw an exception, if it does not, create it and start writing. However, potentially restarting a Job can cause - issues. In normal restart scenarios, the contract is reversed: if the - file exists, start writing to it from the last known good position, - and if it does not, throw an exception. However, what happens if the - file name for this job is always the same? In this case, you would - want to delete the file if it exists, unless it's a restart. Because - of this possibility, the FlatFileItemWriter - contains the property, shouldDeleteIfExists. - Setting this property to true will cause an existing file with the - same name to be deleted when the writer is opened. + issues. In normal restart scenarios, the contract is reversed, if the + file exists start writing to it from the last known good position, if + it does not, throw an exception. However, what happens if the file + name for this job is always the same? In this case, you would want to + delete the file if it exists, unless it's a restart. Because of this + possibility, the FlatFileItemWriter contains + the property, shouldDeleteIfExists. Setting + this property to true will cause an existing file with the same name + to be deleted when the writer is opened. @@ -1520,14 +1449,13 @@ - + + scale="80" width="" /> @@ -1547,14 +1475,13 @@ - + + format="PNG" scale="80" width="" /> @@ -1610,7 +1537,7 @@ - FragmentDeserializer - Unmarshalling + FragmentDeserializer - UnMarshalling facility provided by Spring OXM for mapping the XML fragment to an object. @@ -1641,7 +1568,7 @@ (i.e. root element) and the object type to bind. Then, similar to a FieldSet, the names of the other elements that map to fields within the object type are described as key/value pairs in - the map. In the configuration file we can use a Spring configuration + the map. In the configuration file we can use a spring configuration utility to describe the required alias as follows: @@ -1663,17 +1590,17 @@ </bean> - On input the reader reads the XML resource until it recognizes - that a new fragment is about to start (by matching the tag name by - default). The reader creates a standalone XML document from the fragment - (or at least makes it appear so) and passes the document to a - deserializer (typically a wrapper around a Spring OXM + On input the reader reads the XML resource until it recognizes a + new fragment is about to start (by matching the tag name by default). + The reader creates a standalone XML document from the fragment (or at + least makes it appear so) and passes the document to a deserializer + (typically a wrapper around a Spring OXM Unmarshaller) to map the XML to a Java object. - In summary, this procedure is analogous to the following scripted - Java code which uses the injection provided by the Spring - configuration: + In summary, if you were to see this in scripted code like Java the + injection provided by the spring configuration would look something like + the following: StaxEventItemReader xmlStaxEventItemReader = new StaxEventItemReader() @@ -1695,12 +1622,12 @@ CustomerCredit credit = null; while (hasNext) { - credit = xmlStaxEventItemReader.read(); - if (credit == null) { - hasNext = false; - } else { - println trade; - } + credit = xmlStaxEventItemReader.read(); + if (credit == null) { + hasNext = false; + } else { + println trade; + } } @@ -1752,7 +1679,7 @@ To summarize with a Java example, the following code illustrates all of the points discussed, demonstrating the programmatic setup of the - required properties: + required properties. StaxEventItemWriter staxItemWriter = new StaxEventItemWriter() FileSystemResource resource = new FileSystemResource(File.createTempFile("StaxEventWriterOutputSourceTests", "xml")) @@ -1783,9 +1710,9 @@ Multi-File Input It is a common requirement to process multiple files within a single - Step. Assuming the files all have the same - formatting, the MultiResourceItemReader supports - this type of input for both XML and flat file processing. Consider the + Step. Assuming the files are all formatted the + same, the MultiResourceItemReader supports this + type of input for both XML and flat file processing. Consider the following files in a directory: file-1.txt file-2.txt ignored.txt @@ -1796,7 +1723,7 @@ files by using wildcards: - <bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiResourceItemReader"> + <bean id="multiResourceReader" class="org.springframework.batch.item.SortedMultiResourceItemReader"> <property name="resources" value="classpath:data/multiResourceJob/input/file-*.txt" /> <property name="delegate" ref="flatFileItemReader" /> </bean> @@ -1817,9 +1744,9 @@ Like most enterprise application styles, a database is the central storage mechanism for batch. However, batch differs from other application - styles due to the sheer size of the datasets with which the system must - work. The Spring Core JdbcTemplate illustrates this - problem well. If you use JdbcTemplate with a + styles due to the sheer size of the datasets that must be worked with. The + Spring Core JdbcTemplate illustrates this problem + well. If you use JdbcTemplate with a RowMapper, the RowMapper will be called once for every result returned from the provided query. This causes few issues in scenarios where the dataset is small, but the @@ -1858,26 +1785,24 @@ + scale="80" width="66%" /> - + - This example illustrates the basic pattern. Given a 'FOO' table, + The example illustrates the basic pattern. Given a 'FOO' table, which has three columns: ID, NAME, and BAR, select all rows with an ID - greater than 1 but less than 7. This puts the beginning of the cursor + greater than one but less than 7. This puts the beginning of the cursor (row 1) on ID 2. The result of this row should be a completely mapped - Foo object. Calling read() again moves the - cursor to the next row, which is the Foo with an ID of 3. The results of - these reads will be written out after each - read, thus allowing the objects to be garbage - collected (assuming no instance variables are maintaining references to - them). + Foo object, calling read() again, moves the cursor to the next row, + which is the Foo with an ID of 3. The results of these reads will be + written out after each read, thus allowing the + objects to be garbage collected. (Assuming no instance variables are + maintaining references to them)
JdbcCursorItemReader @@ -1889,24 +1814,24 @@ DataSource. The following database schema will be used as an example: - CREATE TABLE CUSTOMER ( - ID BIGINT IDENTITY PRIMARY KEY, - NAME VARCHAR(45), - CREDIT FLOAT - ); + CREATE TABLE CUSTOMER ( + ID BIGINT IDENTITY PRIMARY KEY, + NAME VARCHAR(45), + CREDIT FLOAT +); Many people prefer to use a domain object for each row, so we'll use an implementation of the RowMapper interface to map a CustomerCredit object: - public class CustomerCreditRowMapper implements RowMapper { + public class CustomerCreditRowMapper implements RowMapper { - public static final String ID_COLUMN = "id"; - public static final String NAME_COLUMN = "name"; - public static final String CREDIT_COLUMN = "credit"; + public static final String ID_COLUMN = "id"; + public static final String NAME_COLUMN = "name"; + public static final String CREDIT_COLUMN = "credit"; - public Object mapRow(ResultSet rs, int rowNum) throws SQLException { + public Object mapRow(ResultSet rs, int rowNum) throws SQLException { CustomerCredit customerCredit = new CustomerCredit(); customerCredit.setId(rs.getInt(ID_COLUMN)); @@ -1914,15 +1839,16 @@ customerCredit.setCredit(rs.getBigDecimal(CREDIT_COLUMN)); return customerCredit; - } + } + } Because JdbcTemplate is so familiar to users of Spring, and the JdbcCursorItemReader - shares key interfaces with it, it is useful to see an example of how - to read in this data with JdbcTemplate, in - order to contrast it with the ItemReader. For - the purposes of this example, let's assume there are 1,000 rows in the + shares key interfaces with it, it's useful to see an example of how to + read in this data with JdbcTemplate, in order + to contrast it with the ItemReader. For the + purposes of this example, let's assume there are 1,000 rows in the CUSTOMER database. The first example will be using JdbcTemplate: @@ -2009,8 +1935,7 @@ Gives the Jdbc driver a hint as to the number of rows that should be fetched from the database when more rows are needed by the ResultSet object used - by the ItemReader. By default, no - hint is given. + by the ItemReader. By default, no hint is given. @@ -2025,10 +1950,10 @@ queryTimeout Sets the number of seconds the driver will wait for a - Statement object to execute to the - given number of seconds. If the limit is exceeded, a + Statement object to execute to the given number of seconds. + If the limit is exceeded, a DataAccessEception is thrown. - (Consult your driver vendor documentation for + (consult your driver vendor documentation for details). @@ -2036,24 +1961,23 @@ verifyCursorPosition Because the same ResultSet - held by the ItemReader is passed to - the RowMapper, it is possible for - users to call ResultSet.next() - themselves, which could cause issues with the reader's - internal count. Setting this value to true will cause an - exception to be thrown if the cursor position is not the - same after the RowMapper call as it - was before. + held by the ItemReader is passed to the + RowMapper, it's possible for users to + call ResultSet.next() themselves, which could cause issues + with the reader's internal count. Settings this value to + true will cause an exception to be thrown if the cursor + position is not the same after the + RowMapper call as it was + before. saveState Indicates whether or not the reader's state should be - saved in the ExecutionContext - provided by - ItemStream#update(ExecutionContext) - The default value is false. + saved in the ExecutionContext provided by + ItemStream#update(ExecutionContext) The default value is + false. @@ -2063,9 +1987,9 @@ supports setting the absolute row on a ResultSet. It is recommended that this is set to true for Jdbc drivers that supports - ResultSet.absolute() as it may - improve performance, especially if a step fails while - working with a large data set. + ResultSet.absolute() as it may improve performance, + especially if a step fails while working with a large data + set. @@ -2077,8 +2001,8 @@ which is the default, then the cursor will be opened using its own connection and will not participate in any transactions started for the rest of the step processing. If - you set this flag to true then you must wrap the - DataSource in an + you set this flag to true then you must wrap the DataSource + in an ExtendedConnectionDataSourceProxy to prevent the connection from being closed and released after each commit. When you set this option to true then the @@ -2099,25 +2023,26 @@ HibernateCursorItemReader Just as normal Spring users make important decisions about - whether or not to use ORM solutions, which affect whether or not they + whether or not to use ORM solutions, which affects whether or not they use a JdbcTemplate or a HibernateTemplate, Spring Batch users have the same options. HibernateCursorItemReader is the Hibernate implementation of the cursor technique. Hibernate's usage in batch has been fairly controversial. This has largely been because - Hibernate was originally developed to support online application + hibernate was originally developed to support online application styles. However, that doesn't mean it can't be used for batch processing. The easiest approach for solving this problem is to use a StatelessSession rather than a standard session. This removes all of the caching and dirty checking hibernate - employs that can cause issues in a batch scenario. For more - information on the differences between stateless and normal hibernate - sessions, refer to the documentation of your specific hibernate - release. The HibernateCursorItemReader allows - you to declare an HQL statement and pass in a - SessionFactory, which will pass back one item - per call to read in the same basic fashion as - the JdbcCursorItemReader. Below is an example + employs that can cause issues when using it in a batch scenario. For + more information on the differences between stateless and normal + hibernate sessions, refer to the documentation of your specific + hibernate release. The + HibernateCursorItemReader allows you to declare + an HQL statement and pass in a SessionFactory, + which will pass back one item per call to + read in the same basic fashion as the + JdbcCursorItemReader. Below is an example configuration using the same 'customer credit' example as the JDBC reader: @@ -2244,7 +2169,7 @@ similar to the Hibernate StatelessSession so we have to use other features provided by the JPA specification. Since JPA supports paging, this is a natural choice when it comes to using - JPA for batch processing. After each page is read, the entities will + JPA for batch processing. After each page is read the entities will become detached and the persistence context will be cleared in order to allow the entities to be garbage collected once the page is processed. @@ -2354,8 +2279,8 @@ - + If items are buffered before being written out, any errors encountered will not be thrown until the buffer is flushed just @@ -2376,12 +2301,12 @@ - + - + @@ -2408,9 +2333,9 @@ existing service needs to act as an ItemReader or ItemWriter, either to satisfy the dependency of another Spring Batch class, or because it truly is the main - ItemReader for a step. It is fairly trivial to - write an adaptor class for each service that needs wrapping, but because - it is such a common concern, Spring Batch provides implementations: + ItemReader for a step. Its fairly trivial to write + an adaptor class for each service that needs wrapping, but because its + such a common concern, Spring Batch provides implementations: ItemReaderAdapter and ItemWriterAdapter. Both classes implement the standard Spring method invoking the delegate pattern and are fairly simple @@ -2522,8 +2447,8 @@ input data to indicate whether or not it has been processed. When a particular record is being read (or written out) the processed flag is flipped from false to true. The SQL statement can then contain an extra - statement in the where clause, such as "where PROCESSED_IND = false", - thereby ensuring that only unprocessed records will be returned in the + statement in the where clause, such as: "where PROCESSED_IND = false", + thereby insuring that only unprocessed records will be returned in the case of a restart. In this scenario, it is preferable to not store any state, such as the current row number, since it will be irrelevant upon restart. For this reason, all readers and writers include the 'saveState' @@ -2539,12 +2464,12 @@ <property name="saveState" value="false" /> <property name="sql"> <value> - SELECT games.player_id, games.year_no, SUM(COMPLETES), - SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), - SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), - SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) - from games, players where players.player_id = - games.player_id group by games.player_id, games.year_no + SELECT games.player_id, games.year_no, SUM(COMPLETES), + SUM(ATTEMPTS), SUM(PASSING_YARDS), SUM(PASSING_TD), + SUM(INTERCEPTIONS), SUM(RUSHES), SUM(RUSH_YARDS), + SUM(RECEPTIONS), SUM(RECEPTIONS_YARDS), SUM(TOTAL_TD) + from games, players where players.player_id = + games.player_id group by games.player_id, games.year_no </value> </property> </bean> @@ -2553,7 +2478,7 @@ The ItemReader configured above will not make any entries in the ExecutionContext for any - executions in which it participates. + executions it participates in.
@@ -2601,9 +2526,9 @@ - This very simple class takes a list of items, and returns them one - at a time, removing each from the list. When the list is empty, it - returns null, thus satisfying the most basic requirements of an + This very simple class takes a list of items, and returns one at a + time, removing it from the list. When the list empty, it returns null, + thus satisfying the most basic requirements of an ItemReader, as illustrated below: List<String> items = new ArrayList<String>(); @@ -2626,15 +2551,16 @@ goes out, and processing begins again, the ItemReader must start at the beginning. This is actually valid in many scenarios, but it is sometimes preferable that - a batch job starts where it left off. The key discriminant is often - whether the reader is stateful or stateless. A stateless reader does - not need to worry about restartability, but a stateful one has to try - and reconstitute its last known state on restart. For this reason, we - recommend that you keep custom readers stateless if possible, so you - don't have to worry about restartability. + a batch job starts off at where it left off. The key discriminant is + often whether the reader is stateful or stateless. A stateless reader + does not need to worry about restartability, but a stateful one has to + try and reconstitute its last known state on restart. For this reason, + we recommend that you keep custom readers stateless as far as + possible, so you don't have to worry about restartability. - If you do need to store state, then the - ItemStream interface should be used: + If you do need to store state, then in Spring Batch, this is + implemented with the ItemStream + interface: public class CustomItemReader<T> implements ItemReader<T>, ItemStream { @@ -2665,23 +2591,22 @@ } } + public void close(ExecutionContext executionContext) throws ItemStreamException {} + public void update(ExecutionContext executionContext) throws ItemStreamException { executionContext.putLong(CURRENT_INDEX, new Long(currentIndex).longValue()); }; - - public void close() throws ItemStreamException {} } - On each call to the ItemStream + On each call to ItemStream update method, the current index of the ItemReader will be stored in the provided ExecutionContext with a key of 'current.index'. When the ItemStream open method is called, the ExecutionContext is - checked to see if it contains an entry with that key. If the key is - found, then the current index is moved to that location. This is a - fairly trivial example, but it still meets the general - contract: + checked to see if it contains an entry with that key, and if so the + current index is moved to that location. This is a fairly trivial + example, but it still meets the general contract: ExecutionContext executionContext = new ExecutionContext(); ((ItemStream)itemReader).open(executionContext); @@ -2704,17 +2629,14 @@ It is also worth noting that the key used within the ExecutionContext should not be trivial. That is because the same ExecutionContext is used for - all ItemStreams within a - Step. In most cases, simply prepending the key - with the class name should be enough to guarantee uniqueness. However, - in the rare cases where two of the same type of - ItemStream are used in the same step (which can - happen if two files are need for output) then a more unique name will - be needed. For this reason, many of the Spring Batch - ItemReader and - ItemWriter implementations have a - setName() property that allows this key name - to be overridden. + all ItemStreams within a Step. In most cases, + simply prepending the key with the class name should be enough to + guarantee uniqueness. However, in the rare cases where two of the same + type of ItemStream are used in the same step + (which can happen if two files are need for output) then a more unique + name will be needed. For this reason, many of the Spring Batch + ItemReader and ItemWriters have a setName() property that allows this + key name to be overridden.
@@ -2725,9 +2647,8 @@ in many ways to the ItemReader example above, but differs in enough ways as to warrant its own example. However, adding restartability is essentially the same, so it won't be covered in this - example. As with the ItemReader example, a - List will be used in order to keep the example as - simple as possible: + example. As with the ItemReader example, a List + will be used in order to keep the example as simple as possible: public class CustomItemWriter<T> implements ItemWriter<T> { @@ -2765,8 +2686,9 @@ ItemStream as well as ItemWriter. Remember also that the client of the writer needs to be aware of the ItemStream, - so you may need to register it as a stream in the configuration - xml. + so you may need to register it with a factory bean (e.g. one of the + StepFactoryBean implementations in Spring Batch + Core). diff --git a/docs/src/site/docbook/reference/schema-appendix.xml b/docs/src/site/docbook/reference/schema-appendix.xml index cc6b0ae1b..49e06c15e 100644 --- a/docs/src/site/docbook/reference/schema-appendix.xml +++ b/docs/src/site/docbook/reference/schema-appendix.xml @@ -16,7 +16,7 @@ respectively. ExecutionContext maps to both BATCH_JOB_EXECUTION_CONTEXT and BATCH_STEP_EXECUTION_CONTEXT. The JobRepository is responsible for saving and storing - each Java object into its correct table. The following appendix describes + each Java object into it's correct table. The following appendix describes the meta-data tables in detail, along with many of the design decisions that were made when creating them. When viewing the various table creation statements below, it is important to realize that the data types used are @@ -26,15 +26,14 @@ their relationships to one another: - + + scale="80" width="" /> - + @@ -49,7 +48,7 @@ to try and save the value, if the version number has change it will throw OptimisticLockingFailureException, indicating there has been an error with concurrent access. This check is - necessary since, even though different batch jobs may be running in + necessary, since even though different batch jobs may be running in different machines, they are all using the same database tables. @@ -57,11 +56,11 @@ Identity BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, and BATCH_STEP_EXECUTION - each contain columns ending in _ID. These fields act as primary keys for - their respective tables. However, they are not database generated keys, - but rather they are generated by separate sequences. This is necessary - because after inserting one of the domain objects into the database, the - key it is given needs to be set on the actual object so that they can be + each contain columns ending in _ID, which act as primary keys for their + respective tables. However, they are not database generated keys, but + rather are generated by separate sequences. This is necessary because + after inserting one of the domain objects into the database, the key it + is given needs to be set on the actual object, so that they can be uniquely identified in Java. Newer database drivers (Jdbc 3.0 and up) support this feature with database generated keys, but rather than requiring it, sequences were used. Each variation of the schema will @@ -431,8 +430,8 @@ INSERT INTO BATCH_JOB_SEQ values(0); The BATCH_JOB_EXECUTION_CONTEXT table holds all information relevant to an Job's ExecutionContext. There is exactly one - Job ExecutionContext per - JobExecution, and it contains all of the job-level + ExecutionContext per + StepExecution, and it contains all of the job-level data that is needed for a particular job execution. This data typically represents the state that must be retrieved after a failure so that a JobInstance can 'start from where it left diff --git a/docs/src/site/docbook/reference/spring-batch-intro.xml b/docs/src/site/docbook/reference/spring-batch-intro.xml index b67b3f70a..0bbd577b7 100644 --- a/docs/src/site/docbook/reference/spring-batch-intro.xml +++ b/docs/src/site/docbook/reference/spring-batch-intro.xml @@ -173,14 +173,13 @@ architecture that supports the extensibility and ease of use for end-user developers. - + + format="PNG" scale="80" scalefit="" width="35%" />
Figure 3.1: XML InputFigure 3.2: OXM Binding Figure 1.1: Spring Batch Layered diff --git a/docs/src/site/docbook/reference/step.xml b/docs/src/site/docbook/reference/step.xml index 3b2e766e6..81f4fc455 100644 --- a/docs/src/site/docbook/reference/step.xml +++ b/docs/src/site/docbook/reference/step.xml @@ -23,16 +23,15 @@ - +
Chunk-Oriented Processing - Spring Batch uses a 'Chunk Oriented' processing style within its + Spring Batch uses a 'Chunk Oriented' processing style within it's most common implementation. Chunk oriented processing refers to reading the data one at a time, and creating 'chunks' that will be written out, within a transaction boundary. One item is read in from an @@ -44,13 +43,13 @@ @@ -61,8 +60,7 @@ List items = new Arraylist(); for(int i = 0; i < commitInterval; i++){ - Object item = itemReader.read() - Object processedItem = itemProcessor.process(item); + Object processedItem = itemProcessor.process(itemReader.read()); items.add(processedItem); } itemWriter.write(items); @@ -121,9 +119,8 @@ It should be noted that, job-repository defaults to "jobRepository" and transaction-manager defaults to "transactionManger". - Furthermore, the ItemProcessor is optional, not - required, since the item could be directly passed from the reader to the - writer. + Furthermore, the ItemProcessor is not required, since the item could be + directly passed from the reader to the writer.
@@ -132,14 +129,13 @@ As mentioned above, a step reads in and writes out items, periodically committing using the supplied PlatformTransactionManager. With a - commit-interval of 1, it will commit after writing each individual item. - This is less than ideal in many situations, since beginning and - committing a transaction is expensive. Ideally, it is preferable to - process as many items as possible in each transaction, which is - completely dependent upon the type of data being processed and the - resources with which the step is interacting. For this reason, the - number of items that are processed within a commit can be - configured. + commit-interval of 1, it will commit after writing only one item. This + is less than ideal in many situations, since beginning and committing a + transaction is expensive. Ideally, it is preferable to process as many + items as possible in each transaction, which is completely dependent + upon the type of data being processed and the resources with which the + step is interacting. For this reason, the number of items that are + processed within a commit can be configured. <job id="sampleJob"> @@ -171,15 +167,14 @@ Setting a StartLimit There are many scenarios where you may want to control the - number of times a Step may be started. For - example, a particular Step might need to be - configured so that it only runs once because it invalidates some - resource that must be fixed manually before it can be run again. This - is configurable on the step level, since different steps may have - different requirements. A Step that may only be - executed once can exist as part of the same Job - as a Step that can be run infinitely. Below is - an example start limit configuration: + number of times a Step may be started. An + example is a Step that may be run only once, + usually because it invalidates some resource that must be fixed + manually before it can be run again. This is configurable on the step + level, since different steps have different requirements. One Step + that may only be executed once can exist as part of the same + Job as Step that can be + run infinitely. Below is an example start limit configuration: <step name="step1"> @@ -327,7 +322,7 @@ playerSummarization is not start, and the job is immediately killed, since this is the third execution of playerSummarization, - and its limit is only 2. The limit must either be raised, or the + and it's limit is only 2. The limit must either be raised, or the Job must be executed as a new JobInstance. @@ -365,15 +360,15 @@ FlatFileParseException is thrown, it will be skipped and counted against the total skip limit of 10. It should be noted that any failures encountered while reading will not count against - the skip limit. In other words, the skip limit is only incremented on - writes (regardless of success or failure). + the commit interval. In other words, the commit interval is only + incremented on writes (regardless of success or failure).
One problem with the example above is that any other exception besides a FlatFileParseException will cause the Job to fail. In certain scenarios this may be the - correct behavior. However, in other scenarios it may be easier to + correct behaviour, however, in certain scenarios it may be easier to identify which exceptions should cause failure and skip everything else: <step name="step1"> @@ -402,8 +397,8 @@ In most cases you want an exception to cause either a skip or Step failure. However, not all exceptions are deterministic. If a FlatFileParseException is - encountered while reading, it will always be thrown for that record; - resetting the ItemReader will not help. However, + encountered while reading, it will always be thrown for that record. + Resetting the ItemReader will not help. However, for other exceptions, such as a DeadlockLoserDataAccessException, which indicates that the current process has attempted to update a record that another @@ -442,8 +437,7 @@ the Step can be configured with a list of exceptions that should not cause rollback. The transaction-attribute attribute is a comma-separated list. Prefixing a class name with the "+" - symbol will indicate that that exception should not cause - rollback. + symbol will indicate that exception should not cause rollback. <step name="step1"> @@ -455,8 +449,8 @@ Transaction attributes can be used to control multiple other - settings such as isolation and propagation behavior. More information on - setting transaction attributes can be found in the spring core + settings such as isolation and propagation behaviour. More information + on setting transaction attributes can be found in the spring core documentation.
@@ -469,8 +463,8 @@ top of a transactional resource, such as a JMS queue. In this case, since the queue is tied to the transaction that is rolled back, the messages that have been pulled from the queue will be put back on. For - this reason, the step can be configured to not buffer the - items: + this reason, the step can be configured to not buffer the items: + <step name="step1"> @@ -488,21 +482,21 @@ The step has to take care of ItemStream callbacks at the necessary points in its lifecycle. (for more - information on the ItemStream interface, please - refer to ) This is vital if a step fails, - and might need to be restarted, because the - ItemStream interface is where the step gets the - information it needs about persistent state between executions. + information on the ItemStream interface, please refer to ) This is vital if a step fails, and might need + to be restarted, because the ItemStream interface + is where the step gets the information it needs about persistent state + between executions. If the ItemReader, ItemProcessor, or ItemWriter itself implements the ItemStream interface, then these will be registered automatically. Any other streams need to be registered - separately. This is often the case where there are indirect dependencies - such as delegates being injected into the reader and writer. A stream - can be registered on the Step through the - 'streams' element, as illustrated below: + separately. This is often the case where there are indirect + dependencies, like delegates being injected into the reader and writer. + To a stream it can be injected into the Step + through the 'streams' element, as illustrated below: <step name="step1"> @@ -531,11 +525,11 @@ ItemStream, but both of its delegates are. Therefore, both delegate writers must be explicitly registered as streams in order for the framework to handle them correctly. The - ItemReader does not need to be explicitly - registered as a stream because it is a direct property of the + ItemReader does not need to explicitly registered + as a stream because it is a direct property of the Step. The step will now be restartable and the - state of the reader and writer will be correctly persisted in the event - of a failure. + state of the reader and writer will be correctly persisted in case of a + failure.
@@ -565,7 +559,7 @@ In addition to the StepListener interfaces, - annotations are provided to address the same concerns. + annotations are provided address the same concerns.
StepExecutionListener @@ -768,10 +762,9 @@
SkipListener - ItemReadListener, - ItemProcessListener, and - ItemWriteListner all provide mechanisms for - being notified of errors, but none will inform you that a record has + Both ItemReadListener and + ItemWriteListner provide a mechanism for being + notified of errors, but neither one will inform you that a record has actually been skipped. onWriteError, for example, will be called even if an item is retried and successful. For this reason, there is a separate interface for tracking skipped @@ -782,9 +775,9 @@ void onSkipInRead(Throwable t); - void onSkipInProcess(T item, Throwable t); - void onSkipInWrite(S item, Throwable t); + + void onSkipInProcess(T item, Throwable t); } @@ -820,8 +813,8 @@ SkipListener is to log out a skipped item, so that another batch process or even human process can be used to evaluate and fix the issue leading to the skip. Because there are - many cases in which the original transaction may be rolled back, - Spring Batch makes two guarantees: + many cases in which the original trasaction may be rolledback, + Spring Batch makes two garantees: @@ -869,8 +862,8 @@ - TaskletStep will automatically register the - tasklet as StepListener if it implements this + TaskletStep will automatically register the tasklet as + StepExecutionListener if it implements this interface @@ -902,7 +895,7 @@ Example Tasklet implementation Many batch jobs contain steps that must be done before the main - processing begins in order to set up various resources or after + processing begins in order to set up various resources, or after processing has completed to cleanup those resources. In the case of a job that works heavily with files, it is often necessary to delete certain files locally after they have been uploaded successfully to @@ -958,6 +951,27 @@ </property> </bean> + +
+ +
+ Executing System Commands + + Many batch jobs may require that an external command be called + from within the batch job. Such a process could be kicked off separately + by the scheduler, but the advantage of common meta-data about the run + would be lost. Furthermore, a multi-step job would also need to be split + up into multiple jobs as well. Because the need is so common, Spring + Batch provides a Tasklet implementation for + calling system commands: + + + <bean class="org.springframework.batch.sample.tasklet.SystemCommandTasklet"> + <property name="command" value="echo hello" /> + <!-- 5 second timeout for the command to complete --> + <property name="timeout" value="5000" /> + </bean> +
@@ -965,11 +979,11 @@
Controlling Step Flow - With the ability to group steps together within an owning job comes + With the ability to group steps together within an owning job, comes the need to be able to control how the job 'flows' from one step to another. The failure of a Step doesn't necessarily - mean that the Job should fail. Furthermore, there - may be more than one type of 'success' which determines which + mean that the Job should fail. Further, there may + be more than one type of 'success', which determines which Step should be executed next. Depending upon how a group of Steps is configured, certain steps may not even be processed at all. @@ -987,14 +1001,13 @@ - + - This can be achieved using the 'next' attribute of the step - element: + This can be achieved using the 'next' attribute of + Step: <job id="job"> @@ -1003,85 +1016,47 @@ <step name="stepC" /> </job> -In the scenario above, 'step A' will execute first because it - is the first Step listed. If 'step A' completes - normally, then 'step B' will execute, and so on. However, if 'step A' - fails, then the entire Job will fail and 'step B' - will not execute. - - - With the Spring Batch namespace, the first step listed in the - configuration will always be the first step - executed by the Job. The order of the other - step elements does not matter, but the first step must always appear - first in the xml. - +In the scenario above, 'step A' will execute first. If 'step + A' completes normally, then 'step B' will execute and so on. However, if + 'step A' fails, then the entire Job will fail and + 'step B' will not execute.
Conditional Flow - In the example above, there are only two possibilities: + In the example above, there's only two possibilities: - The Step is successful and the next - Step should be executed. + The Step is successful and the next Step should be + executed - The Step failed and thus the - Job should fail. + The Step failed and thus the Job should fail. - In many cases, this may be sufficient. However, what about a - scenario in which the failure of a Step should - trigger a different Step, rather than causing - failure? + In many cases this may be sufficient. However, what about a + scenario in which the failure of a Step should trigger a different Step, + rather than causing failure? - + - In order to handle more complex scenarios, the - Spring Batch namespace allows transition elements to be defined within - the step element. One such transition is the "next" element. Like the - "next" attribute, the "next" element will tell the - Job which Step to execute - next. However, unlike the attribute, any number of "next" elements are - allowed on a given Step, and there is no default - behavior the the case of failure. This means that if transition elements - are used, then all of the behavior for the Step's - transitions must be defined explicitly. Note also that a single step - cannot have both a "next" attribute and a transtion element. - - The next element specifies a pattern to match and the step to - execute next: - - - <job id="job"> - <step name="stepA"> - <next on="FAILED" to="stepB" /> - <next on="*" to="stepC" /> - </step> - <step name="stepB" next="stepC" /> - <step name="stepC" /> - </job> - - - - The "on" attribute of a transition element uses a simple - pattern-matching scheme to match the ExitStatus - that results from the exeution of the Step. Only - two special characters are allowed in the pattern: + In order to handle this scenario, the next step can be determined + based on the result of the step by adding a next element to the Step. + The "on" attribute uses a simple pattern-matching scheme to match the + exit code of the Step to the various next elements declared. Only two + special characters are allowed: @@ -1096,17 +1071,27 @@ For example, "c*t" will match "cat" and "count", while "c?t" will match "cat" but not "count". - While there is no limit to the number of transition elements on a - Step, if the Step's - execution results in an ExitStatus that is not - covered by an element, then the framework will throw an exception and - the Job will fail. It is important to note that - the framework will automatically order transitions from most specific to - least specific. This means that even if the elements were swapped for - "stepA" in the example above, an ExitStatus of - "FAILED" would still go to "stepB". + Any number of "next" elements is allowed, but if the step has an + exit code that is not covered by a "next" element, then the framework + will throw an exception and the job will fail. It is important to note + that the framework will automatically order transitions from most + specific to least specific. So even if the "next" elements were swapped + for "stepA" below, an exit status of "FAILED" would still go to + "stepB". -
+ + <job id="job"> + <step name="stepA"> + <next on="FAILED" to="stepB" /> + <next on="*" to="stepC" /> + </step> + <step name="stepB" next="stepC" /> + <step name="stepC" /> + </job> + + + +
Batch Status vs. Exit Status When configuring a Job for conditional @@ -1115,11 +1100,11 @@ ExitStatus. BatchStatus is an enumeration that is a property of both JobExecution and - StepExecution and is used by the framework to + StepExecution, and is used by the framework to record the status of a Job or Step. It can be one of the following values: COMPLETED, STARTING, STARTED, FAILED, STOPPING, STOPPED, or UNKNOWN. - Most of them are self explanatory: COMPLETED is the status set when a + Most of them are self explanatory, COMPLETED is the status set when a step or job has completed successfully, FAILED is set when it fails, and so on. The example above contains the following 'next' element: @@ -1131,18 +1116,18 @@ At first glance, it would appear that the 'on' attribute references the BatchStatus of the - Step to which it belongs. However, it actually - references the ExitStatus of the - Step. As the name implies, - ExitStatus represents the status of a - Step after it finishes execution. More - specifically, the 'next' element above references the exit code of the + Step it belongs to. However, it references the + ExitStatus of the Step. + As the name implies, ExitStatus represents the + status of a Step after it finishes execution. + More specifically, the 'next' element above references the + ExitCode of the ExitStatus. To write it in English, it says: "go to stepB if the exit code is FAILED". By default, the exit code is always the same as the BatchStatus for the Step, which is why the entry above works. However, what if the exit code needs to be different? A good example comes from the skip sample - job within the samples project: + job, within the samples project: <step name="step1"> @@ -1157,189 +1142,83 @@ - The Step failed, in which case the - job should fail. + The step failed, in which case the job should fail. - The Step completed - successfully. + The Step completed successfully. - The Step completed successfully, but - with an exit code of 'COMPLETED WITH SKIPS'. In this case, a - different step should be run to handle the errors. + The Step completed successfully, but with an exit code of + 'COMPLETED WITH SKIPS'. In this case, a different step should be + run to handle the errors. - The above configuration will work. However, something needs to + The above configuration will work, however, something needs to change the exit code based on the condition of the execution having skipped records: - public class SkipCheckingListener extends StepExecutionListenerSupport { + public class SkipCheckingListener implements StepExecutionListener { - public ExitStatus afterStep(StepExecution stepExecution) { - if (!stepExecution.getExitStatus().getExitCode().equals(ExitStatus.FAILED.getExitCode()) - && stepExecution.getSkipCount() > 0) { - return new ExitStatus("COMPLETED WITH SKIPS"); - } else { - return null; - } + public ExitStatus afterStep(StepExecution stepExecution) { + if (!stepExecution.getExitStatus().getExitCode().equals(ExitStatus.FAILED.getExitCode()) + && stepExecution.getSkipCount() > 0) { + return new ExitStatus("COMPLETED WITH SKIPS"); + } else { + return null; } + } - } + ... +} The above code is a StepExecutionListener that first checks to make sure the Step was successful, and next if the skip count on the StepExecution is higher than 0. If both - conditions are met, a new ExitStatus with an - exit code of "COMPLETED WITH SKIPS" is returned. + conditions are met, a new ExitStatus with an exit code of "COMPLETED + WITH SKIPS" is returned.
Configuring for Stop - After the discussion of BatchStatus and - ExitStatus, one might wonder how the - BatchStatus and ExitStatus - are determined for the Job. While these statuses - are determined for the Step by the code that is - executed, the statuses for the Job will be - determined based on the configuration. + If it is desired that the batch job stop under certain conditions, + then either the "stop" tag or the "end" tag may be used. - So far, all of the job configurations discussed have had at least - one final Step with no transitions. For example, - after the following step executes, the Job will - end: + The "stop" tag indicates the job should stop processing with an + exit status of "STOPPED". The "to" attribute tells the framework which + step should be first when the job is subsequently restarted. This + mechanism allows the job to pause temporarily. - <step name="stepC" /> + On the other hand, the "end" tag will stop the job but does not + allow for a "to" attribute. The "status" attribute is optional. It will + determine the exit status of the step if the flow ends at that location. + The only legal values for the "status" are "COMPLETED", "FAILED", and + "STOPPED". If no status is specified, then the default is + "COMPLETED". - If no transitions are defined for a Step, - then the Job's statuses will be defined as - follows: - - - - If the Step ends with - ExitStatus FAILED, then the - Job's BatchStatus and - ExitStatus will both be FAILED. - - - - Otherwise, the the Job's - BatchStatus and - ExitStatus will both be COMPLETED. - - - - While this method of terminating a batch job is sufficient for - some batch jobs, such as a simple sequential step job, custom defined - job-stopping scenarios may be required. For this purpose, Spring Batch - provides three transition elements to stop a Job - (in addition to the "next" element - that we discussed previously). Each of these stopping elements will stop - a Job with a particular - BatchStatus. It is important to note that the - stop transition elements will have no effect on either the - BatchStatus or ExitStatus - of any Steps in the Job: - these elements will only affect the final statuses of the - Job. For example, it is possible for every step - in a job to have a status of FAILED but the job to have a status of - COMPLETED, or vise versa. - -
- The 'End' Element - - The 'end' element instructs a Job to stop - with a BatchStatus of COMPLETED. A - Job that has finished with status COMPLETED - cannot be restarted (the framework will throw a - JobInstanceAlreadyCompleteException). The 'end' - element also allows for an optional 'status' attribute that can be - used to customize the ExitStatus of the - Job. If no 'status' attribute is given, then - the ExitStatus will be "COMPLETED" by default, - to match the BatchStatus. - - In the following scenario, if step2 fails, then the - Job will stop with a - BatchStatus of COMPLETE and an - ExitStatus of "COMPLETED" and step3 will not - execute; otherwise, execution will move to step3. Additionally, if - step2 fails, the Job will not be - restartable. - - <step name="step1" next="step2"> + + <step name="step1"> + <stop on="COMPLETED" to="step2"/> + </step> <step name="step2"> - <end on="FAILED"/> - <next on="*" to="step3"/> + <next on="FOO" to="step3"/> + <end on="*" status="FAILED"/> </step> - <step name="step3"> -
+ <step name="step3" /> -
- The 'Fail' Element - - The 'fail' element instructs a Job to - stop with a BatchStatus of FAILED. Unlike the - 'end' element, the 'fail' element will not prevent the - Job from being restarted. The 'fail' element - also allows for an optional 'status' attribute that can be used to - customize the ExitStatus of the - Job. If no 'status' attribute is given, then - the ExitStatus will be "FAILED" by default, to - match the BatchStatus. - - In the following scenario, if step2 fails, then the - Job will stop with a - BatchStatus of FAILED and an - ExitStatus of "EARLY TERMINATION" and step3 - will not execute; otherwise, execution will move to step3. - Additionally, if step2 fails, and the Job is - restarted, then execution will begin again on step2. - - <step name="step1" next="step2"> - <step name="step2"> - <fail on="FAILED" status="EARLY TERMINATION"/> - <next on="*" to="step3"/> - </step> - <step name="step3"> -
- -
- The 'Pause' Element - - The 'pause' element instructs a Job to - stop with a BatchStatus of STOPPED. Pausing a - Job is a meant to be a temporary break in - processing so that the operator can take some action before restarting - the Job. The 'pause' element requires a 'to' - attribute that specifies the step where execution should pick up once - the Job. - - In the following scenario, if step1 finsihes with COMPLETE, then - the job will then stop. Once it is restarted, execution will begin on - step2. - - <step name="step1"> - <pause on="COMPLETED" to="step2"/> - </step> - <step name="step2"/> -
+
Programmatic flow decisions - In some situations, more information than the - ExitStatus may be required to decide which step - to execute next. In this case, a + In some situations, more information than the exit status may be + required to decide which step to execute next. In this case, a JobExecutionDecider can be used to assist in the decision. @@ -1381,43 +1260,16 @@
- Split Flows + Late binding of Job and Step Attributes - Every scenario described so far has involved a - Job that executes its - Steps one at a time in a linear fashion. In - addition to this typical style, the Spring Batch namespace also allows - for a job to be configured with parallel flows using the 'split' - element. As is seen below, the 'split' element contains one or more - 'flow' elements, where entire separate flows can be defined. A 'split' - element may also contain any of the previously discussed transition - elements such as the 'next' attribute or the 'next', 'end', 'fail', or - 'pause' elements. + Both the XML and Flat File examples above use the Spring + Resource abstraction to obtain a file . This + works because Resource has a + getFile method, which returns a + java.io.File. Both XML and Flat File resources + can be configured using standard Spring constructs: - <split id="split1" next="step4"> - <flow> - <step name="step1" next="step2"/> - <step name="step2"/> - </flow> - <flow> - <step name="step3"/> - </flow> - </split> - <step name="step4"/> -
-
- -
- Late binding of Job and Step Attributes - - Both the XML and Flat File examples above use the Spring - Resource abstraction to obtain a file . This works - because Resource has a getFile - method, which returns a java.io.File. Both XML and - Flat File resources can be configured using standard Spring - constructs: - - + <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" @@ -1426,15 +1278,15 @@ - The above Resource will load the file from - the file system location specified. Note that absolute locations have to - start with a double slash ("//"). In most spring applications, this - solution is good enough because the names of these are known at compile - time. However, in batch scenarios, the file name may need to be determined - at runtime as a parameter to the job. This could be solved using '-D' - parameters, i.e. a system property: + The above Resource will load the file from + the file system, at the location specified. Note that absolute locations + have to start with a double slash ("//"). In most spring applications, + this solution is good enough because the names of these are known at + compile time. However, in batch scenarios, the file name may need to be + determined at runtime as a parameter to the job. This could be solved + using '-D' parameters, i.e. a system property: - + <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" value="${input.file.name}" /> @@ -1442,21 +1294,20 @@ - All that would be required for this solution to work would be a - system argument (-Dinput.file.name="file://file.txt"). (Note that although - a PropertyPlaceholderConfigurer can be used here, - it is not necessary if the system property is always set because the - ResourceEditor in Spring already filters and does - placeholder replacement on system properties.) + All that would be required for this solution to work would be a + system argument (-Dinput.file.name="file://file.txt"). (Note that + although a PropertyPlaceholderConfigurer can be + used here, it is not necessary if the system property is always set + because the ResourceEditor in Spring already + filters and does placeholder replacement on system properties.) - Often in a batch setting it is preferable to parameterize the file - name in the JobParameters of the - job, instead of through system properties, and access them that way. To - accomplish this, Spring Batch allows for the late binding of various Job - and Step attributes: + Often in a batch setting it is preferable to parameterize the file + name in the JobParameters of the + job, instead of through system properties, and access them that way. To + allow for this, Spring Batch allows for the late binding of various Job + and Step attributes: - + <bean id="flatFileItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" value="#{jobParameters[input.file.name]}" /> @@ -1464,34 +1315,34 @@ - Both the JobExecution and - StepExecution level - ExecutionContext can be accessed in the same - way: - - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{jobExecutionContext[input.file.name]}" /> - </bean> - - - - - <bean id="flatFileItemReader" scope="step" - class="org.springframework.batch.item.file.FlatFileItemReader"> - <property name="resource" value="#{stepExecutionContext[input.file.name]}" /> - </bean> - - - -
- Step Scope - - All of the late binding examples from above have a scope of "step" - declared on the bean definition: + Both the JobExecution and + StepExecution level + ExecutionContext can be accessed in the same + way: + <bean id="flatFileItemReader" scope="step" + class="org.springframework.batch.item.file.FlatFileItemReader"> + <property name="resource" value="#{jobExecutionContext[input.file.name]}" /> + </bean> + + + + + <bean id="flatFileItemReader" scope="step" + class="org.springframework.batch.item.file.FlatFileItemReader"> + <property name="resource" value="#{stepExecutionContext[input.file.name]}" /> + </bean> + + + +
+ Step Scope + + All of the late binding examples from above have a scope of + "step" declared on the bean definition: + + <bean id="flatFileItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" value="#{jobParameters[input.file.name]}" /> @@ -1499,16 +1350,17 @@ - Using a scope of Step is required in order - to use late binding since the bean cannot actually be instantiated until - the Step starts, which allows the attributes to - be found. Because it is not part of the Spring container by default, it - must be added explicitly: + Using a scope of Step is required in + order to use late binding, since the bean cannot actually be + instantiated until the Step starts, which + allows the attributes to be found. Because it is not part of the + Spring container by default, it must be added explicitly: - + <bean class="org.springframework.batch.core.scope.StepScope" /> +
diff --git a/docs/src/site/docbook/reference/whatsnew.xml b/docs/src/site/docbook/reference/whatsnew.xml index 95ac171c6..f40fdcee1 100644 --- a/docs/src/site/docbook/reference/whatsnew.xml +++ b/docs/src/site/docbook/reference/whatsnew.xml @@ -35,13 +35,13 @@
Java 5 - The 1.x releases of Spring Batch were all based on Java 1.4. This + The 1.x release of Spring Batch were all based on Java 1.4. This prevented the framework from using many enhancements provided in Java 5 such as generics, parameterized types, etc. The entire framework has been - updated to utilize these features. As a result, Java - 1.4 is no longer supported. Most of the interfaces developers - work with have been updated to support generic types. As an example, the - ItemReader interface from 1.1 is below: + updated to utilize these features. Java 1.4 is no + longer supported. Most of the interfaces developers work with + have been updated to support generic types. As an example, the ItemReader + interface from 1.1 is below: public interface ItemReader { @@ -56,7 +56,7 @@ As you can see, the read method returns an - Object. The 2.0 version is below: + Object. The 2.0 version is below: public interface ItemReader<T> { @@ -69,9 +69,9 @@ As you can see, ItemReader now supports the generic type, T, which is returned from read. You may also notice that - mark and reset have been - removed. This is due to step processing strategy changes, which are - discussed below. Many other interfaces have been similarly updated. + mark and reset have been removed. This is due to step processing strategy + changes, which are discussed below. Many other interfaces have been + similarly updated.
@@ -83,24 +83,22 @@ + fileref="images/item-oriented-processing.png" scale="90" + width="" /> - In item-oriented processing, the ItemReader - returns one Object (the 'item') which is then - handed to the ItemWriter, periodically committing + In item-oriented processing, the ItemReader returns one Object (the + 'item) which is then handed to the ItemWriter, periodically committing when the number of items hits the commit interval. For example, if the - commit interval is 5, ItemReader and - ItemWriter will each be called 5 times. This is - illustrated in a simplified code example below: + commit interval is 5, ItemReader and ItemWriter will each be called 5 + times. This is illustrated in a simplified code example below: for(int i = 0; i < commitInterval; i++){ @@ -110,9 +108,8 @@ - Both the ItemReader and - ItemWriter interfaces were completely geared toward - this approach: + Both the ItemReader and ItemWriter interfaces were completely geared + toward this approach: public interface ItemReader { @@ -138,18 +135,14 @@ Because the 'scope' of the processing was one item, supporting - rollback scenarios required additional methods, which is what - mark, reset, - flush, and clear - provided. If, after successfully reading and writing 2 items, the third - has an error while writing, the transaction would need to be rolled back. - In this case, the clear method on the writer - would be called, indicating that it should clear - its buffer, and reset would be called on the - ItemReader, indicating that it should return back - to the last position it was at when mark was - called. (Both mark and - flush are called on commit) + rollback scenarios required additional methods, which is what mark, reset, + flush, and clear provided. If, after successfully reading and writing 2 + items, the third had an error while writing, the transaction would need to + be rolled back. In this case, the clear method on the writer would be + called, indicating that it should clear its buffer, and reset would be + called on the ItemReader, indicating that it should return back to the + last position it was at when mark was called. (Both mark and flush are + called on commit) In 2.0, this strategy has been changed to a chunk-oriented approach: @@ -158,13 +151,13 @@ + scale="90" width="" /> + scale="90" width="60%" /> @@ -183,8 +176,7 @@ This approach not only allows for much simpler processing and - scalability approaches, it also makes the - ItemReader and ItemWriter + scalability approaches, it also makes the ItemReader and ItemWriter interfaces much cleaner: @@ -205,39 +197,35 @@ - As you can see, the interfaces no longer contain the - mark, reset, - flush, and clear - methods. This makes the creation of readers and writers much more - straightforward for developers. In the case of + As you can see, the interfaces no longer contain the mark, reset, + flush, and clear methods. This makes the creation of readers and writers + much more straightforward for developers. In the case of ItemReader, the interface is now forward-only. The - framework will buffer read items for developers in the case of rollback - (though there are exceptions if the underlying resource is transactional - see: ). - ItemWriter is also simplified, since it gets the - entire 'chunk' of items at once, rather than one at a time, it can decide - to flush any resources (such as a file or hibernate session) before - returning control to the Step. More detailed - information on chunk-oriented processing can be found in ) ItemWriter is also simplified, + since it gets the entire 'chunk' of items at once, rather than one at a + time, it can decide to flush any resources (such as a file or hibernate + session) before returning control to the Step. More + detailed information on chunk-oriented processing can be found in . Reader and writer implementation - information can be found in . + information can be found in 80
ItemProcessor - Previously, Steps had only two - dependencies, ItemReader and + Previously, Steps had only two dependencies, + ItemReader and ItemWriter: - - @@ -253,21 +241,18 @@ - + - This approach works. However, it requires an extra layer between + This approach works, however, it requires an extra layer between either the reader or the writer and the Step. Furthermore, the ItemWriter would need to be registered separately as an ItemStream with the - Step. For this reason, the - ItemTransfomer was renamed to - ItemProcessor and moved up to the same level as - ItemReader and - ItemWriter: + Step. For this reason, the ItemTransfomer was + renamed to ItemProcessor and moved up to the same level as ItemReader + and ItemWriter: @@ -277,8 +262,7 @@ + fileref="images/step.png" scale="60" width="" />
@@ -328,21 +312,20 @@ Meta Data access improvements The JobRepository interface represents basic - CRUD operations with Job meta-data. However, it may - also be useful to query the meta-data. For that reason, the + CRUD operations with Job meta-data. However, it can + be useful to query the meta-data. For that reason, the JobExplorer and JobOperator interfaces have been created: + scale="90" width="" /> - + @@ -362,13 +345,12 @@ + scale="" width="" /> - + @@ -377,13 +359,12 @@ + scale="80" width="40%" /> - + @@ -403,6 +384,6 @@ More details on how to configure non sequential steps can be found - in . + in