diff --git a/docs/.classpath b/docs/.classpath new file mode 100644 index 000000000..4c0d4b967 --- /dev/null +++ b/docs/.classpath @@ -0,0 +1,6 @@ + + + + + + diff --git a/docs/.project b/docs/.project index e0d2794ab..38811d980 100644 --- a/docs/.project +++ b/docs/.project @@ -1,3 +1,28 @@ +<<<<<<< .mine + + + docs + + + + + + org.eclipse.jdt.core.javabuilder + + + + + org.devzuz.q.maven.jdt.core.mavenIncrementalBuilder + + + + + + org.eclipse.jdt.core.javanature + org.devzuz.q.maven.jdt.core.mavenNature + + +======= docs @@ -27,3 +52,4 @@ org.codehaus.groovy.eclipse.groovyNature +>>>>>>> .r10743 diff --git a/docs/src/site/docbook/reference/Discard.xml b/docs/src/site/docbook/reference/Discard.xml new file mode 100644 index 000000000..4f66e8d3e --- /dev/null +++ b/docs/src/site/docbook/reference/Discard.xml @@ -0,0 +1,33 @@ + + + + Discard + +
+ Support Stereotypes + + While item readers and writers serve as the main entry points for + item-oriented processing, they might be supplemented by a number of + support classes that perform specific tasks within the reader / writer + lifecycle. These support stereotypes are useful for dividing the work of + item readers and writers into reusable pieces, as well as abstracting away + the details of processing, such as interaction with external systems. + Additionally, they give us another opportunity to leverage the powerful + configuration features of the Spring framework, as we can switch between + several beans implementing these support interfaces without changing the + driving item reader or writer. + +
+ Item Transformers + + An item transformer is a class that is capable of taking an object + and changing it somehow before processing occurs. For instance, an item + transformer my alter an object by changing its properties or by + replacing it with another object entirely, such as a wrapper or + derivative object. It can also be defined as an adaptor, allowing an + object of one type to be converted for use as an object of a second + type. +
+
+ \ No newline at end of file diff --git a/docs/src/site/docbook/reference/ExecutionEnvironment.png b/docs/src/site/docbook/reference/ExecutionEnvironment.png new file mode 100644 index 000000000..e574236b3 Binary files /dev/null and b/docs/src/site/docbook/reference/ExecutionEnvironment.png differ diff --git a/docs/src/site/docbook/reference/appendix.xml b/docs/src/site/docbook/reference/appendix.xml new file mode 100644 index 000000000..369a7d34d --- /dev/null +++ b/docs/src/site/docbook/reference/appendix.xml @@ -0,0 +1,12 @@ + + + + List of ItemReaders + +
+ + + +
+ \ No newline at end of file diff --git a/docs/src/site/docbook/reference/core.xml b/docs/src/site/docbook/reference/core.xml index f2630be07..9ba6f0999 100644 --- a/docs/src/site/docbook/reference/core.xml +++ b/docs/src/site/docbook/reference/core.xml @@ -9,8 +9,8 @@ To any experienced batch architect, the overall concepts of batch processing described above should be familiar and comfortable. There are - “Jobs” and “Steps” and a developer supplied processing unit that Spring - Batch refers to as the “Tasklet.” The following diagram is only a slight + “Jobs” and “Steps” and a developer supplied processing units called + ItemReaders and ItemWriters. The following diagram is only a slight variation of the batch reference architecture that has been used for decades. JCL and COBOL developers are likely to be as comfortable with the concepts as C++, C# and Java developers. However, because of the Spring @@ -113,7 +113,7 @@ In addition the components describe the batch interaction and services stereotypes that are the domain language and interfaces - implemented by developers in constructing a batch solution. As he diagram + implemented by developers in constructing a batch solution. As the diagram illustrates, custom applicaton archifacts, generally created by the developer, are the following: @@ -140,11 +140,11 @@ - Define how the batch jobs will be launched + Define how batch jobs will be launched - Define the Job Execution Environment + Job construction and Configuration @@ -190,21 +190,15 @@ configuration.
- - Job - - The job could be described as the heart of the Spring Batch framework. It is represented by a Spring bean that implements the Job interface and contains all of the information necessary to define the operations performed by a job. A job configuration is typically contained within a Spring XML configuration file and the job's name is determined by the "id" attribute associated - with the job configuration bean. The job configuration contains: - - + with the job configuration bean. The job configuration contains @@ -225,31 +219,13 @@ - A default simple implementation of the - - Job - - interface is provided by Spring Batch in the form of the JobSupport class. Jobs can be defined by creating beans from subclasses of - - JobSupport - - . The - - JobSupport - - class, however, does not provide much in the way of functionality. The provided - - SimpleJob - - class creates some standard functionality on top of - - JobSupport - - , namely a standard execution logic that all jobs should utilize. In general, all job configurations should be defined using a bean of type - - SimpleJob - - . + A default simple implementation of the Job interface is provided by Spring Batch in the + form of the SimpleJob class which creates some standard functionality on + top of Job, namely a standard execution + logic that all jobs should utilize. In general, all job configurations + should be defined using a bean of type SimpleJob.
@@ -261,12 +237,12 @@ time "foo" is supposed to run would be an instance of the "foo" job. Each instance would be uniquely identified as each one represents a distinct batch need. Further, each instance might have attempted several - times to complete its work. Each attempt is represented by a [#Job - Execution], described below. A job instance is not considered to be - complete until an associated job execution completes successfully. As - such, a single job instance may have many executions. To keep track of - this, every job instance provides a reference to the last execution - attempt. + times to complete its work. Each attempt is represented by a JobExecution, described below. A job instance is + not considered to be complete until an associated job execution + completes successfully. As such, a single job instance may have many + executions. To keep track of this, every job instance provides a + reference to the last execution attempt. For example, a unique instance might be identified by just a job name, or by the combination of a job name and a scheduled date. Using @@ -284,6 +260,13 @@ identifies this job instance.
+
+ Job Parameters + + Job Parameters represent parameters to a job that + +
+
Job Execution @@ -305,11 +288,11 @@ Job executions are represented by objects of the JobExecution class. These job executions are created by an implementation of the JobExecutorFacade interface from a given + role="bold">JobRepository interface from a given JobInstance corresponding to a unique JobParameters object. Each job - execution contains a reference to its corresponding job instance, - related Step Executions and step/chunk context data. + execution contains a reference to its corresponding job instance and + related Step Executions.
@@ -319,15 +302,20 @@ This section describes stereotypes relating to the concept of a batch step. A step is an entity that encapsulates a single, independent phase of a batch job. Therefore, every batch job is composed entirely of - one or more batch steps. + one or more batch steps. Steps should be thought of as unique processing streams that + will be executed in sequence. For example, if you have one step that loads a file into a database, + another that reads from the database, validates the data, preforms processing, and then + writes to another table, and another that reads from that table and writes out to a file. Each + of these steps will be performed completely before moving on to the next step. The file will + be completely read into the database before step 2 can begin.
- Step Configuration + Step - The step bean contains all of the information necessary to define - a discrete set of business logic within a job configuration. This is a + A batch step contains all of the information necessary to define + a discrete set of business logic within a job. This is a necessarily vague description because the contents of any given step - configuration are at the discretion of the developer writing jobs. A + are at the discretion of the developer writing jobs. A step can be as narrowly defined as a single line of code or as broadly defined as necessary to complete the entire work of your job. There are several factors that will affect the breadth of your step @@ -352,55 +340,17 @@ - Step configurations are defined by instantiating implementations - of the Step interface. Additionally, - the utility class StepSupport and its - abstract subclass, AbstractStep provide - a basic implementation of Step with - default functionality that should be common to any concrete Step - implementation. Generally, all step configuration implementations should - extend from these classes. - - Two step implementation classes are available in the Spring Batch + Steps are defined by instantiating implementations + of the Step interface. Two step implementation + classes are available in the Spring Batch framework, and they are each discussed in detail in other sections of this guide. For most situations, the SimpleStep implementation is sufficient, but + role="bold">ItemOrientedStep implementation is sufficient, but custom control flow behavior and transaction management behavior can also be configured by using a RepeatOperationsStep.
-
- Step Instance - - A step instance, represented by the StepInstance class, represents the business - concept of a single step within a job invocation. That is to say, every - job instance contains one or more step instances. - - For example, suppose we have a job instance called - "foo-01-01-2008" that is an instance of a job configuration containing - three steps. Suppose these steps are named "step1", "step2" and "step3." - There will be corresponding "foo-01-01-2008#step1", - "foo-01-01-2008#step2" and "foo-01-01-2008#step3" step instances, which - will be distinct from the step instances of any other job instance (e.g. - those of "foo-01-02-2008"). - - - - - The step instance naming here is for clarity, this is not necessarily how the instance will be named internally within the framework. - </note> - - <para>Each step instance will contain the current status of the batch - execution and a reference to its corresponding <emphasis - role="bold">JobInstance</emphasis>. Additionally, a step instance keeps - track of how many attempts are made to run the corresponding step. Each - attempt to run a step will create a <emphasis - role="bold">StepExecution</emphasis>, so a single job instance might - have several corresponding step executions.</para> - </section> - <section> <title id="s.2.1.3">Step Execution @@ -418,58 +368,33 @@ Step executions are represented by objects of the StepExecution class. These step executions are created by an implementation of the JobExecutor interface from a given StepInstance and Job interface from a given JobExecution. Each step execution contains a - reference to its corresponding step instance and job execution, and + reference to its corresponding step and job execution, and transaction related data such as commit and rollback count and start and end times. Additionally, each step execution will contain a set of execution attributes, which will contain statistics and restart data. - -
- -
- Chunk - - Each step execution is divided into one or more transactions. These - individual transactions are sometimes referred to as "chunks." Each - chunk may represent one or many individual operations within a step - execution, depending on configuration. Although chunk-based processing - is an important feature of Spring Batch, there is no domain object that - directly corresponds to a chunk, as chunks are implicitly created through - the opening and closing of transactions. - -
- -
- Step Contribution - - A step contribution represents the metadata for an uncommitted portion, or chunk, - of a step execution that is being "buffered" until a transaction boundary is reached - and the operations within the chunk are successfully committed. Each step contribution - is created by the enclosing step execution, and each contribution is later "applied" - by the enclosing step executor, allowing metadata to be applied only after each processed - chunk succeeds. -
Tasklets A tasklet represents the execution of a logical unit of work, as - defined by its implementation of the Spring Batch provided _Tasklet_ + defined by its implementation of the Spring Batch provided Tasklet interface. Tasklets are used when defining step configurations to specify the work done by the step. Subsequently, the logic in a tasklet is atomic in terms of transactions. A transaction will never commit until an entire tasklet execution is complete (unless an exception occurs - a transaction might either commit or rollback if that behavior - is specified in the step's exception management strategy). + is specified in the step's exception management strategy). There is a + specific implementation of the Step interface, TaskletStep, that works + directly with a Tasklet.
- Item-Oriented Processing Stereotypes + Processing Stereotypes A powerful batch processing paradigm implemented by the Spring Batch framework is the concept of item-oriented processing. That is, doing work @@ -478,66 +403,39 @@ processing might entail, such as file or database operations.
- Reader and Writer Stereotypes - - There are two basic stereotypes that represent the first-class - participants in item-oriented processing, item providers and item - processors. They are each represented by a simple interface provided by - the Spring Batch framework, which allows free reign over their - implementations and improves our ability to leverage the Spring - framework's dependency injection capabilities. + Reader and Writer Stereotypes -
- Item Readers + There are two basic stereotypes that represent the first-class + participants in item-oriented processing, item providers and item + processors. They are each represented by a simple interface provided by + the Spring Batch framework, which allows free reign over their + implementations and improves our ability to leverage the Spring + framework's dependency injection capabilities. - An item reader is an object that is used to retrieve the inputs - for a step, one at a time. When the item reader has exhausted the items - it can provide, it will indicate this in a meaningful way (generally by - returning null). When coupled with an item processor, this forms a - complete item-oriented process, as each item taken from the provider is - then processed by the writer. -
+
+ Item Reader -
- Item Writers/Processors + An item reader is an object that is used to retrieve the inputs + for a step, one at a time. When the item reader has exhausted the + items it can provide, it will indicate this in a meaningful way + (generally by returning null). When coupled with an item processor, + this forms a complete item-oriented process, as each item taken from + the provider is then processed by the writer. +
- An item writer is an object that is used to perform processing - for a step, one item at a time. Generally, an item writer has no - knowledge of the input it will receive next, only the item that that was - passed in its current invocation. As a result, item writers will - generally make no assumptions about the input they receive an treat - every item the same way and keep track of its own state between - invocations. When coupled with an item provider, this forms a complete - item-oriented process, as each item taken from the provider is then - processed by the processor. -
-
-
+
+ Item Writer -
- Support Stereotypes - - While item readers and writers serve as the main entry points - for item-oriented processing, they might be supplemented by a number of support - classes that perform specific tasks within the reader / writer - lifecycle. These support stereotypes are useful for dividing the work of - item readers and writers into reusable pieces, as well as abstracting - away the details of processing, such as interaction with external systems. - Additionally, they give us another opportunity to leverage the powerful - configuration features of the Spring framework, as we can switch between - several beans implementing these support interfaces without changing the - driving item reader or writer. - -
- Item Transformers - - An item transformer is a class that is capable of taking an object - and changing it somehow before processing occurs. For instance, an item - transformer my alter an object by changing its properties or by - replacing it with another object entirely, such as a wrapper or - derivative object. It can also be defined as an adaptor, allowing an - object of one type to be converted for use as an object of a second - type. + An item writer is an object that is used to perform processing + for a step, one item at a time. Generally, an item writer has no + knowledge of the input it will receive next, only the item that that + was passed in its current invocation. As a result, item writers will + generally make no assumptions about the input they receive an treat + every item the same way and keep track of its own state between + invocations. When coupled with an item provider, this forms a complete + item-oriented process, as each item taken from the provider is then + processed by the processor. +
@@ -640,4 +538,4 @@ application termination and wrap-up -- provided there are no additional Steps to execute. - + \ No newline at end of file diff --git a/docs/src/site/docbook/reference/glossary-test.xml b/docs/src/site/docbook/reference/glossary-test.xml new file mode 100644 index 000000000..a0d8fed82 --- /dev/null +++ b/docs/src/site/docbook/reference/glossary-test.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/src/site/docbook/reference/index.xml b/docs/src/site/docbook/reference/index.xml index 578076866..e9e2d1096 100644 --- a/docs/src/site/docbook/reference/index.xml +++ b/docs/src/site/docbook/reference/index.xml @@ -30,8 +30,8 @@ - + diff --git a/docs/src/site/docbook/reference/spring-batch-intro.xml b/docs/src/site/docbook/reference/spring-batch-intro.xml index 328ed883e..d52f6db43 100644 --- a/docs/src/site/docbook/reference/spring-batch-intro.xml +++ b/docs/src/site/docbook/reference/spring-batch-intro.xml @@ -27,7 +27,11 @@ productivity, POJO-based development approach, and general ease of use capabilities people have come to know from the Spring Framework, while making it easy for developers to access and leverage more advance - enterprise services when necessary. + enterprise services when necessary. Spring Batch is not a scheduling + framework. There are many good enterprise schedulers available in both the + commerical and open source spaces such as Quartz, Tivoli, Control-M, etc. + It is intended to work in conjunction with a scheduler, not replace a + scheduler. Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, @@ -38,10 +42,6 @@ complex, high-volume batch jobs can leverage the framework in a highly scalable manner to process significant volumes of information. - Spring Batch is part of the Spring - Portfolio. -
Spring Batch Architecture @@ -116,42 +116,6 @@
-
- Roadmap - - Once the framework is released it can be used immediately to - simplify batch optimisations and automatic retries. The framework is - oriented around application developers not needing to know any details - of the framework - there are a few application developer interfaces that - can be used for convenient construction of data processing pipelines, - but apart from that we support as close to a POJO programming model as - is practical. This is similar to the approach taken in Spring Core in - the area of DAO implementation. - - A Partitioned Batch Execution Environment is also being developed - that will provide an alternate scaling solution. This execution - environment will provide more advance technical services and features to - enable extremely high-volume and high performance batch jobs though - proven optimization and partitioning techniques. Proven scaling - techniques will be provided as partitioned strategies allowing users to - spread the load across a pool of clustered J2EE application servers. - There are also discussions to leverage grid technologies as an alternate - scaling solution. - - Matt Welsh's work shows that SEDA has - enormous benefits over more rigid processing architectures, and - messaging environments provde a lot of resilience out of the box. So we - also want to provide a more SEDA flavoured execution environment, as - well as supporting the more traditional ETL style approach. There might - be a tie in with Mule and/or other ESB tools here, giving the benefit of - a very scalable architecture, where the choice of transport and - distribution strategy can be made as late as possible. The same - application code could be used in principle for a standalone tool - processing a small amount of data, and a massive enterprise-scale - bulk-processing engine. -
-
Background @@ -165,9 +129,9 @@ in-house solutions developed within client enterprise IT functions. - Interface21 and Accenture are collaborating to change this. + SpringSource and Accenture have collaborated to change this. Accenture's hands-on industry and technical experience in implementing - batch architectures, Interface21's depth of technical experience, and + batch architectures, SpringSource's depth of technical experience, and Spring's proven programming model together mark a natural and powerful partnership to create high-quality, market relevant software aimed at filling an important gap in enterprise Java. Both companies are also @@ -175,18 +139,18 @@ developing Spring-based batch architecture solutions. This has provided some useful additional detail and real-life constraints helping to ensure the solution can be applied to the real-world problems posed by - clients. For these reasons and many more, Interface21 and Accenture have - teamed to collaborate on the development of Spring Batch. + clients. For these reasons and many more, SpringSource and Accenture + have teamed to collaborate on the development of Spring Batch. - Accenture is contributing previously proprietary batch processing + Accenture has contributed previously proprietary batch processing architecture frameworks -- based upon decades worth of experience in building batch architectures with the last several generations of platforms (i.e., COBOL/Mainframe, C++/Unix, and now Java/anywhere) -- to the Spring Batch project along with committer resources to drive support, enhancements, and the future roadmap. - The collaborative effort between Accenture and Interface21 aims to - promote the standardization of software processing approaches, + The collaborative effort between Accenture and SpringSource aims + to promote the standardization of software processing approaches, frameworks, and tools that can be consistently leveraged by enterprise users when creating batch applications. Companies and government agencies desiring to deliver standard, proven solutions to their