Migrate documentation to Antora

Issue #4422
This commit is contained in:
Rob Winch
2023-07-20 17:07:16 -05:00
committed by Mahmoud Ben Hassine
parent e36a44788d
commit 2e8d5063f7
149 changed files with 9472 additions and 9114 deletions

View File

@@ -0,0 +1,62 @@
[[commitInterval]]
= The Commit Interval
As mentioned previously, a step reads in and writes out items, periodically committing
by using the supplied `PlatformTransactionManager`. With a `commit-interval` of 1, it
commits after writing each individual item. This is less than ideal in many situations,
since beginning and committing a transaction is expensive. Ideally, it is preferable to
process as many items as possible in each transaction, which is completely dependent upon
the type of data being processed and the resources with which the step is interacting.
For this reason, you can configure the number of items that are processed within a commit.
[tabs]
====
Java::
+
The following example shows a `step` whose `tasklet` has a `commit-interval`
value of 10 as it would be defined in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Job sampleJob(JobRepository jobRepository) {
return new JobBuilder("sampleJob", jobRepository)
.start(step1())
.build();
}
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.build();
}
----
XML::
+
The following example shows a `step` whose `tasklet` has a `commit-interval`
value of 10 as it would be defined in XML:
+
.XML Configuration
[source, xml]
----
<job id="sampleJob">
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" commit-interval="10"/>
</tasklet>
</step>
</job>
----
====
In the preceding example, 10 items are processed within each transaction. At the
beginning of processing, a transaction is begun. Also, each time `read` is called on the
`ItemReader`, a counter is incremented. When it reaches 10, the list of aggregated items
is passed to the `ItemWriter`, and the transaction is committed.

View File

@@ -0,0 +1,142 @@
[[configuringSkip]]
= Configuring Skip Logic
There are many scenarios where errors encountered while processing should not result in
`Step` failure but should be skipped instead. This is usually a decision that must be
made by someone who understands the data itself and what meaning it has. Financial data,
for example, may not be skippable because it results in money being transferred, which
needs to be completely accurate. Loading a list of vendors, on the other hand, might
allow for skips. If a vendor is not loaded because it was formatted incorrectly or was
missing necessary information, there probably are not issues. Usually, these bad
records are logged as well, which is covered later when discussing listeners.
[tabs]
====
Java::
+
The following Java example shows an example of using a skip limit:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(flatFileItemReader())
.writer(itemWriter())
.faultTolerant()
.skipLimit(10)
.skip(FlatFileParseException.class)
.build();
}
----
XML::
+
The following XML example shows an example of using a skip limit:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="flatFileItemReader" writer="itemWriter"
commit-interval="10" skip-limit="10">
<skippable-exception-classes>
<include class="org.springframework.batch.item.file.FlatFileParseException"/>
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
----
====
In the preceding example, a `FlatFileItemReader` is used. If, at any point, a
`FlatFileParseException` is thrown, the item is skipped and counted against the total
skip limit of 10. Exceptions (and their subclasses) that are declared might be thrown
during any phase of the chunk processing (read, process, or write). Separate counts
are made of skips on read, process, and write inside
the step execution, but the limit applies across all skips. Once the skip limit is
reached, the next exception found causes the step to fail. In other words, the eleventh
skip triggers the exception, not the tenth.
One problem with the preceding example is that any other exception besides a
`FlatFileParseException` causes the `Job` to fail. In certain scenarios, this may be the
correct behavior. However, in other scenarios, it may be easier to identify which
exceptions should cause failure and skip everything else.
[tabs]
====
Java::
+
The following Java example shows an example excluding a particular exception:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(flatFileItemReader())
.writer(itemWriter())
.faultTolerant()
.skipLimit(10)
.skip(Exception.class)
.noSkip(FileNotFoundException.class)
.build();
}
----
XML::
+
The following XML example shows an example excluding a particular exception:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="flatFileItemReader" writer="itemWriter"
commit-interval="10" skip-limit="10">
<skippable-exception-classes>
<include class="java.lang.Exception"/>
<exclude class="java.io.FileNotFoundException"/>
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
----
====
By identifying `java.lang.Exception` as a skippable exception class, the configuration
indicates that all `Exceptions` are skippable. However, by "`excluding`"
`java.io.FileNotFoundException`, the configuration refines the list of skippable
exception classes to be all `Exceptions` __except__ `FileNotFoundException`. Any excluded
exception class is fatal if encountered (that is, they are not skipped).
For any exception encountered, the skippability is determined by the nearest superclass
in the class hierarchy. Any unclassified exception is treated as 'fatal'.
[tabs]
====
Java::
+
The order of the `skip` and `noSkip` method calls does not matter.
XML::
+
The order of the `<include/>` and `<exclude/>` elements does not matter.
====

View File

@@ -0,0 +1,94 @@
[[configuringAStep]]
= Configuring a Step
Despite the relatively short list of required dependencies for a `Step`, it is an
extremely complex class that can potentially contain many collaborators.
[tabs]
====
Java::
+
When using Java configuration, you can use the Spring Batch builders, as the
following example shows:
+
.Java Configuration
[source, java]
----
/**
* Note the JobRepository is typically autowired in and not needed to be explicitly
* configured
*/
@Bean
public Job sampleJob(JobRepository jobRepository, Step sampleStep) {
return new JobBuilder("sampleJob", jobRepository)
.start(sampleStep)
.build();
}
/**
* Note the TransactionManager is typically autowired in and not needed to be explicitly
* configured
*/
@Bean
public Step sampleStep(JobRepository jobRepository, // <2>
PlatformTransactionManager transactionManager) { // <1>
return new StepBuilder("sampleStep", jobRepository)
.<String, String>chunk(10, transactionManager) // <3>
.reader(itemReader())
.writer(itemWriter())
.build();
}
----
<1> `transactionManager`: Spring's `PlatformTransactionManager` that begins and commits
transactions during processing.
<2> `repository`: The Java-specific name of the `JobRepository` that periodically stores
the `StepExecution` and `ExecutionContext` during processing (just before committing).
<3> `chunk`: The Java-specific name of the dependency that indicates that this is an
item-based step and the number of items to be processed before the transaction is
committed.
+
NOTE: Note that `repository` defaults to `jobRepository` (provided through `@EnableBatchProcessing`)
and `transactionManager` defaults to `transactionManager` (provided from the application context).
Also, the `ItemProcessor` is optional, since the item could be
directly passed from the reader to the writer.
XML::
+
To ease configuration, you can use the Spring Batch XML namespace, as
the following example shows:
+
.XML Configuration
[source, xml]
----
<job id="sampleJob" job-repository="jobRepository"> <!--2-->
<step id="step1">
<tasklet transaction-manager="transactionManager"> <!--1-->
<chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> <!--3-->
</tasklet>
</step>
</job>
----
<1> `transaction-manager`: Spring's `PlatformTransactionManager` that begins and commits
transactions during processing.
<2> `job-repository`: The XML-specific name of the `JobRepository` that periodically stores
the `StepExecution` and `ExecutionContext` during processing (just before committing). For
an in-line `<step/>` (one defined within a `<job/>`), it is an attribute on the `<job/>`
element. For a standalone `<step/>`, it is defined as an attribute of the `<tasklet/>`.
<3> `commit-interval`: The XML-specific name of the number of items to be processed
before the transaction is committed.
+
NOTE: Note that `job-repository` defaults to `jobRepository` and
`transaction-manager` defaults to `transactionManager`. Also, the `ItemProcessor` is
optional, since the item could be directly passed from the reader to the writer.
====
The preceding configuration includes the only required dependencies to create a item-oriented
step:
* `reader`: The `ItemReader` that provides items for processing.
* `writer`: The `ItemWriter` that processes the items provided by the `ItemReader`.

View File

@@ -0,0 +1,103 @@
[[controllingRollback]]
= Controlling Rollback
By default, regardless of retry or skip, any exceptions thrown from the `ItemWriter`
cause the transaction controlled by the `Step` to rollback. If skip is configured as
described earlier, exceptions thrown from the `ItemReader` do not cause a rollback.
However, there are many scenarios in which exceptions thrown from the `ItemWriter` should
not cause a rollback, because no action has taken place to invalidate the transaction.
For this reason, you can configure the `Step` with a list of exceptions that should not
cause rollback.
[tabs]
====
Java::
+
In Java, you can control rollback as follows:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(2, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.faultTolerant()
.noRollback(ValidationException.class)
.build();
}
----
XML::
+
In XML, you can control rollback as follows:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" commit-interval="2"/>
<no-rollback-exception-classes>
<include class="org.springframework.batch.item.validator.ValidationException"/>
</no-rollback-exception-classes>
</tasklet>
</step>
----
====
[[transactionalReaders]]
== Transactional Readers
The basic contract of the `ItemReader` is that it is forward-only. The step buffers
reader input so that, in case of a rollback, the items do not need to be re-read
from the reader. However, there are certain scenarios in which the reader is built on
top of a transactional resource, such as a JMS queue. In this case, since the queue is
tied to the transaction that is rolled back, the messages that have been pulled from the
queue are put back on. For this reason, you can configure the step to not buffer the
items.
[tabs]
====
Java::
+
The following example shows how to create a reader that does not buffer items in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(2, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.readerIsTransactionalQueue()
.build();
}
----
XML::
+
The following example shows how to create a reader that does not buffer items in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" commit-interval="2"
is-reader-transactional-queue="true"/>
</tasklet>
</step>
----
====

View File

@@ -0,0 +1,108 @@
[[inheriting-from-a-parent-step]]
= Inheriting from a Parent `Step`
[role="xmlContent"]
If a group of `Steps` share similar configurations, then it may be helpful to define a
"`parent`" `Step` from which the concrete `Steps` may inherit properties. Similar to class
inheritance in Java, the "`child`" `Step` combines its elements and attributes with the
parent's. The child also overrides any of the parent's `Steps`.
[role="xmlContent"]
In the following example, the `Step`, `concreteStep1`, inherits from `parentStep`. It is
instantiated with `itemReader`, `itemProcessor`, `itemWriter`, `startLimit=5`, and
`allowStartIfComplete=true`. Additionally, the `commitInterval` is `5`, since it is
overridden by the `concreteStep1` `Step`, as the following example shows:
[source, xml, role="xmlContent"]
----
<step id="parentStep">
<tasklet allow-start-if-complete="true">
<chunk reader="itemReader" writer="itemWriter" commit-interval="10"/>
</tasklet>
</step>
<step id="concreteStep1" parent="parentStep">
<tasklet start-limit="5">
<chunk processor="itemProcessor" commit-interval="5"/>
</tasklet>
</step>
----
[role="xmlContent"]
The `id` attribute is still required on the step within the job element. This is for two
reasons:
* The `id` is used as the step name when persisting the `StepExecution`. If the same
standalone step is referenced in more than one step in the job, an error occurs.
[role="xmlContent"]
* When creating job flows, as described xref:step/controlling-flow.adoc[later in this chapter], the `next` attribute
should refer to the step in the flow, not the standalone step.
[[abstractStep]]
[role="xmlContent"]
[[abstract-step]]
== Abstract `Step`
[role="xmlContent"]
Sometimes, it may be necessary to define a parent `Step` that is not a complete `Step`
configuration. If, for instance, the `reader`, `writer`, and `tasklet` attributes are
left off of a `Step` configuration, then initialization fails. If a parent must be
defined without one or more of these properties, the `abstract` attribute should be used. An
`abstract` `Step` is only extended, never instantiated.
[role="xmlContent"]
In the following example, the `Step` (`abstractParentStep`) would not be instantiated if it
were not declared to be abstract. The `Step`, (`concreteStep2`) has `itemReader`,
`itemWriter`, and `commit-interval=10`.
[source, xml, role="xmlContent"]
----
<step id="abstractParentStep" abstract="true">
<tasklet>
<chunk commit-interval="10"/>
</tasklet>
</step>
<step id="concreteStep2" parent="abstractParentStep">
<tasklet>
<chunk reader="itemReader" writer="itemWriter"/>
</tasklet>
</step>
----
[[mergingListsOnStep]]
[role="xmlContent"]
[[merging-lists]]
== Merging Lists
[role="xmlContent"]
Some of the configurable elements on `Steps` are lists, such as the `<listeners/>` element.
If both the parent and child `Steps` declare a `<listeners/>` element, the
child's list overrides the parent's. To allow a child to add additional
listeners to the list defined by the parent, every list element has a `merge` attribute.
If the element specifies that `merge="true"`, then the child's list is combined with the
parent's instead of overriding it.
[role="xmlContent"]
In the following example, the `Step` "concreteStep3", is created with two listeners:
`listenerOne` and `listenerTwo`:
[source, xml, role="xmlContent"]
----
<step id="listenersParentStep" abstract="true">
<listeners>
<listener ref="listenerOne"/>
<listeners>
</step>
<step id="concreteStep3" parent="listenersParentStep">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" commit-interval="5"/>
</tasklet>
<listeners merge="true">
<listener ref="listenerTwo"/>
<listeners>
</step>
----

View File

@@ -0,0 +1,266 @@
[[interceptingStepExecution]]
= Intercepting `Step` Execution
Just as with the `Job`, there are many events during the execution of a `Step` where a
user may need to perform some functionality. For example, to write out to a flat
file that requires a footer, the `ItemWriter` needs to be notified when the `Step` has
been completed so that the footer can be written. This can be accomplished with one of many
`Step` scoped listeners.
You can apply any class that implements one of the extensions of `StepListener` (but not that interface
itself, since it is empty) to a step through the `listeners` element.
The `listeners` element is valid inside a step, tasklet, or chunk declaration. We
recommend that you declare the listeners at the level at which its function applies
or, if it is multi-featured (such as `StepExecutionListener` and `ItemReadListener`),
declare it at the most granular level where it applies.
[tabs]
====
Java::
+
The following example shows a listener applied at the chunk level in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(reader())
.writer(writer())
.listener(chunkListener())
.build();
}
----
XML::
+
The following example shows a listener applied at the chunk level in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="reader" writer="writer" commit-interval="10"/>
<listeners>
<listener ref="chunkListener"/>
</listeners>
</tasklet>
</step>
----
====
An `ItemReader`, `ItemWriter`, or `ItemProcessor` that itself implements one of the
`StepListener` interfaces is registered automatically with the `Step` if using the
namespace `<step>` element or one of the `*StepFactoryBean` factories. This only
applies to components directly injected into the `Step`. If the listener is nested inside
another component, you need to explicitly register it (as described previously under
xref:step/chunk-oriented-processing/registering-item-streams.adoc[Registering `ItemStream` with a `Step`]).
In addition to the `StepListener` interfaces, annotations are provided to address the
same concerns. Plain old Java objects can have methods with these annotations that are
then converted into the corresponding `StepListener` type. It is also common to annotate
custom implementations of chunk components, such as `ItemReader` or `ItemWriter` or
`Tasklet`. The annotations are analyzed by the XML parser for the `<listener/>` elements
as well as registered with the `listener` methods in the builders, so all you need to do
is use the XML namespace or builders to register the listeners with a step.
[[stepExecutionListener]]
== `StepExecutionListener`
`StepExecutionListener` represents the most generic listener for `Step` execution. It
allows for notification before a `Step` is started and after it ends, whether it ended
normally or failed, as the following example shows:
[source, java]
----
public interface StepExecutionListener extends StepListener {
void beforeStep(StepExecution stepExecution);
ExitStatus afterStep(StepExecution stepExecution);
}
----
`ExitStatus` has a return type of `afterStep`, to give listeners the chance to
modify the exit code that is returned upon completion of a `Step`.
The annotations corresponding to this interface are:
* `@BeforeStep`
* `@AfterStep`
[[chunkListener]]
== `ChunkListener`
A "`chunk`" is defined as the items processed within the scope of a transaction. Committing a
transaction, at each commit interval, commits a chunk. You can use a `ChunkListener` to
perform logic before a chunk begins processing or after a chunk has completed
successfully, as the following interface definition shows:
[source, java]
----
public interface ChunkListener extends StepListener {
void beforeChunk(ChunkContext context);
void afterChunk(ChunkContext context);
void afterChunkError(ChunkContext context);
}
----
The beforeChunk method is called after the transaction is started but before reading begins
on the `ItemReader`. Conversely, `afterChunk` is called after the chunk has been
committed (or not at all if there is a rollback).
The annotations corresponding to this interface are:
* `@BeforeChunk`
* `@AfterChunk`
* `@AfterChunkError`
You can apply a `ChunkListener` when there is no chunk declaration. The `TaskletStep` is
responsible for calling the `ChunkListener`, so it applies to a non-item-oriented tasklet
as well (it is called before and after the tasklet).
[[itemReadListener]]
== `ItemReadListener`
When discussing skip logic previously, it was mentioned that it may be beneficial to log
the skipped records so that they can be dealt with later. In the case of read errors,
this can be done with an `ItemReaderListener`, as the following interface
definition shows:
[source, java]
----
public interface ItemReadListener<T> extends StepListener {
void beforeRead();
void afterRead(T item);
void onReadError(Exception ex);
}
----
The `beforeRead` method is called before each call to read on the `ItemReader`. The
`afterRead` method is called after each successful call to read and is passed the item
that was read. If there was an error while reading, the `onReadError` method is called.
The exception encountered is provided so that it can be logged.
The annotations corresponding to this interface are:
* `@BeforeRead`
* `@AfterRead`
* `@OnReadError`
[[itemProcessListener]]
== `ItemProcessListener`
As with the `ItemReadListener`, the processing of an item can be "`listened`" to, as
the following interface definition shows:
[source, java]
----
public interface ItemProcessListener<T, S> extends StepListener {
void beforeProcess(T item);
void afterProcess(T item, S result);
void onProcessError(T item, Exception e);
}
----
The `beforeProcess` method is called before `process` on the `ItemProcessor` and is
handed the item that is to be processed. The `afterProcess` method is called after the
item has been successfully processed. If there was an error while processing, the
`onProcessError` method is called. The exception encountered and the item that was
attempted to be processed are provided, so that they can be logged.
The annotations corresponding to this interface are:
* `@BeforeProcess`
* `@AfterProcess`
* `@OnProcessError`
[[itemWriteListener]]
== `ItemWriteListener`
You can "`listen`" to the writing of an item with the `ItemWriteListener`, as the
following interface definition shows:
[source, java]
----
public interface ItemWriteListener<S> extends StepListener {
void beforeWrite(List<? extends S> items);
void afterWrite(List<? extends S> items);
void onWriteError(Exception exception, List<? extends S> items);
}
----
The `beforeWrite` method is called before `write` on the `ItemWriter` and is handed the
list of items that is written. The `afterWrite` method is called after the item has been
successfully written. If there was an error while writing, the `onWriteError` method is
called. The exception encountered and the item that was attempted to be written are
provided, so that they can be logged.
The annotations corresponding to this interface are:
* `@BeforeWrite`
* `@AfterWrite`
* `@OnWriteError`
[[skipListener]]
== `SkipListener`
`ItemReadListener`, `ItemProcessListener`, and `ItemWriteListener` all provide mechanisms
for being notified of errors, but none informs you that a record has actually been
skipped. `onWriteError`, for example, is called even if an item is retried and
successful. For this reason, there is a separate interface for tracking skipped items, as
the following interface definition shows:
[source, java]
----
public interface SkipListener<T,S> extends StepListener {
void onSkipInRead(Throwable t);
void onSkipInProcess(T item, Throwable t);
void onSkipInWrite(S item, Throwable t);
}
----
`onSkipInRead` is called whenever an item is skipped while reading. It should be noted
that rollbacks may cause the same item to be registered as skipped more than once.
`onSkipInWrite` is called when an item is skipped while writing. Because the item has
been read successfully (and not skipped), it is also provided the item itself as an
argument.
The annotations corresponding to this interface are:
* `@OnSkipInRead`
* `@OnSkipInWrite`
* `@OnSkipInProcess`
[[skipListenersAndTransactions]]
=== SkipListeners and Transactions
One of the most common use cases for a `SkipListener` is to log out a skipped item, so
that another batch process or even human process can be used to evaluate and fix the
issue that leads to the skip. Because there are many cases in which the original transaction
may be rolled back, Spring Batch makes two guarantees:
* The appropriate skip method (depending on when the error happened) is called only once
per item.
* The `SkipListener` is always called just before the transaction is committed. This is
to ensure that any transactional resources call by the listener are not rolled back by a
failure within the `ItemWriter`.

View File

@@ -0,0 +1,92 @@
[[registeringItemStreams]]
= Registering `ItemStream` with a `Step`
The step has to take care of `ItemStream` callbacks at the necessary points in its
lifecycle. (For more information on the `ItemStream` interface, see
xref:readers-and-writers/item-stream.adoc[ItemStream]). This is vital if a step fails and might
need to be restarted, because the `ItemStream` interface is where the step gets the
information it needs about persistent state between executions.
If the `ItemReader`, `ItemProcessor`, or `ItemWriter` itself implements the `ItemStream`
interface, these are registered automatically. Any other streams need to be
registered separately. This is often the case where indirect dependencies, such as
delegates, are injected into the reader and writer. You can register a stream on the
`step` through the `stream` element.
[tabs]
====
Java::
+
The following example shows how to register a `stream` on a `step` in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(2, transactionManager)
.reader(itemReader())
.writer(compositeItemWriter())
.stream(fileItemWriter1())
.stream(fileItemWriter2())
.build();
}
/**
* In Spring Batch 4, the CompositeItemWriter implements ItemStream so this isn't
* necessary, but used for an example.
*/
@Bean
public CompositeItemWriter compositeItemWriter() {
List<ItemWriter> writers = new ArrayList<>(2);
writers.add(fileItemWriter1());
writers.add(fileItemWriter2());
CompositeItemWriter itemWriter = new CompositeItemWriter();
itemWriter.setDelegates(writers);
return itemWriter;
}
----
XML::
+
The following example shows how to register a `stream` on a `step` in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="compositeWriter" commit-interval="2">
<streams>
<stream ref="fileItemWriter1"/>
<stream ref="fileItemWriter2"/>
</streams>
</chunk>
</tasklet>
</step>
<beans:bean id="compositeWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<beans:property name="delegates">
<beans:list>
<beans:ref bean="fileItemWriter1" />
<beans:ref bean="fileItemWriter2" />
</beans:list>
</beans:property>
</beans:bean>
----
====
In the preceding example, the `CompositeItemWriter` is not an `ItemStream`, but both of its
delegates are. Therefore, both delegate writers must be explicitly registered as streams
for the framework to handle them correctly. The `ItemReader` does not need to be
explicitly registered as a stream because it is a direct property of the `Step`. The step
is now restartable, and the state of the reader and writer is correctly persisted in the
event of a failure.

View File

@@ -0,0 +1,247 @@
[[stepRestart]]
= Configuring a `Step` for Restart
In the "`xref:job.adoc[Configuring and Running a Job]`" section , restarting a
`Job` was discussed. Restart has numerous impacts on steps, and, consequently, may
require some specific configuration.
[[startLimit]]
== Setting a Start Limit
There are many scenarios where you may want to control the number of times a `Step` can
be started. For example, you might need to configure a particular `Step` might so that it
runs only once because it invalidates some resource that must be fixed manually before it can
be run again. This is configurable on the step level, since different steps may have
different requirements. A `Step` that can be executed only once can exist as part of the
same `Job` as a `Step` that can be run infinitely.
[tabs]
====
Java::
+
The following code fragment shows an example of a start limit configuration in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.startLimit(1)
.build();
}
----
XML::
+
The following code fragment shows an example of a start limit configuration in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet start-limit="1">
<chunk reader="itemReader" writer="itemWriter" commit-interval="10"/>
</tasklet>
</step>
----
====
The step shown in the preceding example can be run only once. Attempting to run it again
causes a `StartLimitExceededException` to be thrown. Note that the default value for the
start-limit is `Integer.MAX_VALUE`.
[[allowStartIfComplete]]
== Restarting a Completed `Step`
In the case of a restartable job, there may be one or more steps that should always be
run, regardless of whether or not they were successful the first time. An example might
be a validation step or a `Step` that cleans up resources before processing. During
normal processing of a restarted job, any step with a status of `COMPLETED` (meaning it
has already been completed successfully), is skipped. Setting `allow-start-if-complete` to
`true` overrides this so that the step always runs.
[tabs]
====
Java::
+
The following code fragment shows how to define a restartable job in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.allowStartIfComplete(true)
.build();
}
----
XML::
+
The following code fragment shows how to define a restartable job in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet allow-start-if-complete="true">
<chunk reader="itemReader" writer="itemWriter" commit-interval="10"/>
</tasklet>
</step>
----
====
[[stepRestartExample]]
== `Step` Restart Configuration Example
[tabs]
====
Java::
+
The following Java example shows how to configure a job to have steps that can be
restarted:
+
.Java Configuration
[source, java]
----
@Bean
public Job footballJob(JobRepository jobRepository) {
return new JobBuilder("footballJob", jobRepository)
.start(playerLoad())
.next(gameLoad())
.next(playerSummarization())
.build();
}
@Bean
public Step playerLoad(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("playerLoad", jobRepository)
.<String, String>chunk(10, transactionManager)
.reader(playerFileItemReader())
.writer(playerWriter())
.build();
}
@Bean
public Step gameLoad(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("gameLoad", jobRepository)
.allowStartIfComplete(true)
.<String, String>chunk(10, transactionManager)
.reader(gameFileItemReader())
.writer(gameWriter())
.build();
}
@Bean
public Step playerSummarization(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("playerSummarization", jobRepository)
.startLimit(2)
.<String, String>chunk(10, transactionManager)
.reader(playerSummarizationSource())
.writer(summaryWriter())
.build();
}
----
XML::
+
The following XML example shows how to configure a job to have steps that can be
restarted:
+
.XML Configuration
[source, xml]
----
<job id="footballJob" restartable="true">
<step id="playerload" next="gameLoad">
<tasklet>
<chunk reader="playerFileItemReader" writer="playerWriter"
commit-interval="10" />
</tasklet>
</step>
<step id="gameLoad" next="playerSummarization">
<tasklet allow-start-if-complete="true">
<chunk reader="gameFileItemReader" writer="gameWriter"
commit-interval="10"/>
</tasklet>
</step>
<step id="playerSummarization">
<tasklet start-limit="2">
<chunk reader="playerSummarizationSource" writer="summaryWriter"
commit-interval="10"/>
</tasklet>
</step>
</job>
----
====
The preceding example configuration is for a job that loads in information about football
games and summarizes them. It contains three steps: `playerLoad`, `gameLoad`, and
`playerSummarization`. The `playerLoad` step loads player information from a flat file,
while the `gameLoad` step does the same for games. The final step,
`playerSummarization`, then summarizes the statistics for each player, based upon the
provided games. It is assumed that the file loaded by `playerLoad` must be loaded only
once but that `gameLoad` can load any games found within a particular directory,
deleting them after they have been successfully loaded into the database. As a result,
the `playerLoad` step contains no additional configuration. It can be started any number
of times is skipped if complete. The `gameLoad` step, however, needs to be run
every time in case extra files have been added since it last ran. It has
`allow-start-if-complete` set to `true` to always be started. (It is assumed
that the database table that games are loaded into has a process indicator on it, to ensure
new games can be properly found by the summarization step). The summarization step,
which is the most important in the job, is configured to have a start limit of 2. This
is useful because, if the step continually fails, a new exit code is returned to the
operators that control job execution, and it can not start again until manual
intervention has taken place.
NOTE: This job provides an example for this document and is not the same as the `footballJob`
found in the samples project.
The remainder of this section describes what happens for each of the three runs of the
`footballJob` example.
Run 1:
. `playerLoad` runs and completes successfully, adding 400 players to the `PLAYERS`
table.
. `gameLoad` runs and processes 11 files worth of game data, loading their contents
into the `GAMES` table.
. `playerSummarization` begins processing and fails after 5 minutes.
Run 2:
. `playerLoad` does not run, since it has already completed successfully, and
`allow-start-if-complete` is `false` (the default).
. `gameLoad` runs again and processes another 2 files, loading their contents into the
`GAMES` table as well (with a process indicator indicating they have yet to be
processed).
. `playerSummarization` begins processing of all remaining game data (filtering using the
process indicator) and fails again after 30 minutes.
Run 3:
. `playerLoad` does not run, since it has already completed successfully, and
`allow-start-if-complete` is `false` (the default).
. `gameLoad` runs again and processes another 2 files, loading their contents into the
`GAMES` table as well (with a process indicator indicating they have yet to be
processed).
. `playerSummarization` is not started and the job is immediately killed, since this is
the third execution of `playerSummarization`, and its limit is only 2. Either the limit
must be raised or the `Job` must be executed as a new `JobInstance`.

View File

@@ -0,0 +1,58 @@
[[retryLogic]]
= Configuring Retry Logic
In most cases, you want an exception to cause either a skip or a `Step` failure. However,
not all exceptions are deterministic. If a `FlatFileParseException` is encountered while
reading, it is always thrown for that record. Resetting the `ItemReader` does not help.
However, for other exceptions (such as a `DeadlockLoserDataAccessException`, which
indicates that the current process has attempted to update a record that another process
holds a lock on), waiting and trying again might result in success.
[tabs]
====
Java::
+
In Java, retry should be configured as follows:
+
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(2, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.faultTolerant()
.retryLimit(3)
.retry(DeadlockLoserDataAccessException.class)
.build();
}
----
XML::
+
In XML, retry should be configured as follows:
+
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter"
commit-interval="2" retry-limit="3">
<retryable-exception-classes>
<include class="org.springframework.dao.DeadlockLoserDataAccessException"/>
</retryable-exception-classes>
</chunk>
</tasklet>
</step>
----
====
The `Step` allows a limit for the number of times an individual item can be retried and a
list of exceptions that are "`retryable`". You can find more details on how retry works in
<<retry.adoc#retry, retry>>.

View File

@@ -0,0 +1,57 @@
[[transactionAttributes]]
= Transaction Attributes
You can use transaction attributes to control the `isolation`, `propagation`, and
`timeout` settings. You can find more information on setting transaction attributes in
the
https://docs.spring.io/spring/docs/current/spring-framework-reference/data-access.html#transaction[Spring
core documentation].
[tabs]
====
Java::
+
The following example sets the `isolation`, `propagation`, and `timeout` transaction
attributes in Java:
+
.Java Configuration
[source, java]
----
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
attribute.setPropagationBehavior(Propagation.REQUIRED.value());
attribute.setIsolationLevel(Isolation.DEFAULT.value());
attribute.setTimeout(30);
return new StepBuilder("step1", jobRepository)
.<String, String>chunk(2, transactionManager)
.reader(itemReader())
.writer(itemWriter())
.transactionAttribute(attribute)
.build();
}
----
XML::
+
The following example sets the `isolation`, `propagation`, and `timeout` transaction
attributes in XML:
+
.XML Configuration
[source, xml]
----
<step id="step1">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" commit-interval="2"/>
<transaction-attributes isolation="DEFAULT"
propagation="REQUIRED"
timeout="30"/>
</tasklet>
</step>
----
====