diff --git a/docs/src/site/docbook/reference/repeat.xml b/docs/src/site/docbook/reference/repeat.xml index def9a31e1..1a15c52da 100644 --- a/docs/src/site/docbook/reference/repeat.xml +++ b/docs/src/site/docbook/reference/repeat.xml @@ -5,7 +5,7 @@ Repeat
- Repeat Template + RepeatTemplate Batch processing is about repetitive actions - either as a simple optimisation, or as part of a job. To strategise and generalise the @@ -192,7 +192,7 @@ template.iterate(new RepeatCallback() { RepeatCallback, the RepeatTemplate consults an ExceptionHandler which can decide whether or not to - re-throw the exception. + re-throw the exception. public interface ExceptionHandler { @@ -247,7 +247,7 @@ template.iterate(new RepeatCallback() { close callbacks come before and after the entire iteration, and before, after and onError apply - to the individual RepeatCallback calls. + to the individual RepeatCallback calls. Note that when there is more than one listener, they are in a list, so there is an order. In this case open and @@ -270,4 +270,53 @@ template.iterate(new RepeatCallback() { iteration in the same thread (the same as a normal RepeatTemplate).
+ +
+ Declarative Iteration + + Sometimes there is some business processing that you know you want + to repeat every time it happens. The classic example of this is the + optimization of a message pipeline - it is more efficient to process a + batch of messages, if they are arriving frequently, than to bear the cost + of a separate transaction for every message. Spring Batch provides an AOP + interceptor that wraps a method call in a + RepeatOperations for just this purpose. The + RepeatOperationsInterceptor executes the + intercepted method and repeats according to the + CompetionPolicy in the provided + RepeatTemplate. + + Here is an example of declarative iteration using the Spring AOP + namespace to repeat a service call to a method called + processMessage (for more detail on how to + configure AOP interceptors see the Spring User Guide): + + <aop:config> + <aop:pointcut id="transactional" + expression="execution(* com...*Service.processMessage(..))" /> + <aop:advisor pointcut-ref="transactional" + advice-ref="retryAdvice" order="-1"/> +</aop:config> + +<bean id="retryAdvice" + class="org.springframework.batch.repeat.interceptor.RepeatOperationsInterceptor"/> + + + The example above uses a default + RepeatTemplate inside the interceptor. To change + the policies, listeners etc. you only need to inject an instance of + RepeatTemplate into the interceptor. + + If the intercepted method returns void then the + interceptor always returns ExitStatus.CONTINUABLE (so there is a danger of + an infinite loop if the CompletionPolicy does not + have a finite end point). Otherwise it returns + ExitStatus.CONTINUABLE until the return value from the + intercepted method is null, at which point it returns + ExitStatus.FINISHED. So the business logic inside the target + method can signal that there is no more work to do by returning + null, or by throwing an exception that is re-thrown by the + ExceptionHandler in the provided + RepeatTemplate. +
\ No newline at end of file diff --git a/docs/src/site/docbook/reference/retry.xml b/docs/src/site/docbook/reference/retry.xml index ed92363f1..ac6bbd22d 100644 --- a/docs/src/site/docbook/reference/retry.xml +++ b/docs/src/site/docbook/reference/retry.xml @@ -1,12 +1,284 @@ - + Retry
- + RetryTemplate - + To make processing more robust and less prone to failure, sometimes + it helps to automatically retry a failed operation in case it might + succeed on a subsequent attempt. Errors that are susceptible to this kind + of treatment are transient in nature, for example a remote call to a web + service or RMI service that fails because of a network glitch, or a + DeadLockLoserException in a database update. To + automate the retry of such operations Spring Batch has the + RetryOperations strategy. The + RetryOperations interface looks like this: + + public interface RetryOperations { + + Object execute(RetryCallback retryCallback) throws Exception; + +}where the callback is a simple interface that allows you to + insert some business logic to be retried + + public interface RetryCallback { + + Object doWithRetry(RetryContext context) throws Throwable; + +}The callback is executed and if it fails (by throwing an + Exception), it will be retried until either it is + successful, or the implementation decides to abort. + + The simplest general purpose implementation of + RetryOperations is + RetryTemplate. It could be used like this + + RetryTemplate template = new RetryTemplate(); + +template.setRetryPolicy(new TimeoutRetryPolicy(30000L)); + +Object result = template.execute(new RetryCallback() { + + public Object doWithRetry(RetryContext context) { + // Do stuff that might fail, e.g. webservice operation + return result; + } + +}); + + In the example we execute a web service call and return the result + to the user. If that call fails then it is retried until a timeout is + reached. + +
+ RetryContext + + The method parameter for the RetryCallback + is a RetryContext. Many callbacks will simply + ignore the context, but if necessary it can be used as an attribute bag + to store data for the duration of the iteration. + + A RetryContext will have a parent context + if there is a nested retry in progress in the same thread. The parent + context is occasionally useful for storing data that need to be shared + between calls to execute. +
+
+ +
+ Retry Policies + + Inside a RetryTemplate the decision to retry + or fail in the execute method is determined by a + RetryPolicy which is also a factory for the + RetryContext. The + RetryTemplate has the reponsibility to use the + current policy to create a RetryContext and pass + that in to the RetryCallback at every attempt. + After a callback fails the RetryTemplate has to + make a call to the RetryPolicy to ask it to update + its state (which will be stored in the + RetryContext), then it asks the policy if another + attempt can be made. If another attempt cannot be made (e.g. a limit is + reached or a timeout is detected) then the policy is also responsible for + handling the exhausted state. Simple implementations will just throw + RetryExhaustedException, and any enclosing + transaction will be rolled back. More sophisticated implementations might + attempt to take some recovery action, in which case the transaction can + remain intact. + +
+ Stateless Retry + + In the simplest case a retry is just a while loop - the + RetryTemplate can just keep trying until it + either succeeds or fails. The RetryContext is + storing some state to determine whether to retry or abort, but this + state is on the stack and there is no need to store it anywhere + globally, so we call this stateless retry. The distinction between + stateless and stateful retry is contained in the implementation of the + RetryPolicy (the + RetryTemplate can handle both). + + Spring Batch provides some simple general purpose implementations + of stateless RetryPolicy, for example a + SimpleRetryPolicy, and the + TimeoutRetryPolicy used in the example above. The + SimpleRetryPolicy just allows a retry on any of a + named list of exception types, up to a fixed number of times. + + Users might need to implement their own retry policies for more + customized decisions, e.g. if there is a well-known solution-specific + classification of exceptions into retryable and not retryable. +
+ +
+ Stateful Retry + + Where the failure has caused a transactional resource to become + invalid there are some special considerations. This does not apply to a + simple remote call because there was no transactional resource + (usually), but it does sometimes apply to a database update, especially + when using Hibernate. In this case it only makes sense to rethrow the + exception that called the failure immediately, so that the transaction + can roll back, and we can start a new valid one. + + In these cases a stateless retry is not good enough because the + re-throw and roll back necessarily involve leaving the + RetryOperations.execute() method and potentially losing the + context that was on the stack. To avoid losing it we have to introduce a + storage strategy to lift it off the stack and put it (at a minimum) in + heap storage. For this purpose Spring Batch provides the + ItemReaderRetryPolicy and a storage strategy + RetryContextCache. The default implementation of + the RetryContextCache is in memory, using a + simple Map. Advanced usage with multiple + processes in a clustered environment might also consider implementing + the RetryContextCache with a cluster cache of + some sort (even in a clustered environment this might be + overkill). + +
+ Item processing and stateful retry + + Part of the reponsibility of a stateful retry policy is to + recognise the failed operations when they come back in a new + transaction. To facilitate this in the commonest case where an object + of some type (like a message or message payload) is being processed, + Spring Batch provides the + ItemReaderRetryPolicy. This works in + conjunction with a special RetryCallback + implementation ItemReaderRetryCallback, which + in turn relies on the user providing an + ItemReader and an + ItemWriter. This callback implements the common + pattern where it reads from a reader and passes the result to a + writer. + + The way the failed operations are recognised in this + implementation is by recognising the object that is returned from the + ItemReader. To recognise the item the user can + provide an ItemKeyGenerator strategy, either by + injecting it directly into the + ItemReaderRetryCallback, or by implementing the + interface in the ItemReader, or by accepting + the default which is to simply use the item to identify itself. + + When the retry is exhausted, because a stateful retry is always + in a fresh transaction, there is also the option to handle the failed + item in a different way, instead of calling the + RetryCallback (which is presumed now to be + almost certain to fail). This option is provided by the + ItemRecoverer strategy. Like the key generator, + it can be directly injected or provided by implementing the interface + in either the ItemReader or + ItemWriter. + + The decision to retry or not is actually delegated to a regular + stateless retry policy, so the usual concerns about limits and + timeouts can be injected into the + ItemReaderRetryPolicy through the delegate + property. +
+
+
+ +
+ Backoff Policies + + When retrying after a transient failure it often helps to wait a bit + before trying again, because usually the failure is caused by some problem + that will only be resolved by waiting. If a + RetryCallback fails, the + RetryTemplate can pause execution according to the + BackoffPolicy in place. + + public interface BackoffPolicy { + + BackOffContext start(RetryContext context); + + void backOff(BackOffContext backOffContext) + throws BackOffInterruptedException; + +}A BackoffPolicy is free to implement + the backOff in any way it chooses. The policies provided by Spring Batch + out of the box all use Object.wait(). A common use case is to + backoff with an exponentially increasing wait period, to avoid two retries + getting into lock step and both failing - this is a lesson learned from + the ethernet. For this purpose Spring Batch provides the + ExponentialBackoffPolicy. +
+ +
+ Listeners + + Often it is useful to be able to receive additional callbacks for + cross cutting concerns across a number of different retries. For this + purpose Spring Batch provides the RetryListener + interface. The RetryTemplate allows users to + register RetryListeners, and they will be given + callbacks with the RetryContext and + Throwable where available during the + iteration. + + The interface looks like this: + + public interface RetryListener { + + void open(RetryContext context, RetryCallback callback); + + void onError(RetryContext context, RetryCallback callback, Throwable e); + + void close(RetryContext context, RetryCallback callback, Throwable e); +} +The open and + close callbacks come before and after the entire + retry in the simplest case, and onError applies + to the individual RetryCallback calls. The close + method might also receive a Throwable, if there has + been an error it is the last one thrown by the + RetryCallback. + + Note that when there is more than one listener, they are in a list, + so there is an order. In this case open is called + in the same order, and onError and + close are called in reverse order. +
+ +
+ Declarative Retry + + Sometimes there is some business processing that you know you want + to retry every time it happens. The classic example of this is the remote + service call. Spring Batch provides an AOP interceptor that wraps a method + call in a RetryOperations for just this purpose. + The RetryOperationsInterceptor executes the + intercepted method and retries on failure according to the + RetryPolicy in the provided + RepeatTemplate. + + Here is an example of declarative iteration using the Spring AOP + namespace to repeat a service call to a method called + remoteCall (for more detail on how to configure + AOP interceptors see the Spring User Guide): + + <aop:config> + <aop:pointcut id="transactional" + expression="execution(* com...*Service.remoteCall(..))" /> + <aop:advisor pointcut-ref="transactional" + advice-ref="retryAdvice" order="-1"/> +</aop:config> + +<bean id="retryAdvice" + class="org.springframework.batch.retry.interceptor.RetryOperationsInterceptor"/> + + + The example above uses a default + RetryTemplate inside the interceptor. To change the + policies, listeners etc. you only need to inject an instance of + RetryTemplate into the interceptor.
\ No newline at end of file diff --git a/spring-batch-infrastructure/src/main/java/org/springframework/batch/retry/backoff/BackOffPolicy.java b/spring-batch-infrastructure/src/main/java/org/springframework/batch/retry/backoff/BackOffPolicy.java index 8200d506c..980a82583 100644 --- a/spring-batch-infrastructure/src/main/java/org/springframework/batch/retry/backoff/BackOffPolicy.java +++ b/spring-batch-infrastructure/src/main/java/org/springframework/batch/retry/backoff/BackOffPolicy.java @@ -33,7 +33,6 @@ import org.springframework.batch.retry.RetryContext; * the corresponding {@link BackOffContext} object created by the call to * {@link #start}. * - * @since 2.1 * @author Rob Harrop * @author Dave Syer */ diff --git a/spring-batch-infrastructure/src/site/apt/index.apt b/spring-batch-infrastructure/src/site/apt/index.apt index d048b71f5..11434fe7d 100644 --- a/spring-batch-infrastructure/src/site/apt/index.apt +++ b/spring-batch-infrastructure/src/site/apt/index.apt @@ -19,15 +19,15 @@ template.setCompletionPolicy(new FixedChunkSizeCompletionPolicy(2)); template.iterate(new RepeatCallback() { - public boolean doInIteration(RepeatContext context) { + public ExitStatus doInIteration(RepeatContext context) { // Do stuff in batch... - return true; // Return false to signal exhausted data + return ExitStatus.CONTINUABLE; // Return ExitStatus.FINISHED to signal exhausted data } }); +--- - The callback is executed repeatedly, until the termination policy + The callback is executed repeatedly, until the completion policy determines that the batch should end. The framework provides <<>> for automatic retry of