From 736b92613768ac9da71bcac4c4892b9e81d5babe Mon Sep 17 00:00:00 2001 From: Jay Bryant Date: Thu, 19 Oct 2017 11:03:03 -0500 Subject: [PATCH] 90-character pass for transaction-appendix.adoc I removed extraneous white spice from non-code lines and made the non-code lines break as close to 90 characters as possible. I also fixed a typo. Applying cod review changes on merge --- .../asciidoc/transaction-appendix.adoc | 294 +++++++----------- 1 file changed, 115 insertions(+), 179 deletions(-) diff --git a/spring-batch-docs/asciidoc/transaction-appendix.adoc b/spring-batch-docs/asciidoc/transaction-appendix.adoc index bf10a3e4d..db61583bd 100644 --- a/spring-batch-docs/asciidoc/transaction-appendix.adoc +++ b/spring-batch-docs/asciidoc/transaction-appendix.adoc @@ -8,15 +8,11 @@ == Batch Processing and Transactions [[transactionsNoRetry]] - - === Simple Batching with No Retry -Consider the following simple example of a nested batch with no - retries. It shows a common scenario for batch processing: - An input source is processed until exhausted, and we commit - periodically at the end of a "chunk" of processing. - +Consider the following simple example of a nested batch with no retries. It shows a +common scenario for batch processing: An input source is processed until exhausted, and +we commit periodically at the end of a "chunk" of processing. ---- @@ -33,24 +29,19 @@ Consider the following simple example of a nested batch with no ---- -The input operation (3.1) could be a message-based receive - (such as from JMS), or a file-based read, but to recover and continue - processing with a chance of completing the whole job, it must be - transactional. The same applies to the operation at 3.2. It must - be either transactional or idempotent. +The input operation (3.1) could be a message-based receive (such as from JMS), or a +file-based read, but to recover and continue processing with a chance of completing the +whole job, it must be transactional. The same applies to the operation at 3.2. It must +be either transactional or idempotent. -If the chunk at `REPEAT` (3) fails because of a database exception at - 3.2, then `TX` (2) must roll back the whole chunk. +If the chunk at `REPEAT` (3) fails because of a database exception at 3.2, then `TX` (2) +must roll back the whole chunk. [[transactionStatelessRetry]] - - === Simple Stateless Retry -It is also useful to use a retry for an operation which is not - transactional, such as a call to a web-service or other remote - resource, as shown in the following example: - +It is also useful to use a retry for an operation which is not transactional, such as a +call to a web-service or other remote resource, as shown in the following example: ---- @@ -64,21 +55,17 @@ It is also useful to use a retry for an operation which is not ---- -This is actually one of the most useful applications of a retry, - since a remote call is much more likely to fail and be retryable - than a database update. As long as the remote access (2.1) - eventually succeeds, the transaction, `TX` (0), commits. If the - remote access (2.1) eventually fails, then the transaction, `TX` (0), is - guaranteed to roll back. +This is actually one of the most useful applications of a retry, since a remote call is +much more likely to fail and be retryable than a database update. As long as the remote +access (2.1) eventually succeeds, the transaction, `TX` (0), commits. If the remote +access (2.1) eventually fails, then the transaction, `TX` (0), is guaranteed to roll +back. [[repeatRetry]] - - === Typical Repeat-Retry Pattern -The most typical batch processing pattern is to add a retry to the - inner block of the chunk, as shown in the following example: - +The most typical batch processing pattern is to add a retry to the inner block of the +chunk, as shown in the following example: ---- @@ -103,76 +90,57 @@ The most typical batch processing pattern is to add a retry to the ---- The inner `RETRY` (4) block is marked as "stateful". See <> for a description of a stateful - retry. This means that if the the retry `PROCESS` (5) block fails, the - behavior of the `RETRY` (4) is as follows: +typical use case>> for a description of a stateful retry. This means that if the +retry `PROCESS` (5) block fails, the behavior of the `RETRY` (4) is as follows: +. Throw an exception, rolling back the transaction, `TX` (2), at the chunk level, and +allowing the item to be re-presented to the input queue. +. When the item re-appears, it might be retried depending on the retry policy in place, +executing `PROCESS` (5) again. The second and subsequent attempts might fail again and +re-throw the exception. +. Eventually, the item reappears for the final time. The retry policy disallows another +attempt, so `PROCESS` (5) is never executed. In this case, we follow the `RECOVER` (6) +path, effectively "skipping" the item that was received and is being processed. -. Throw an exception, rolling back the transaction, `TX` (2), at the - chunk level, and allowing the item to be re-presented to the input - queue. +Note that the notation used for the `RETRY` (4) in the plan above explicitly shows that +the input step (4.1) is part of the retry. It also makes clear that there are two +alternate paths for processing: the normal case, as denoted by `PROCESS` (5), and the +recovery path, as denoted in a separate block by `RECOVER` (6). The two alternate paths +are completely distinct. Only one is ever taken in normal circumstances. +In special cases (such as a special `TranscationValidException` type), the retry policy +might be able to determine that the `RECOVER` (6) path can be taken on the last attempt +after `PROCESS` (5) has just failed, instead of waiting for the item to be re-presented. +This is not the default behavior, because it requires detailed knowledge of what has +happened inside the `PROCESS` (5) block, which is not usually available. For example, if +the output included write access before the failure, then the exception should be +re-thrown to ensure transactional integrity. -. When the item re-appears, it might be retried depending on the - retry policy in place, executing `PROCESS` (5) again. The second and - subsequent attempts might fail again and re-throw the exception. +The completion policy in the outer `REPEAT` (1) is crucial to the success of the above +plan. If the output (5.1) fails, it may throw an exception (it usually does, as +described), in which case the transaction, `TX` (2), fails, and the exception could +propagate up through the outer batch `REPEAT` (1). We do not want the whole batch to +stop, because the `RETRY` (4) might still be successful if we try again, so we add +`exception=not critical` to the outer `REPEAT` (1). - -. Eventually, the item reappears for the final time. The retry - policy disallows another attempt, so `PROCESS` (5) is never - executed. In this case, we follow the `RECOVER` (6) path, effectively - "skipping" the item that was received and is being processed. - -Note that the notation used for the `RETRY` (4) in the plan above - explicitly shows that the the input step (4.1) is part of the retry. - It also makes clear that there are two alternate paths for - processing: the normal case, as denoted by `PROCESS` (5), and the - recovery path, as denoted in a separate block by `RECOVER` (6). The two alternate - paths are completely distinct. Only one is ever taken in normal - circumstances. - -In special cases (such as a special `TranscationValidException` - type), the retry policy might be able to determine that the - `RECOVER` (6) path can be taken on the last attempt after `PROCESS` (5) - has just failed, instead of waiting for the item to be re-presented. - This is not the default behavior, because it requires detailed - knowledge of what has happened inside the `PROCESS` (5) block, which is - not usually available. For example, if the output included write - access before the failure, then the exception should be re-thrown to - ensure transactional integrity. - -The completion policy in the outer `REPEAT` (1) is crucial to the - success of the above plan. If the output (5.1) fails, it may throw an - exception (it usually does, as described), in which case the - transaction, `TX` (2), fails, and the exception could propagate up through - the outer batch `REPEAT` (1). We do not want the whole batch to stop, - because the `RETRY` (4) might still be successful if we try again, so - we add `exception=not critical` to the outer `REPEAT` (1). - -Note, however, that if the `TX` (2) fails and we __do__ try again, by - virtue of the outer completion policy, the item that is next - processed in the inner `REPEAT` (3) is not guaranteed to be the one - that just failed. It might be, but it depends on the - implementation of the input (4.1). Thus, the output (5.1) might fail - again on either a new item or the old one. The client of the batch - should not assume that each `RETRY` (4) attempt is going to process the - same items as the last one that failed. For example, if the termination - policy for `REPEAT` (1) is to fail after 10 attempts, it fails - after 10 consecutive attempts but not necessarily at the same item. - This is consistent with the overall retry strategy. The inner - `RETRY` (4) is aware of the history of each item and can decide - whether or not to have another attempt at it. +Note, however, that if the `TX` (2) fails and we __do__ try again, by virtue of the outer +completion policy, the item that is next processed in the inner `REPEAT` (3) is not +guaranteed to be the one that just failed. It might be, but it depends on the +implementation of the input (4.1). Thus, the output (5.1) might fail again on either a +new item or the old one. The client of the batch should not assume that each `RETRY` (4) +attempt is going to process the same items as the last one that failed. For example, if +the termination policy for `REPEAT` (1) is to fail after 10 attempts, it fails after 10 +consecutive attempts but not necessarily at the same item. This is consistent with the +overall retry strategy. The inner `RETRY` (4) is aware of the history of each item and +can decide whether or not to have another attempt at it. [[asyncChunkProcessing]] - - === Asynchronous Chunk Processing -The inner batches or chunks in the <> - can be executed concurrently by configuring the outer batch to - use an `AsyncTaskExecutor`. The outer batch waits for all the - chunks to complete before completing. The following example shows asynchronous chunk processing: - +The inner batches or chunks in the <> can be executed +concurrently by configuring the outer batch to use an `AsyncTaskExecutor`. The outer +batch waits for all the chunks to complete before completing. The following example shows +asynchronous chunk processing: ---- @@ -197,16 +165,12 @@ The inner batches or chunks in the <> ---- [[asyncItemProcessing]] - - === Asynchronous Item Processing -The individual items in chunks in the <> - can also, in principle, be processed concurrently. In this case, the - transaction boundary has to move to the level of the individual - item, so that each transaction is on a single thread, as shown in the following example: - - +The individual items in chunks in the <> can also, in +principle, be processed concurrently. In this case, the transaction boundary has to move +to the level of the individual item, so that each transaction is on a single thread, as +shown in the following example: ---- @@ -230,25 +194,19 @@ The individual items in chunks in the <> ---- -This plan sacrifices the optimization benefit, which the simple plan - had, of having all the transactional resources chunked together. It - is only useful if the cost of the processing (5) is much higher than - the cost of transaction management (3). +This plan sacrifices the optimization benefit, which the simple plan had, of having all +the transactional resources chunked together. It is only useful if the cost of the +processing (5) is much higher than the cost of transaction management (3). [[transactionPropagation]] - - === Interactions Between Batching and Transaction Propagation -There is a tighter coupling between batch-retry and transaction management - than we would ideally like. In particular, a stateless retry cannot - be used to retry database operations with a transaction manager that - does not support NESTED propagation. - +There is a tighter coupling between batch-retry and transaction management than we would +ideally like. In particular, a stateless retry cannot be used to retry database +operations with a transaction manager that does not support NESTED propagation. The following example uses retry without repeat: - ---- 1 | TX { @@ -265,13 +223,11 @@ The following example uses retry without repeat: ---- -Again, and for the same reason, the inner transaction, `TX` (3), can - cause the outer transaction, `TX` (1), to fail, even if the `RETRY` (2) is - eventually successful. - -Unfortunately, the same effect percolates from the retry block up to - the surrounding repeat batch if there is one, as shown in the following example: +Again, and for the same reason, the inner transaction, `TX` (3), can cause the outer +transaction, `TX` (1), to fail, even if the `RETRY` (2) is eventually successful. +Unfortunately, the same effect percolates from the retry block up to the surrounding +repeat batch if there is one, as shown in the following example: ---- @@ -291,40 +247,32 @@ Unfortunately, the same effect percolates from the retry block up to ---- -Now, if TX (3) rolls back, it can pollute the whole batch at TX (1) and - force it to roll back at the end. +Now, if TX (3) rolls back, it can pollute the whole batch at TX (1) and force it to roll +back at the end. What about non-default propagation? +* In the preceding example, `PROPAGATION_REQUIRES_NEW` at `TX` (3) prevents the outer +`TX` (1) from being polluted if both transactions are eventually successful. But if `TX` +(3) commits and `TX` (1) rolls back, then `TX` (3) stays committed, so we violate the +transaction contract for `TX` (1). If `TX` (3) rolls back, `TX` (1) does not necessarily +(but it probably does in practice, because the retry throws a roll back exception). -* In the preceding example, `PROPAGATION_REQUIRES_NEW` at `TX` (3) - prevents the outer `TX` (1) from being polluted if both transactions - are eventually successful. But if `TX` (3) commits and `TX` (1) rolls - back, then `TX` (3) stays committed, so we violate the transaction - contract for `TX` (1). If `TX` (3) rolls back, `TX` (1) does not necessarily (but it probably - does in practice, because the retry throws a roll back - exception). +* `PROPAGATION_NESTED` at `TX` (3) works as we require in the retry case (and for a +batch with skips): `TX` (3) can commit but subsequently be rolled back by the outer +transaction, `TX` (1). If `TX` (3) rolls back, `TX` (1) rolls back in practice. This +option is only available on some platforms, not including Hibernate or +JTA, but it is the only one that consistently works. - -* `PROPAGATION_NESTED` at `TX` (3) works as we require in the retry - case (and for a batch with skips): `TX` (3) can commit but - subsequently be rolled back by the outer transaction, `TX` (1). If - `TX` (3) rolls back, `TX` (1) rolls back in practice. This - option is only available on some platforms, not including Hibernate or - JTA, but it is the only one that consistently works. - -Consequently, the `NESTED` pattern is best if the retry block contains any database access. +Consequently, the `NESTED` pattern is best if the retry block contains any database +access. [[specialTransactionOrthonogonal]] - - === Special Case: Transactions with Orthogonal Resources -Default propagation is always OK for simple cases where there are no - nested database transactions. Consider the following example, where the `SESSION` and - `TX` are not global `XA` resources, so their resources are orthogonal: - - +Default propagation is always OK for simple cases where there are no nested database +transactions. Consider the following example, where the `SESSION` and `TX` are not +global `XA` resources, so their resources are orthogonal: ---- @@ -339,34 +287,25 @@ Default propagation is always OK for simple cases where there are no ---- -Here there is a transactional message `SESSION` (0), but it does nt - participate in other transactions with - `PlatformTransactionManager`, so it does not propagate when `TX` (3) - starts. There is no database access outside the `RETRY` (2) block. If - `TX` (3) fails and then eventually succeeds on a retry, `SESSION` (0) can - commit (independently of a `TX` block). This is similar - to the vanilla "best-efforts-one-phase-commit" scenario. The worst - that can happen is a duplicate message when the `RETRY` (2) succeeds - and the `SESSION` (0) cannot commit (for example, because the message system is - unavailable). +Here there is a transactional message `SESSION` (0), but it does not participate in other +transactions with `PlatformTransactionManager`, so it does not propagate when `TX` (3) +starts. There is no database access outside the `RETRY` (2) block. If `TX` (3) fails and +then eventually succeeds on a retry, `SESSION` (0) can commit (independently of a `TX` +block). This is similar to the vanilla "best-efforts-one-phase-commit" scenario. The +worst that can happen is a duplicate message when the `RETRY` (2) succeeds and the +`SESSION` (0) cannot commit (for example, because the message system is unavailable). [[statelessRetryCannotRecover]] - - === Stateless Retry Cannot Recover -The distinction between a stateless and a stateful retry in the - typical example above is important. It is actually - ultimately a transactional constraint that forces the distinction, - and this constraint also makes it obvious why the distinction - exists. - - -We start with the observation that there is no way to skip an item - that failed and successfully commit the rest of the chunk unless we - wrap the item processing in a transaction. Consequently, we simplify the - typical batch execution plan to be as follows: +The distinction between a stateless and a stateful retry in the typical example above is +important. It is actually ultimately a transactional constraint that forces the +distinction, and this constraint also makes it obvious why the distinction exists. +We start with the observation that there is no way to skip an item that failed and +successfully commit the rest of the chunk unless we wrap the item processing in a +transaction. Consequently, we simplify the typical batch execution plan to be as +follows: ---- @@ -392,18 +331,15 @@ We start with the observation that there is no way to skip an item ---- The preceding example shows a stateless `RETRY` (3) with a `RECOVER` (5) path that kicks - in after the final attempt fails. The `stateless` label means - that the block is repeated without re-throwing any exception up - to some limit. This only works if the transaction, `TX` (4), has - propagation NESTED. +in after the final attempt fails. The `stateless` label means that the block is repeated +without re-throwing any exception up to some limit. This only works if the transaction, +`TX` (4), has propagation NESTED. -If the inner `TX` (4) has default propagation properties and rolls back, - it pollutes the outer `TX` (1). The inner transaction is assumed by - the transaction manager to have corrupted the transactional - resource, so it cannot be used again. +If the inner `TX` (4) has default propagation properties and rolls back, it pollutes the +outer `TX` (1). The inner transaction is assumed by the transaction manager to have +corrupted the transactional resource, so it cannot be used again. -Support for NESTED propagation is sufficiently rare that we choose - not to support recovery with stateless retries in the current versions of - Spring Batch. The same effect can always be achieved (at the - expense of repeating more processing) by using the - typical pattern above. +Support for NESTED propagation is sufficiently rare that we choose not to support +recovery with stateless retries in the current versions of Spring Batch. The same effect +can always be achieved (at the expense of repeating more processing) by using the +typical pattern above.