The constructors of the wrapper classes for primitives (Double,
Integer, Long) have been deprecated since JDK 9 and should no longer be
called. Instead the static factory #valueOf should be used.
Override SimpleStepBuilder.faultTolerant() in FaultTolerantStepBuilder
to prevent creation of a new FaultTolerantStepBuilder when calling
faultTolerant() on an existing FaultTolerantStepBuilder. Otherwise
configuration, like chunkListeners, are lost.
Issue #3840
Before this commit, the logic that validates if a job
instance is complete or not was based only on the size
of previous parameters, without checking if they are
identifying or not.
This commit introduces a change that uses only
identifying job parameters to identify a job instance
and validate its previous executions.
Issue #1221
Guarded the logging statements which do concatenation of strings
or calling toString on objects. When a log level isn't enabled
this still would produce garbage that would need to be collected.
Guarded all logging up to info, warn and error can be assumed to be
enabled on a production system.
This commit temporarily ignores tests that are failing
intermittently on Windows. A link to the corresponding
issue has been added to each test.
FTR, I tried to ignore those tests only on Windows by
using `Assume.assumeFalse(SystemUtils.IS_OS_WINDOWS);`
where `SystemUtils` is from `org.apache.commons.lang3`
but this assumption does not seem to work as expected.
Some tests in PartitionParserTests were
failing intermittently due to the usage
of non-synchronized shared state between
concurrent threads.
This commit updates the test code to use
`AtomicInteger` instead of `int` for the
state shared between concurrent threads.
This test was failing intermittently due to an incorrect
way of waiting for a job execution to finish, which is:
```
while(jobExecution.isRunning()) {
// wait for async launched job to complete execution
}
```
`JobExecution#isRunning()` is based on the status
of the job execution in memory which might not be
persisted yet. Here is an excerpt from the Javadoc:
```
Test if this JobExecution indicates that it is
running. It should be noted that this does not
necessarily mean that it has been persisted as
such yet.
```
That's why in the case where `isRunning` returns
false and the JobExecution is not persisted yet
(which is still in a running status in db),
the second attempt of re-running the job fails
with a `JobExecutionAlreadyRunningException`.
This commit fixes the loop by continuously
checking the status of the Job execution
in the job repository until it reaches one
of the end statuses.
Issue #1121
JsrSplitParsingTests#test and DecisionStepTests#testDecisionAfterSplit
were failing intermittently with:
```
org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
at org.springframework.batch.core.jsr.job.flow.JsrFlowJob.doExecute(JsrFlowJob.java:88) ~[main/:?]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) [main/:?]
at org.springframework.batch.core.jsr.launch.JsrJobOperator$2.run(JsrJobOperator.java:674) [main/:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=flow1.step1 at state=flow1.step1.flow1.step1 with exception
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:178) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
... 1 more
Caused by: org.springframework.dao.ConcurrencyFailureException: PreparedStatementCallback; SQL [INSERT INTO BATCH_STEP_EXECUTION_CONTEXT (SHORT_CONTEXT, SERIALIZED_CONTEXT, STEP_EXECUTION_ID) VALUES(?, ?, ?)]; transaction rollback: serialization failure; nested exception is java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:73) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:79) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1541) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:667) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
... 1 more
Caused by: java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
... 1 more
Caused by: org.hsqldb.HsqlException: transaction rollback: serialization failure
at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.error.Error.error(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.Session.handleAbortTransaction(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.Session.executeCompiledStatement(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.Session.execute(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source) ~[hsqldb-2.5.1.jar:2.5.1]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:136) ~[commons-dbcp2-2.8.0.jar:2.8.0]
at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.0.jar:5.3.0]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.persistSerializedContext(JdbcExecutionContextDao.java:236) ~[main/:?]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.saveExecutionContext(JdbcExecutionContextDao.java:189) ~[main/:?]
at org.springframework.batch.core.repository.support.SimpleJobRepository.add(SimpleJobRepository.java:177) ~[main/:?]
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:371) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:134) ~[spring-tx-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.0.jar:5.3.0]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.0.jar:5.3.0]
at com.sun.proxy.$Proxy41.add(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[main/:?]
at org.springframework.batch.core.jsr.job.flow.support.state.JsrStepState.handle(JsrStepState.java:53) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) ~[main/:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:94) ~[main/:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[main/:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_232]
... 1 more
```
This failure is due to a concurrency issue when executing
the split flow with the default SimpleAsyncTaskExecutor.
Several attempts have been made to fix this issue with no success:
* use `@DirtiesContext` annotation
* use a separate db for each test
* use `READ_COMMITTED` isolation level in the job repository
* downgrade hsqldb from v2.5.1 to v2.4.1 (since v2.5.1 has
introduced several changes in the MVCC mode)
The issue seems to be related to how the in-memory database
is shared between tests *and* how the test context is cached
(ie the combination of both).
This commit moves these tests to a separate test class which
seem to fix the issue.
Before this commit, the RunIdIncrementer was failing
with a ClassCastException if the run.id parameter is
not passed as a Long.
This commit makes the RunIdIncrementer more liberal
in what it accepts by trying to parse the parameter
to a Long.
Resolves#3799
This commit deprecates the Map-based JobRepository
and JobExplorer factory beans with the associated
DAOs in favor of using the JDBC-based implementations
with an in-memory database.
Resolves#3780
Calling jobExecution#stop manually is prone to
forgetting to save the job execution in the job
repository.
The way to request a job execution to stop is
by using JobOperator#stop or by using the
CommandLineJobRunner with the "-stop" option.
Both will correctly set the status to STOPPING
and update the job execution in the repository.
Resolves#1605
