This commit improves the performance of splitting a step execution.
It moves the logic of finding the last step execution of a job instance
to the database (instead of doing it in memory).
Resolves BATCH-2716
Before this commit, getting the running job executions loaded
all job executions from the database and filtered them in-memory.
This commit uses `JobExplorer#findRunningJobExecutions` which issues
a query that does the filtering on the database side. This change
considerably improves the performance of stopping a job.
Resolves BATCH-2422
Before this commit, getting the last job instance/execution required to
load all job instances/executions from the database and filter them
on the client side in memory.
This commit introduces new methods that use database queries to get
the last job instance/execution without the need to load all job
instances/executions. This change improves memory consumption as well as
the performance of starting the next instance of a job.
Resolves BATCH-1784
There is no such concept of job Id. A job has a name but not an Id.
A job instance however has an Id. The goal of the getter suggested in
BATCH-2769 is to access the JobInstance Id.
This commit renames the newly added StepContext#getJobId to
StepContext#getJobInstanceId which is more explicit.
Resolves BATCH-2769
Tests modified in this change set fail randomly due to shared
mutable state between multiple threads.
This commit ensures that the shared state is correctly
synchronised between threads or re-initialized before each test run.
The command `ping 1.1.1.1 -n 1 -w 5000` sends only one packet to the
remote address and might finish before the configured timeout of 10ms
which makes some tests to fail. Moreover, pinging 1.1.1.1 requires the
host (which can be the CI build server) to have internet connection.
This command can also fail if there is no internet connection which
makes some tests (expecting the command to succeed) to fail too.
This commit uses the command `ping 127.0.0.1` which does not require
an internet connection and which will, by default [1], send 4 packets
and wait for a timeout of 4 seconds for each request. This should take
more time than the configured timeout of 10ms.
Resolves BATCH-2722
[1]: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/ping
Before this commit, JobParametersBuilder#getNextJobParameters was checking
for restartability conditions. This is not only already done by the
JobLauncher, but also makes it inconsistent with the behaviour of
CommandLineJobRunner when used with "-next" option and
JobOperator#startNextInstance, which is starting the next instance in
sequence based on the incrementer without dealing with restartability.
This commit fixes the JobParametersBuilder#getNextJobParameters to behave
like the CommandLineJobRunner and JobOperator in regards to starting
the next instance.
Resolves BATCH-2711
This is to exclude from capturing erroneous Job Executions. An example is
whenever a TaskRejectedException is thrown after submitting to the
taskExecutor in SimpleJobLauncher#run(), the JobExecution is left without
a Start or End Time. Also related tests are fixed.
Resolves BATCH-2675
1. The first issue is:
```
> Task :api
DefaultBatchConfigurer.java:18: error: cannot find symbol
import javax.annotation.PostConstruct;
^
symbol: class PostConstruct
location: package javax.annotation
DataSourceConfiguration.java:18: error: cannot find symbol
import javax.annotation.PostConstruct;
^
symbol: class PostConstruct
location: package javax.annotation
DefaultBatchConfigurer.java:94: error: cannot find symbol
@PostConstruct
^
symbol: class PostConstruct
location: class DefaultBatchConfigurer
DataSourceConfiguration.java:46: error: cannot find symbol
@PostConstruct
^
symbol: class PostConstruct
location: class DataSourceConfiguration
4 errors
```
This issue is fixed by adding the `javax.annotation-api` dependency
2. The second issue is:
```
javadoc: error - An internal exception has occurred.
(com.sun.tools.javac.code.ClassFinder$BadClassFile: bad class file:
/org/springframework/batch/core/configuration/support/
GenericApplicationContextFactory$ResourceAnnotationApplicationContext$1.class
class file contains malformed variable arity method:
GenericApplicationContextFactory$ResourceAnnotationApplicationContext$1
Please remove or make sure it appears in the correct subdirectory of the classpath.)
```
The workaround to this issue is to use a named inner class instead of
an anonymous one.
Upgrade jacoco to version 0.8.2
Fix failing tests on Java 9
* `javax.xml.bind` is no longer contained in the default class path in
Java SE 9. This commit adds the module `java.xml.bind` to the JVM args
for tests
* JsrBeanDefinitionDocumentReaderTests are failing because
`ClassLoader.class.getResourceAsStream` has a different behaviour on Java 9.
According to `https://stackoverflow.com/a/45173837/5019386`, it's best to
use the resource-lookup methods in Class rather than those in ClassLoader.
* DefaultJobParametersExtractorJobParametersTests#testGetAllJobParameters
is failing because jobParameters.toString() returns "{foo=bar, spam=bucket}"
on Java 9 and "{spam=bucket, foo=bar}" on Java 8. The fix asserts that
jobParameters contains the expected key/value pairs without relying on the
toString method
JIRA: BATCH-2751
Before this commit, it was not possible to override the transaction
manager by subclassing DefaultBatchConfigurer and overriding the
getTransactionManager method.
This commit uses the getTransactionManager method in the initialize method
in order to take into account the transaction manager provided by the user.
Resolves BATCH-2294
Currently, when a method of an annotated listener throws an exception,
the exception is wrapped in a InvocationTargetException (by the reflection
API) which in turn is wrapped in a IllegalArgumentException (by Spring Batch).
This requires the user to unwrap the original exception from the
StepListenerFailedException. This behavior is not consistent with
interface based listeners where the original exception is the root
cause of StepListenerFailedException.
This commit unwraps the original exception and make it the root cause of
StepListenerFailedException.
Resolves BATCH-2213
This commit aligns the XML and Java based validations.
When using XML to configure a chunk oriented step both an ItemReader and
ItemWriter are requied. However when using Java based configuration the
ItemWriter is optional if an ItemProcessor is present.
Having no ItemWriter and only an ItemProcessor lead to strange results
in one of our batch jobs, which was accidentily configured without an
ItemProcessor.
Related: BATCH-1520
Fixes: BATCH-2624
Before this commit, the AutomaticJobRegistrar was started on
ContextRefreshedEvent event. This is sometimes too late to register jobs
especially when other life cycle components that use the jobs are started
before this registrar.
This commit changes the AutomaticJobRegistrar to implement SmartLifeCycle
and makes its autoStartup and phase properties configurable.
It should be noted that the "onApplicationEvent" method has been removed
even if it is a public API. This method is not intended to be used by
client code and even if it was, its usage is considered wrong anyway.
Resolves BATCH-2564
Before this commit, annotation based chunk listeners were not registered
when using a non-fault tolerant step builder.
This commit moves the code of chunk listener annotations handling to the
AbstractTaskletStepBuilder so that other tasklet builders can use it.
Resolves BATCH-2445
Before this commit, the JsrPartitionHandler was polling partitions
completion continuously. This causes a high CPU usage.
This commit makes the polling thread sleep for a configurable amount
of time in order to decrease CPU usage during the polling period.
Resolves BATCH-2401
Currently, the Jackson2ExecutionContextStringSerializer fails to
deserialize json representations of:
* empty JobParameters instances due to the presence of "empty":true
in the serialized json String. In this case, Jackson is not able to find a
property named "empty" in the target type (JobParameters)
* JobParameter instances due to the presence of several constructors.
In this case, Jackson does not know which constructor to use.
This commit fixes these two issues by adding a custom Jackson module with:
* a mixin to ignore the "isEmpty" getter in JobParameters type
* a custom deserializer for JobParameter type
Resolves BATCH-2680
Currently, when multiple data sources are defined in the context, an
IllegalStateException is thrown even if one of the data sources is
annotated with @Primary (which should be the one to use).
This commit makes it possible to use the data source annotated with
@Primary when multiple data sources are defined. Note that the context
initialization will still fail (with a UnsatisfiedDependencyException
from Spring's bean factory) if multiple data sources are defined and
none of them is annotated with @Primary. If multiple data sources are
defined and none of them is annotated with @Primary but one of them is
named "dataSource", this data source will be used by the batch
configuration due to autowiring by name (this detail has been documented
in the javadoc of @EnableBatchProcessing).
Resolves BATCH-2537
When a tasklet is declared with xml using the shortcut version, the
MethodInvokingTaskletAdapter that is created automatically does not
address passing parameters expected by Tasklet#execute (which is
incorrect since the documentation of the schema attribute "method"
says the bean should define a method with the same signature).
This commit fixes parameters passing when using the shortcut version.
Resolves BATCH-2397
Before this commit, the filter count of the contribution was applied
for each item of a scanned chunk. For example, with a chunk of 30,
if the filter count is 10 and an item is skipped during write, then the
filter count is equal to 210 (10 + 20 * 10).
If this commit is applied, the filter count will be re-initialized when
scanning the chunk.
Resolves BATCH-2663
Before this commit, when a job execution is stopped from a different
JVM than the one running the job, a warning says that the job cannot
be found. This was confusing to some users since the job can be found
in the database (but is actually not defined in the job registry of
the application context of the second JVM).
After this commit is applied, the warning will be more explicit to
inform the user that the job cannot be found in the job registry
(to not be confused with the database)
Resolves BATCH-2667
In a previous commit, the JobParametersBuilder was updated to include
some code from Spring Boot that handled the incrementing of
JobParameters for a previous job. That commit brought over a private
`merge` method that is actually useful for general consumption.
This commit adds a `addJobParameters` method to the builder providing
the same functionality the `merge` method did in a public method.
This commit adds a new `ItemProcessor` that delegates to a
`java.util.function.Function`. It also enables users to configure a
`Function` as an `ItemProcessor` via the builders (so they don't need to
worry about the `ItemProcessor` wrapper in the first place).
Resolves BATCH-2641
The current default for serializing the ExecutionContext is via XStream
using Jettison's driver. However, recent updates to Jettison make it
incompatible with XStream with no progress on a fix. Because of this,
and to encourage the use of well supported library combinations, the
default ExecutionContext serialization mechanism for Spring Batch has
been switched to Jackson (via the
Jackson2ExecutionContextStringSerializer). This commit also depricates
the XStreamExecutionContextStringSerializer for removal at a later date.
However, it is still available for users that require that during a
migration.
Resolves BATCH-2575
