Commit Graph

1867 Commits

Author SHA1 Message Date
Mahmoud Ben Hassine
9f39449bb0 Fix ClassCastException in RunIdIncrementer
Before this commit, the RunIdIncrementer was failing
with a ClassCastException if the run.id parameter is
not passed as a Long.

This commit makes the RunIdIncrementer more liberal
in what it accepts by trying to parse the parameter
to a Long.

Resolves #3799
2020-11-09 12:06:31 +01:00
Mahmoud Ben Hassine
791cfd5e8a Polish 0c500794f6
This commit adds a test case for building a flow
job with a job-scoped step.

Issue #857
2020-10-27 14:40:39 +01:00
Artyom Gabeev
0c500794f6 Do not call getName on step/flow - they may be Job/Step scoped proxies
Resolves #857
2020-10-27 13:45:41 +01:00
Mahmoud Ben Hassine
a0bcd7ce45 Add ability to amend trusted classes in Jackson2ExecutionContextStringSerializer
This commit adds the ability to specify additional trusted
classes without having to provide a custom object mapper.

Issue #3765
2020-10-16 11:36:18 +02:00
Mahmoud Ben Hassine
75de738b04 Deprecate SQLFire support
SQLFire was announced [1] to be in EOL as of November 1st, 2014.
This commit marks SQL scripts for SQLFire as deprecated and
scheduled for removal in v5.

[1]: https://www.vmware.com/latam/products/pivotal-sqlfire.html

Resolves #815
2020-09-30 16:20:58 +02:00
Mahmoud Ben Hassine
342a1a35dc Document usage of non-identifying job parameters
Resolves #3633
2020-09-24 10:10:29 +02:00
Mahmoud Ben Hassine
b079805353 Deprecate ScheduledJobParametersFactory
This commit deprecates ScheduledJobParametersFactory
in favor of DefaultJobParametersConverter.

Resolves #3781
2020-09-16 21:00:25 +02:00
Mahmoud Ben Hassine
0958cf4f3d Deprecate the Map-based JobRepository and JobExplorer
This commit deprecates the Map-based JobRepository
and JobExplorer factory beans with the associated
DAOs in favor of using the JDBC-based implementations
with an in-memory database.

Resolves #3780
2020-09-16 14:47:05 +02:00
Mahmoud Ben Hassine
982ad7d5e9 Deprecate JobExecution#stop
Calling jobExecution#stop manually is prone to
forgetting to save the job execution in the job
repository.

The way to request a job execution to stop is
by using JobOperator#stop or by using the
CommandLineJobRunner with the "-stop" option.
Both will correctly set the status to STOPPING
and update the job execution in the repository.

Resolves #1605
2020-09-15 13:54:23 +02:00
Michael Minella
b708dfb025 Updates for GraalVM
This commit updates all places where Spring Batch uses an @Configuration
annotation to no proxy bean methods. This is needed for GraalVM support.
2020-09-08 21:54:24 +02:00
Mahmoud Ben Hassine
744d1834fe Add support for annotation based listeners in JobBuilder
Resolves #817
2020-08-21 10:31:03 +02:00
Mahmoud Ben Hassine
a333c16459 Add JobParametersIncrementer implementation based on a DataFieldMaxValueIncrementer
Issue #1521
2020-08-11 21:13:20 +02:00
Mahmoud Ben Hassine
ac485ee326 Tag spring.batch.job.active metric with the job name
Resolves #3750
2020-07-24 12:20:43 +02:00
Mahmoud Ben Hassine
9a79b4572d Update Jackson2ExecutionContextStringSerializer 2020-06-10 20:11:08 +02:00
Yanming Zhou
28768dc0c6 Support microsecond accuracy for MySQL
see 320cce6446
2020-05-15 11:13:28 +02:00
Yanming Zhou
1322f91492 Improve performance in JdbcStepExecutionDao
It's similar to cafd35c95f
2020-05-15 09:36:25 +02:00
Parikshit Dutta
320cce6446 Updated MySQL DDL and Added Migration Script for v4.3 2020-05-12 16:32:17 +02:00
Maxim Cheban
cafd35c95f Improve performance in JdbcStepExecutionDao
Optimize SQL in GET_LAST_STEP_EXECUTION to work fast on large amount of records in %PREFIX%JOB_EXECUTION table
2020-04-25 06:53:20 +02:00
Baris Cubukcuoglu
b50c665f04 Add method StepExecutionDao.countStepExecutions to eliminate for-loop in SimpleJobRepository.getStepExecutionCount
Resolves #3657
2020-04-24 07:08:46 +02:00
Mahmoud Ben Hassine
6cca32fde0 Fix metrics collection in FaultTolerantChunkProcessor
Before this commit, metrics were not collected in a fault-tolerant step.
This commit updates the FaultTolerantChunkProcessor to collect metrics.

For the record, chunk scanning is not covered for two reasons:

1. When scanning a chunk, there is a single item in each write operation,
so it would be incorrect to report a metric called "chunk.write" for a
single item. We could argue that it is a singleton chunk, but still..
If we want to time scanned (aka individual) items, we need a more fine
grained timer called "scanned.item.write" for example.

2. The end result can be confusing and might distort the overall metrics
view in case of errors (because of the noisy metrics of additional transactions
for individual items).

As a reminder, the goal of the "chunk.write" metric is to give an overview
of the write operation time of the whole chunk and not to time each item
individually (this could be done using an `ItemWriteListener` if needed).

Resolves #3664
2020-03-31 16:37:12 +02:00
jinwook han
0ecc052190 fix documentation of JobExecutionNotRunningException
execution -> checked exception
2020-03-24 16:15:20 +01:00
Michael Minella
8e8f9c8101 Added comparitor for state transitions when using java config
Spring Batch orders the transitions as it goes from state to state based
on specificity.  The XML configuration has always had this
functionality.  However, when creating the JSR-352 implementation, the
mechanism for which this occured was refactored.  That occured at about
the same time as the java builders were introduced.  Because of this
crossing of paths, the java configuration option for defining jobs has
never correctly sorted the transitions.  This PR applys the sorting
algorithm to the java configuration, making XML and java configuration
behave the same.

Resolves #3638
2020-02-07 16:27:12 +01:00
Pascal Schumacher
2916da30dd Improve formatting of IllegalArgumentException message thrown by DefaultExecutionContextSerializer 2020-01-17 11:51:02 +01:00
Mahmoud Ben Hassine
f19f32fd79 Remove unused generateSql gradle task and its corresponding resources 2020-01-07 10:51:24 +01:00
larryjung
193f82005a Update outdated comment 2019-11-26 13:50:28 +01:00
Mahmoud Ben Hassine
f55d3bc81e Update docs with expected behaviour in regards to skippable exceptions
Before this commit, the expected behaviour when a skippbale exception
occurs in a fault tolerant chunk-oriented step was not documented in
details.

This commit update the docs and adds a sample for each case (when a
skippable exception occurs during read, process and write).

Resolves BATCH-2541
2019-11-25 11:24:21 +01:00
Chase Diem
4ad06b5081 BATCH-2845 fix for XSD documentation 2019-11-15 14:56:16 +01:00
Yanming Zhou
af4549bc6a Add https schemas 2019-11-15 11:04:24 +01:00
klauswolfgang
cc6f60871e BATCH-2480 Fix regression due to the fix of BATCH-2442 2019-11-14 11:41:04 +01:00
Mahmoud Ben Hassine
5b46c919ea Deprecate ListPreparedStatementSetter in favor of ArgumentPreparedStatementSetter
ListPreparedStatementSetter is almost a duplicate of
ArgumentPreparedStatementSetter except that it accepts a List of
arguments instead of an array of arguments.

Resolves BATCH-2796
2019-09-27 11:05:33 +02:00
Mahmoud Ben Hassine
2306141826 Polish a7092a21e4 2019-09-17 17:06:44 +02:00
mikeldpl
a7092a21e4 Added @Nullable annotation to overriding methods.
Resolves BATCH-2839
2019-09-17 16:50:44 +02:00
Mahmoud Ben Hassine
4ec227cd6e Revert 276acd1960
This commit reverts the change to `StepLocator` as it introduces
an inconsistency with `JobLocator` (one returns null and the other
throws an exception when no step/job is found with the given name).
2019-09-17 16:50:44 +02:00
Mahmoud Ben Hassine
63d3e4e043 Remove unused methods in BatchMetrics
Related to BATCH-2774 (See PR #690)
2019-09-06 14:52:25 +02:00
Michael Minella
fd569c4b83 Polish 2019-09-05 14:07:08 -05:00
Mahmoud Ben Hassine
62a8f441ab Improve the performance of step partitioning
This commit improves the performance of splitting a step execution.
It moves the logic of finding the last step execution of a job instance
to the database (instead of doing it in memory).

Resolves BATCH-2716
2019-09-05 13:47:53 -05:00
Mahmoud Ben Hassine
a15a6bfad8 Fix object mapper customization in Jackson2ExecutionContextStringSerializer
This commit fixes the setter of object mapper to copy the configuration
of the custom object mapper provided by the user and "augment" it with
the JobParametersModule.

Resolves BATCH-2828
2019-09-04 12:52:28 +02:00
Michael Minella
117e52bde5 Updates 2019-08-02 22:02:10 -05:00
Michael Minella
4fa15dd53d Still working on the DecisionStepTests#testDecisionAfterSplit 2019-08-02 21:57:50 -05:00
Michael Minella
4287f5a2ce Updated to address DecisionStepTests#testDecisionAfterSplit occasional failure 2019-08-02 21:28:49 -05:00
Michael Minella
4b9554fcce Polish 2019-08-02 14:33:45 -05:00
Mahmoud Ben Hassine
872845cae8 Improve the lookup for running job executions in CommandLineJobRunner
Before this commit, getting the running job executions loaded
all job executions from the database and filtered them in-memory.

This commit uses `JobExplorer#findRunningJobExecutions` which issues
a query that does the filtering on the database side. This change
considerably improves the performance of stopping a job.

Resolves BATCH-2422
2019-07-26 10:24:20 -05:00
Mahmoud Ben Hassine
7c1fbede1b Add new methods to get the last job instance/execution in JobInstance/JobExecution DAOs
Before this commit, getting the last job instance/execution required to
load all job instances/executions from the database and filter them
on the client side in memory.

This commit introduces new methods that use database queries to get
the last job instance/execution without the need to load all job
instances/executions. This change improves memory consumption as well as
the performance of starting the next instance of a job.

Resolves BATCH-1784
2019-07-25 10:56:19 -05:00
Mahmoud Ben Hassine
4d3090ac02 Show step execution duration in the logs
Resolves BATCH-2823
2019-05-20 16:11:14 +02:00
Mahmoud Ben Hassine
10f2281996 Show job execution duration in the logs
Resolves BATCH-2775
2019-05-16 14:52:21 +02:00
Mahmoud Ben Hassine
56bd4d651d Update Spring Framework version to 5.2.0.M2 2019-05-13 22:40:23 +02:00
Mahmoud Ben Hassine
57c8d70ff3 Initial support for batch metrics with Micrometer
Implements BATCH-2774
2019-04-11 15:53:30 +02:00
Saikat Bhadra
30365edf58 Fix typo in log message
Resolves BATCH-2782
2019-04-02 10:07:06 +02:00
Mahmoud Ben Hassine
a9fc6be6be Fix schema location of jsr 352 namespace 2019-03-26 21:53:14 +01:00
Spring Operator
d8fc58338d URL Cleanup
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://www.apache.org/licenses/ with 2 occurrences migrated to:
  https://www.apache.org/licenses/ ([https](https://www.apache.org/licenses/) result 200).
* [ ] http://www.apache.org/licenses/LICENSE-2.0 with 1796 occurrences migrated to:
  https://www.apache.org/licenses/LICENSE-2.0 ([https](https://www.apache.org/licenses/LICENSE-2.0) result 200).
2019-03-22 10:08:19 +01:00