Add builders to simplify remote partitioning setup
Resolves BATCH-2687 * use hard coded constants in docs * change visibility of method `beanFactory` to public in builders
This commit is contained in:
committed by
Michael Minella
parent
9f2b752f3f
commit
39676a45c4
Binary file not shown.
|
After Width: | Height: | Size: 362 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 376 KiB |
@@ -945,7 +945,7 @@ two beans that can be autowired in the application context:
|
||||
* `RemoteChunkingMasterStepBuilderFactory`: used to configure the master step
|
||||
* `RemoteChunkingWorkerBuilder`: used to configure the remote worker integration flow
|
||||
|
||||
These APIs will take care of configuring a number of components as described in the following diagram:
|
||||
These APIs take care of configuring a number of components as described in the following diagram:
|
||||
|
||||
.Remote Chunking Configuration
|
||||
image::{batch-asciidoc}images/remote-chunking-config.png[Remote Chunking Configuration, scaledwidth="80%"]
|
||||
@@ -1261,3 +1261,92 @@ You must also ensure that the partition `handler` attribute maps to the `partiti
|
||||
|
||||
You can find a complete example of a remote partitioning job
|
||||
link:$$https://github.com/spring-projects/spring-batch/tree/master/spring-batch-samples#remote-partitioning-sample$$[here].
|
||||
|
||||
The `@EnableBatchIntegration` annotation that can be used to simplify a remote
|
||||
partitioning setup. This annotation provides two beans useful for remote partitioning:
|
||||
|
||||
* `RemotePartitioningMasterStepBuilderFactory`: used to configure the master step
|
||||
* `RemotePartitioningWorkerStepBuilderFactory`: used to configure the worker step
|
||||
|
||||
These APIs take care of configuring a number of components as described in the following diagram:
|
||||
|
||||
.Remote Partitioning Configuration (with job repository polling)
|
||||
image::{batch-asciidoc}images/remote-partitioning-polling-config.png[Remote Partitioning Configuration (with job repository polling), scaledwidth="80%"]
|
||||
|
||||
.Remote Partitioning Configuration (with replies aggregation)
|
||||
image::{batch-asciidoc}images/remote-partitioning-aggregation-config.png[Remote Partitioning Configuration (with replies aggregation), scaledwidth="80%"]
|
||||
|
||||
On the master side, the `RemotePartitioningMasterStepBuilderFactory` allows you to
|
||||
configure a master step by declaring:
|
||||
|
||||
* the `Partitioner` used to partition data
|
||||
* the output channel ("Outgoing requests") to send requests to workers
|
||||
* the input channel ("Incoming replies") to receive replies from workers (when configuring replies aggregation)
|
||||
* the poll interval and timeout parameters (when configuring job repository polling)
|
||||
|
||||
The `MessageChannelPartitionHandler` and the `MessagingTemplate` are not needed to be explicitly configured
|
||||
(Those can still be explicitly configured if required).
|
||||
|
||||
On the worker side, the `RemotePartitioningWorkerStepBuilderFactory` allows you to configure a worker to:
|
||||
|
||||
* listen to requests sent by the master on the input channel ("Incoming requests")
|
||||
* call the `handle` method of `StepExecutionRequestHandler` for each request
|
||||
* send replies on the output channel ("Outgoing replies") to the master
|
||||
|
||||
There is no need to explicitly configure the `StepExecutionRequestHandler` (which can be explicitly configured if required).
|
||||
|
||||
The following example shows how to use these APIs:
|
||||
|
||||
[source, java]
|
||||
----
|
||||
@Configuration
|
||||
@EnableBatchProcessing
|
||||
@EnableBatchIntegration
|
||||
public class RemotePartitioningJobConfiguration {
|
||||
|
||||
@Configuration
|
||||
public static class MasterConfiguration {
|
||||
|
||||
@Autowired
|
||||
private RemotePartitioningMasterStepBuilderFactory masterStepBuilderFactory;
|
||||
|
||||
@Bean
|
||||
public Step masterStep() {
|
||||
return this.masterStepBuilderFactory
|
||||
.get("masterStep")
|
||||
.partitioner("workerStep", partitioner())
|
||||
.gridSize(10)
|
||||
.outputChannel(outgoingRequestsToWorkers())
|
||||
.inputChannel(incomingRepliesFromWorkers())
|
||||
.build();
|
||||
}
|
||||
|
||||
// Middleware beans setup omitted
|
||||
|
||||
}
|
||||
|
||||
@Configuration
|
||||
public static class WorkerConfiguration {
|
||||
|
||||
@Autowired
|
||||
private RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory;
|
||||
|
||||
@Bean
|
||||
public Step workerStep() {
|
||||
return this.workerStepBuilderFactory
|
||||
.get("workerStep")
|
||||
.inputChannel(incomingRequestsFromMaster())
|
||||
.outputChannel(outgoingRepliesToMaster())
|
||||
.chunk(100)
|
||||
.reader(itemReader())
|
||||
.processor(itemProcessor())
|
||||
.writer(itemWriter())
|
||||
.build();
|
||||
}
|
||||
|
||||
// Middleware beans setup omitted
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
----
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
The Spring Batch 4.1 release adds the following features:
|
||||
|
||||
* A new `@SpringBatchTest` annotation to simplify testing batch components
|
||||
* A new `@EnableBatchIntegration` annotation to simplify remote chunking configuration
|
||||
* A new `@EnableBatchIntegration` annotation to simplify remote chunking and partitioning configuration
|
||||
* A new `JsonItemReader` and `JsonFileItemWriter` to support the JSON format
|
||||
* Add support for validating items with the Bean Validation API
|
||||
|
||||
@@ -125,6 +125,55 @@ that uses these new APIs in the
|
||||
link:$$https://github.com/spring-projects/spring-batch/tree/master/spring-batch-samples#remote-chunking-sample$$[samples module]
|
||||
as well as more details in the <<spring-batch-integration.adoc#remote-chunking,Spring Batch Integration>> chapter.
|
||||
|
||||
Just like the remote chunking configuration simplification, this version also
|
||||
introduces new APIs to simplify a remote partitioning setup:
|
||||
`RemotePartitioningMasterStepBuilder` and `RemotePartitioningWorkerStepBuilder`.
|
||||
Those can be autowired in your configuration class if the
|
||||
`@EnableBatchIntegration` is present as shown in the following example:
|
||||
|
||||
[source, java]
|
||||
----
|
||||
@Configuration
|
||||
@EnableBatchProcessing
|
||||
@EnableBatchIntegration
|
||||
public class RemotePartitioningAppConfig {
|
||||
|
||||
@Autowired
|
||||
private RemotePartitioningMasterStepBuilderFactory masterStepBuilderFactory;
|
||||
|
||||
@Autowired
|
||||
private RemotePartitioningWorkerStepBuilderFactory workerStepBuilderFactory;
|
||||
|
||||
@Bean
|
||||
public Step masterStep() {
|
||||
return this.masterStepBuilderFactory
|
||||
.get("masterStep")
|
||||
.partitioner("workerStep", partitioner())
|
||||
.gridSize(10)
|
||||
.outputChannel(outgoingRequestsToWorkers())
|
||||
.inputChannel(incomingRepliesFromWorkers())
|
||||
.build();
|
||||
}
|
||||
|
||||
@Bean
|
||||
public Step workerStep() {
|
||||
return this.workerStepBuilderFactory
|
||||
.get("workerStep")
|
||||
.inputChannel(incomingRequestsFromMaster())
|
||||
.outputChannel(outgoingRepliesToMaster())
|
||||
.chunk(100)
|
||||
.reader(itemReader())
|
||||
.processor(itemProcessor())
|
||||
.writer(itemWriter())
|
||||
.build();
|
||||
}
|
||||
|
||||
// Middleware beans setup omitted
|
||||
}
|
||||
----
|
||||
|
||||
You can find more details about these new APIs in the <<spring-batch-integration.adoc#remote-partitioning,Spring Batch Integration>> chapter.
|
||||
|
||||
[[whatsNewJson]]
|
||||
=== JSON support
|
||||
|
||||
|
||||
Reference in New Issue
Block a user