Spring/spring-batch
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Replacing "master" and "slave"

Browse Source 
Per a request from Michael Minella, I replaced "master" with "manager" and "slave" with "worker" in the Spring Batch Reference Guide. I heartily concur with replacing terms that have such dark connotations.
This commit is contained in:
Jay Bryant
2019-05-21 10:40:28 -05:00
committed by Michael Minella
parent 3743894fa4
commit 85b3e90026
5 changed files with 59 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
spring-batch-docs/asciidoc/images/partitioning-overview.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.0 KiB

After

Width:  |  Height:  |  Size: 6.5 KiB

BIN
spring-batch-docs/asciidoc/images/remote-chunking.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 6.9 KiB

18
spring-batch-docs/asciidoc/jsr-352.adoc
View File

@@ -559,7 +559,7 @@ Traditional Spring Batch jobs have four ways of scaling (the last two capable of
* Multiple threads - Executing a single step via multiple threads. * Multiple threads - Executing a single step via multiple threads.
* Partitioning - Dividing the data up for parallel processing (master/slave). * Partitioning - Dividing the data up for parallel processing (manager/worker).
* Remote Chunking - Executing the processor piece of logic remotely. * Remote Chunking - Executing the processor piece of logic remotely.
@@ -584,8 +584,8 @@ JSR-352 provides two options for scaling batch jobs. Both options support only
==== Partitioning ==== Partitioning
Conceptually, partitioning in JSR-352 is the same as it is in Spring Batch. Meta-data is provided Conceptually, partitioning in JSR-352 is the same as it is in Spring Batch. Meta-data is provided
to each slave to identify the input to be processed with the slaves reporting back to the master the to each worker to identify the input to be processed, with the workers reporting back to the manager the
results upon completion. However, there are some important differences: results upon completion. However, there are some important differences:
* Partitioned `Batchlet` - This will run multiple instances of the * Partitioned `Batchlet` - This will run multiple instances of the
configured `Batchlet` on multiple threads. Each instance will have configured `Batchlet` on multiple threads. Each instance will have
@@ -610,10 +610,10 @@ Conceptually, partitioning in JSR-352 is the same as it is in Spring Batch. Met
* `StepExecutions` - In Spring Batch, partitioned steps are run as * `StepExecutions` - In Spring Batch, partitioned steps are run as
master/slave. Within JSR-352, the same configuration occurs. However, the slave steps do manager/worker. Within JSR-352, the same configuration occurs. However, the worker steps do
not get official `StepExecutions`. Because of that, calls to not get official `StepExecutions`. Because of that, calls to
`JsrJobOperator#getStepExecutions(long jobExecutionId)` will only `JsrJobOperator#getStepExecutions(long jobExecutionId)` will only
return the `StepExecution` for the master. return the `StepExecution` for the manager.
[NOTE] [NOTE]
==== ====
@@ -623,16 +623,16 @@ via the `JobExplorer` and Spring Batch Admin.
==== ====
* Compensating logic - Since Spring Batch implements the master/slave logic of * Compensating logic - Since Spring Batch implements the manager/worker logic of
partitioning using steps, `StepExecutionListeners` can be used to partitioning using steps, `StepExecutionListeners` can be used to
handle compensating logic if something goes wrong. However, since the slaves JSR-352 handle compensating logic if something goes wrong. However, since the workers JSR-352
provides a collection of other components for the ability to provide compensating logic when provides a collection of other components for the ability to provide compensating logic when
errors occur and to dynamically set the exit status. These components include the following: errors occur and to dynamically set the exit status. These components include the following:
|=============== |===============
|__Artifact Interface__|__Description__ |__Artifact Interface__|__Description__
|`javax.batch.api.partition.PartitionCollector`|Provides a way for slave steps to send information back to the |`javax.batch.api.partition.PartitionCollector`|Provides a way for worker steps to send information back to the
master. There is one instance per slave thread. manager. There is one instance per worker thread.
|`javax.batch.api.partition.PartitionAnalyzer`|End point that receives the information collected by the |`javax.batch.api.partition.PartitionAnalyzer`|End point that receives the information collected by the
`PartitionCollector` as well as the resulting `PartitionCollector` as well as the resulting
statuses from a completed partition. statuses from a completed partition.

34
spring-batch-docs/asciidoc/scalability.adoc
View File

@@ -238,13 +238,13 @@ pattern:
.Remote Chunking .Remote Chunking
image::{batch-asciidoc}images/remote-chunking.png[Remote Chunking, scaledwidth="60%"] image::{batch-asciidoc}images/remote-chunking.png[Remote Chunking, scaledwidth="60%"]
The master component is a single process, and the slaves are multiple remote processes. The manager component is a single process, and the workers are multiple remote processes.
This pattern works best if the master is not a bottleneck, so the processing must be more This pattern works best if the manager is not a bottleneck, so the processing must be more
expensive than the reading of items (as is often the case in practice). expensive than the reading of items (as is often the case in practice).
The master is an implementation of a Spring Batch `Step` with the `ItemWriter` replaced The manager is an implementation of a Spring Batch `Step` with the `ItemWriter` replaced
by a generic version that knows how to send chunks of items to the middleware as by a generic version that knows how to send chunks of items to the middleware as
messages. The slaves are standard listeners for whatever middleware is being used (for messages. The workers are standard listeners for whatever middleware is being used (for
example, with JMS, they would be `MesssageListener` implementations), and their role is example, with JMS, they would be `MesssageListener` implementations), and their role is
to process the chunks of items using a standard `ItemWriter` or `ItemProcessor` plus to process the chunks of items using a standard `ItemWriter` or `ItemProcessor` plus
`ItemWriter`, through the `ChunkProcessor` interface. One of the advantages of using this `ItemWriter`, through the `ChunkProcessor` interface. One of the advantages of using this
@@ -274,12 +274,12 @@ pattern:
image::{batch-asciidoc}images/partitioning-overview.png[Partitioning Overview, scaledwidth="60%"] image::{batch-asciidoc}images/partitioning-overview.png[Partitioning Overview, scaledwidth="60%"]
The `Job` runs on the left-hand side as a sequence of `Step` instances, and one of the The `Job` runs on the left-hand side as a sequence of `Step` instances, and one of the
`Step` instances is labeled as a master. The slaves in this picture are all identical `Step` instances is labeled as a manager. The workers in this picture are all identical
instances of a `Step`, which could in fact take the place of the master, resulting in the instances of a `Step`, which could in fact take the place of the manager, resulting in the
same outcome for the `Job`. The slaves are typically going to be remote services but same outcome for the `Job`. The workers are typically going to be remote services but
could also be local threads of execution. The messages sent by the master to the slaves could also be local threads of execution. The messages sent by the manager to the workers
in this pattern do not need to be durable or have guaranteed delivery. Spring Batch in this pattern do not need to be durable or have guaranteed delivery. Spring Batch
metadata in the `JobRepository` ensures that each slave is executed once and only once for metadata in the `JobRepository` ensures that each worker is executed once and only once for
each `Job` execution. each `Job` execution.
The SPI in Spring Batch consists of a special implementation of `Step` (called the The SPI in Spring Batch consists of a special implementation of `Step` (called the
@@ -290,7 +290,7 @@ and their role is shown in the following sequence diagram:
.Partitioning SPI .Partitioning SPI
image::{batch-asciidoc}images/partitioning-spi.png[Partitioning SPI, scaledwidth="60%"] image::{batch-asciidoc}images/partitioning-spi.png[Partitioning SPI, scaledwidth="60%"]
The `Step` on the right in this case is the "remote" slave, so, potentially, there are The `Step` on the right in this case is the "`remote`" worker, so, potentially, there are
many objects and or processes playing this role, and the `PartitionStep` is shown driving many objects and or processes playing this role, and the `PartitionStep` is shown driving
the execution. the execution.
@@ -299,7 +299,7 @@ The following example shows the `PartitionStep` configuration:
[source, xml, role="xmlContent"] [source, xml, role="xmlContent"]
---- ----
<step id="step1.master"> <step id="step1.manager">
<partition step="step1" partitioner="partitioner"> <partition step="step1" partitioner="partitioner">
<handler grid-size="10" task-executor="taskExecutor"/> <handler grid-size="10" task-executor="taskExecutor"/>
</partition> </partition>
@@ -313,8 +313,8 @@ The following example shows the `PartitionStep` configuration using java configu
[source, java, role="javaContent"] [source, java, role="javaContent"]
---- ----
@Bean @Bean
public Step step1Master() { public Step step1Manager() {
return stepBuilderFactory.get("step1.master") return stepBuilderFactory.get("step1.manager")
.<String, String>partitioner("step1", partitioner()) .<String, String>partitioner("step1", partitioner())
.step(step1()) .step(step1())
.gridSize(10) .gridSize(10)
@@ -332,7 +332,7 @@ https://github.com/spring-projects/spring-batch/tree/master/spring-batch-samples
Batch Samples] (see `Partition*Job.xml` configuration). Batch Samples] (see `Partition*Job.xml` configuration).
Spring Batch creates step executions for the partitions called "step1:partition0", and so Spring Batch creates step executions for the partitions called "step1:partition0", and so
on. Many people prefer to call the master step "step1:master" for consistency. You can on. Many people prefer to call the manager step "step1:manager" for consistency. You can
use an alias for the step (by specifying the `name` attribute instead of the `id` use an alias for the step (by specifying the `name` attribute instead of the `id`
attribute). attribute).
@@ -367,7 +367,7 @@ following example:
[source, xml, role="xmlContent"] [source, xml, role="xmlContent"]
---- ----
<step id="step1.master"> <step id="step1.manager">
<partition step="step1" handler="handler"/> <partition step="step1" handler="handler"/>
</step> </step>
@@ -386,8 +386,8 @@ as shown in the following example:
[source, java, role="javaContent"] [source, java, role="javaContent"]
---- ----
@Bean @Bean
public Step step1Master() { public Step step1Manager() {
return stepBuilderFactory.get("step1.master") return stepBuilderFactory.get("step1.manager")
.partitioner("step1", partitioner()) .partitioner("step1", partitioner())
.partitionHandler(partitionHandler()) .partitionHandler(partitionHandler())
.build(); .build();

66
spring-batch-docs/asciidoc/spring-batch-integration.adoc
View File

@@ -718,12 +718,12 @@ public Job chunkJob() {
---- ----
The `ItemReader` reference points to the bean you want The `ItemReader` reference points to the bean you want
to use for reading data on the master. The `ItemWriter` reference to use for reading data on the manager. The `ItemWriter` reference
points to a special `ItemWriter` points to a special `ItemWriter`
(called `ChunkMessageChannelItemWriter`), (called `ChunkMessageChannelItemWriter`),
as described above. The processor (if any) is left off the as described above. The processor (if any) is left off the
master configuration, as it is configured on the worker. The manager configuration, as it is configured on the worker. The
following configuration provides a basic master setup. You following configuration provides a basic manager setup. You
should check any additional component properties, such as should check any additional component properties, such as
throttle limits and so on, when implementing your use case. throttle limits and so on, when implementing your use case.
@@ -878,7 +878,7 @@ public org.apache.activemq.ActiveMQConnectionFactory connectionFactory() {
} }
/* /*
* Configure inbound flow (requests coming from the master) * Configure inbound flow (requests coming from the manager)
*/ */
@Bean @Bean
public DirectChannel requests() { public DirectChannel requests() {
@@ -894,7 +894,7 @@ public IntegrationFlow inboundFlow(ActiveMQConnectionFactory connectionFactory)
} }
/* /*
* Configure outbound flow (replies going to the master) * Configure outbound flow (replies going to the manager)
*/ */
@Bean @Bean
public DirectChannel replies() { public DirectChannel replies() {
@@ -925,7 +925,7 @@ public ChunkProcessorChunkHandler<Integer> chunkProcessorChunkHandler() {
---- ----
Most of these configuration items should look familiar from the Most of these configuration items should look familiar from the
master configuration. Workers do not need access to manager configuration. Workers do not need access to
the Spring Batch `JobRepository` nor the Spring Batch `JobRepository` nor
to the actual job configuration file. The main bean of interest to the actual job configuration file. The main bean of interest
is the `chunkProcessorChunkHandler`. The is the `chunkProcessorChunkHandler`. The
@@ -933,7 +933,7 @@ is the `chunkProcessorChunkHandler`. The
configured `SimpleChunkProcessor`, which is where you would provide a reference to your configured `SimpleChunkProcessor`, which is where you would provide a reference to your
`ItemWriter` (and, optionally, your `ItemWriter` (and, optionally, your
`ItemProcessor`) that will run on the worker `ItemProcessor`) that will run on the worker
when it receives chunks from the master. when it receives chunks from the manager.
For more information, see the section of the "Scalability" chapter on For more information, see the section of the "Scalability" chapter on
link:$$https://docs.spring.io/spring-batch/reference/html/scalability.html#remoteChunking$$[Remote Chunking]. link:$$https://docs.spring.io/spring-batch/reference/html/scalability.html#remoteChunking$$[Remote Chunking].
@@ -942,7 +942,7 @@ Starting from version 4.1, Spring Batch Integration introduces the `@EnableBatch
annotation that can be used to simplify a remote chunking setup. This annotation provides annotation that can be used to simplify a remote chunking setup. This annotation provides
two beans that can be autowired in the application context: two beans that can be autowired in the application context:
* `RemoteChunkingMasterStepBuilderFactory`: used to configure the master step * `RemoteChunkingMasterStepBuilderFactory`: used to configure the manager step
* `RemoteChunkingWorkerBuilder`: used to configure the remote worker integration flow * `RemoteChunkingWorkerBuilder`: used to configure the remote worker integration flow
These APIs take care of configuring a number of components as described in the following diagram: These APIs take care of configuring a number of components as described in the following diagram:
@@ -950,8 +950,8 @@ These APIs take care of configuring a number of components as described in the f
.Remote Chunking Configuration .Remote Chunking Configuration
image::{batch-asciidoc}images/remote-chunking-config.png[Remote Chunking Configuration, scaledwidth="80%"] image::{batch-asciidoc}images/remote-chunking-config.png[Remote Chunking Configuration, scaledwidth="80%"]
On the master side, the `RemoteChunkingMasterStepBuilderFactory` allows you to On the manager side, the `RemoteChunkingMasterStepBuilderFactory` lets you
configure a master step by declaring: configure a manager step by declaring:
* the item reader to read items and send them to workers * the item reader to read items and send them to workers
* the output channel ("Outgoing requests") to send requests to workers * the output channel ("Outgoing requests") to send requests to workers
@@ -962,10 +962,10 @@ A `ChunkMessageChannelItemWriter` and the `MessagingTemplate` are not needed to
On the worker side, the `RemoteChunkingWorkerBuilder` allows you to configure a worker to: On the worker side, the `RemoteChunkingWorkerBuilder` allows you to configure a worker to:
* listen to requests sent by the master on the input channel ("Incoming requests") * listen to requests sent by the manager on the input channel ("Incoming requests")
* call the `handleChunk` method of `ChunkProcessorChunkHandler` for each request * call the `handleChunk` method of `ChunkProcessorChunkHandler` for each request
with the configured `ItemProcessor` and `ItemWriter` with the configured `ItemProcessor` and `ItemWriter`
* send replies on the output channel ("Outgoing replies") to the master * send replies on the output channel ("Outgoing replies") to the manager
There is no need to explicitly configure the `SimpleChunkProcessor` There is no need to explicitly configure the `SimpleChunkProcessor`
and the `ChunkProcessorChunkHandler` (Those can be explicitly configured if required). and the `ChunkProcessorChunkHandler` (Those can be explicitly configured if required).
@@ -979,14 +979,14 @@ The following example shows how to use these APIs:
public class RemoteChunkingJobConfiguration { public class RemoteChunkingJobConfiguration {
@Configuration @Configuration
public static class MasterConfiguration { public static class ManagerConfiguration {
@Autowired @Autowired
private RemoteChunkingMasterStepBuilderFactory masterStepBuilderFactory; private RemoteChunkingMasterStepBuilderFactory managerStepBuilderFactory;
@Bean @Bean
public TaskletStep masterStep() { public TaskletStep managerStep() {
return this.masterStepBuilderFactory.get("masterStep") return this.managerStepBuilderFactory.get("managerStep")
.chunk(100) .chunk(100)
.reader(itemReader()) .reader(itemReader())
.outputChannel(requests()) // requests sent to workers .outputChannel(requests()) // requests sent to workers
@@ -1009,8 +1009,8 @@ public class RemoteChunkingJobConfiguration {
return this.workerBuilder return this.workerBuilder
.itemProcessor(itemProcessor()) .itemProcessor(itemProcessor())
.itemWriter(itemWriter()) .itemWriter(itemWriter())
.inputChannel(requests()) // requests received from the master .inputChannel(requests()) // requests received from the manager
.outputChannel(replies()) // replies sent to the master .outputChannel(replies()) // replies sent to the manager
.build(); .build();
} }
@@ -1130,7 +1130,7 @@ configuration:
---- ----
/* /*
* Configuration of the master side * Configuration of the manager side
*/ */
@Bean @Bean
public PartitionHandler partitionHandler() { public PartitionHandler partitionHandler() {
@@ -1239,7 +1239,7 @@ You must also ensure that the partition `handler` attribute maps to the `partiti
[source, xml, role="xmlContent"] [source, xml, role="xmlContent"]
---- ----
<job id="personJob"> <job id="personJob">
<step id="step1.master"> <step id="step1.manager">
<partition partitioner="partitioner" handler="partitionHandler"/> <partition partitioner="partitioner" handler="partitionHandler"/>
... ...
</step> </step>
@@ -1251,7 +1251,7 @@ You must also ensure that the partition `handler` attribute maps to the `partiti
---- ----
public Job personJob() { public Job personJob() {
return jobBuilderFactory.get("personJob") return jobBuilderFactory.get("personJob")
.start(stepBuilderFactory.get("step1.master") .start(stepBuilderFactory.get("step1.manager")
.partitioner("step1.worker", partitioner()) .partitioner("step1.worker", partitioner())
.partitionHandler(partitionHandler()) .partitionHandler(partitionHandler())
.build()) .build())
@@ -1265,7 +1265,7 @@ link:$$https://github.com/spring-projects/spring-batch/tree/master/spring-batch-
The `@EnableBatchIntegration` annotation that can be used to simplify a remote The `@EnableBatchIntegration` annotation that can be used to simplify a remote
partitioning setup. This annotation provides two beans useful for remote partitioning: partitioning setup. This annotation provides two beans useful for remote partitioning:
* `RemotePartitioningMasterStepBuilderFactory`: used to configure the master step * `RemotePartitioningMasterStepBuilderFactory`: used to configure the manager step
* `RemotePartitioningWorkerStepBuilderFactory`: used to configure the worker step * `RemotePartitioningWorkerStepBuilderFactory`: used to configure the worker step
These APIs take care of configuring a number of components as described in the following diagram: These APIs take care of configuring a number of components as described in the following diagram:
@@ -1276,8 +1276,8 @@ image::{batch-asciidoc}images/remote-partitioning-polling-config.png[Remote Part
.Remote Partitioning Configuration (with replies aggregation) .Remote Partitioning Configuration (with replies aggregation)
image::{batch-asciidoc}images/remote-partitioning-aggregation-config.png[Remote Partitioning Configuration (with replies aggregation), scaledwidth="80%"] image::{batch-asciidoc}images/remote-partitioning-aggregation-config.png[Remote Partitioning Configuration (with replies aggregation), scaledwidth="80%"]
On the master side, the `RemotePartitioningMasterStepBuilderFactory` allows you to On the manager side, the `RemotePartitioningMasterStepBuilderFactory` allows you to
configure a master step by declaring: configure a manager step by declaring:
* the `Partitioner` used to partition data * the `Partitioner` used to partition data
* the output channel ("Outgoing requests") to send requests to workers * the output channel ("Outgoing requests") to send requests to workers
@@ -1289,9 +1289,9 @@ The `MessageChannelPartitionHandler` and the `MessagingTemplate` are not needed
On the worker side, the `RemotePartitioningWorkerStepBuilderFactory` allows you to configure a worker to: On the worker side, the `RemotePartitioningWorkerStepBuilderFactory` allows you to configure a worker to:
* listen to requests sent by the master on the input channel ("Incoming requests") * listen to requests sent by the manager on the input channel ("Incoming requests")
* call the `handle` method of `StepExecutionRequestHandler` for each request * call the `handle` method of `StepExecutionRequestHandler` for each request
* send replies on the output channel ("Outgoing replies") to the master * send replies on the output channel ("Outgoing replies") to the manager
There is no need to explicitly configure the `StepExecutionRequestHandler` (which can be explicitly configured if required). There is no need to explicitly configure the `StepExecutionRequestHandler` (which can be explicitly configured if required).
@@ -1305,15 +1305,15 @@ The following example shows how to use these APIs:
public class RemotePartitioningJobConfiguration { public class RemotePartitioningJobConfiguration {
@Configuration @Configuration
public static class MasterConfiguration { public static class ManagerConfiguration {
@Autowired @Autowired
private RemotePartitioningMasterStepBuilderFactory masterStepBuilderFactory; private RemotePartitioningMasterStepBuilderFactory managerStepBuilderFactory;
@Bean @Bean
public Step masterStep() { public Step managerStep() {
return this.masterStepBuilderFactory return this.managerStepBuilderFactory
.get("masterStep") .get("managerStep")
.partitioner("workerStep", partitioner()) .partitioner("workerStep", partitioner())
.gridSize(10) .gridSize(10)
.outputChannel(outgoingRequestsToWorkers()) .outputChannel(outgoingRequestsToWorkers())
@@ -1335,8 +1335,8 @@ public class RemotePartitioningJobConfiguration {
public Step workerStep() { public Step workerStep() {
return this.workerStepBuilderFactory return this.workerStepBuilderFactory
.get("workerStep") .get("workerStep")
.inputChannel(incomingRequestsFromMaster()) .inputChannel(incomingRequestsFromManager())
.outputChannel(outgoingRepliesToMaster()) .outputChannel(outgoingRepliesToManager())
.chunk(100) .chunk(100)
.reader(itemReader()) .reader(itemReader())
.processor(itemProcessor()) .processor(itemProcessor())