Rob Winch
committed by
Mahmoud Ben Hassine
Mahmoud Ben Hassine
parent
e36a44788d
commit
2e8d5063f7
@@ -0,0 +1,38 @@
|
||||
[[availableAttributesOfTheJobLaunchingGateway]]
|
||||
= Available Attributes of the Job-Launching Gateway
|
||||
|
||||
The job-launching gateway has the following attributes that you can set to control a job:
|
||||
|
||||
* `id`: Identifies the underlying Spring bean definition, which is an instance of either:
|
||||
** `EventDrivenConsumer`
|
||||
** `PollingConsumer`
|
||||
(The exact implementation depends on whether the component's input channel is a
|
||||
`SubscribableChannel` or a `PollableChannel`.)
|
||||
* `auto-startup`: Boolean flag to indicate that the endpoint should start automatically on
|
||||
startup. The default is `true`.
|
||||
* `request-channel`: The input `MessageChannel` of this endpoint.
|
||||
* `reply-channel`: `MessageChannel` to which the resulting `JobExecution` payload is sent.
|
||||
* `reply-timeout`: Lets you specify how long (in milliseconds) this gateway waits for the reply message
|
||||
to be sent successfully to the reply channel before throwing
|
||||
an exception. This attribute applies only when the channel
|
||||
might block (for example, when using a bounded queue channel
|
||||
that is currently full). Also, keep in mind that, when sending to a
|
||||
`DirectChannel`, the invocation occurs
|
||||
in the sender's thread. Therefore, the failing of the send
|
||||
operation may be caused by other components further downstream.
|
||||
The `reply-timeout` attribute maps to the
|
||||
`sendTimeout` property of the underlying
|
||||
`MessagingTemplate` instance. If not specified, the attribute
|
||||
defaults to -1,
|
||||
meaning that, by default, the `Gateway` waits indefinitely.
|
||||
* `job-launcher`: Optional. Accepts a
|
||||
custom
|
||||
`JobLauncher`
|
||||
bean reference.
|
||||
If not specified, the adapter
|
||||
re-uses the instance that is registered under the `id` of
|
||||
`jobLauncher`. If no default instance
|
||||
exists, an exception is thrown.
|
||||
* `order`: Specifies the order of invocation when this endpoint is connected as a subscriber
|
||||
to a `SubscribableChannel`.
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
[[launching-batch-jobs-through-messages]]
|
||||
= Launching Batch Jobs through Messages
|
||||
|
||||
When starting batch jobs by using the core Spring Batch API, you
|
||||
basically have two options:
|
||||
|
||||
* From the command line, with the `CommandLineJobRunner`
|
||||
* Programmatically, with either `JobOperator.start()` or `JobLauncher.run()`
|
||||
|
||||
For example, you may want to use the
|
||||
`CommandLineJobRunner` when invoking batch jobs by
|
||||
using a shell script. Alternatively, you can use the
|
||||
`JobOperator` directly (for example, when using
|
||||
Spring Batch as part of a web application). However, what about
|
||||
more complex use cases? Maybe you need to poll a remote (S)FTP
|
||||
server to retrieve the data for the Batch Job or your application
|
||||
has to support multiple different data sources simultaneously. For
|
||||
example, you may receive data files not only from the web but also from
|
||||
FTP and other sources. Maybe additional transformation of the input files is
|
||||
needed before invoking Spring Batch.
|
||||
|
||||
Therefore, it would be much more powerful to execute the batch job
|
||||
by using Spring Integration and its numerous adapters. For example,
|
||||
you can use a _File Inbound Channel Adapter_ to
|
||||
monitor a directory in the file-system and start the batch job as
|
||||
soon as the input file arrives. Additionally, you can create Spring
|
||||
Integration flows that use multiple different adapters to easily
|
||||
ingest data for your batch jobs from multiple sources
|
||||
simultaneously by using only configuration. Implementing all these
|
||||
scenarios with Spring Integration is easy, as it allows for
|
||||
decoupled, event-driven execution of the
|
||||
`JobLauncher`.
|
||||
|
||||
Spring Batch Integration provides the
|
||||
`JobLaunchingMessageHandler` class that you can
|
||||
use to launch batch jobs. The input for the
|
||||
`JobLaunchingMessageHandler` is provided by a
|
||||
Spring Integration message, which has a payload of type
|
||||
`JobLaunchRequest`. This class is a wrapper around the `Job`
|
||||
to be launched and around the `JobParameters` that are
|
||||
necessary to launch the Batch job.
|
||||
|
||||
The following image shows the typical Spring Integration
|
||||
message flow that is needed to start a Batch job. The
|
||||
link:$$https://www.enterpriseintegrationpatterns.com/toc.html$$[EIP (Enterprise Integration Patterns) website]
|
||||
provides a full overview of messaging icons and their descriptions.
|
||||
|
||||
.Launch Batch Job
|
||||
image::launch-batch-job.png[Launch Batch Job, scaledwidth="60%"]
|
||||
|
||||
|
||||
[[transforming-a-file-into-a-joblaunchrequest]]
|
||||
== Transforming a File into a JobLaunchRequest
|
||||
|
||||
The following example transforms a file into a `JobLaunchRequest`:
|
||||
|
||||
[source, java]
|
||||
----
|
||||
package io.spring.sbi;
|
||||
|
||||
import org.springframework.batch.core.Job;
|
||||
import org.springframework.batch.core.JobParametersBuilder;
|
||||
import org.springframework.batch.integration.launch.JobLaunchRequest;
|
||||
import org.springframework.integration.annotation.Transformer;
|
||||
import org.springframework.messaging.Message;
|
||||
|
||||
import java.io.File;
|
||||
|
||||
public class FileMessageToJobRequest {
|
||||
private Job job;
|
||||
private String fileParameterName;
|
||||
|
||||
public void setFileParameterName(String fileParameterName) {
|
||||
this.fileParameterName = fileParameterName;
|
||||
}
|
||||
|
||||
public void setJob(Job job) {
|
||||
this.job = job;
|
||||
}
|
||||
|
||||
@Transformer
|
||||
public JobLaunchRequest toRequest(Message<File> message) {
|
||||
JobParametersBuilder jobParametersBuilder =
|
||||
new JobParametersBuilder();
|
||||
|
||||
jobParametersBuilder.addString(fileParameterName,
|
||||
message.getPayload().getAbsolutePath());
|
||||
|
||||
return new JobLaunchRequest(job, jobParametersBuilder.toJobParameters());
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
[[the-jobexecution-response]]
|
||||
== The JobExecution Response
|
||||
|
||||
When a batch job is being executed, a
|
||||
`JobExecution` instance is returned. You can use this
|
||||
instance to determine the status of an execution. If
|
||||
a `JobExecution` is able to be created
|
||||
successfully, it is always returned, regardless of whether
|
||||
or not the actual execution is successful.
|
||||
|
||||
The exact behavior on how the `JobExecution`
|
||||
instance is returned depends on the provided
|
||||
`TaskExecutor`. If a
|
||||
`synchronous` (single-threaded)
|
||||
`TaskExecutor` implementation is used, the
|
||||
`JobExecution` response is returned only
|
||||
`after` the job completes. When using an
|
||||
`asynchronous`
|
||||
`TaskExecutor`, the
|
||||
`JobExecution` instance is returned
|
||||
immediately. You can then take the `id` of
|
||||
`JobExecution` instance
|
||||
(with `JobExecution.getJobId()`) and query the
|
||||
`JobRepository` for the job's updated status
|
||||
using the `JobExplorer`. For more
|
||||
information, see
|
||||
xref:job/advanced-meta-data.adoc#queryingRepository[Querying the Repository].
|
||||
|
||||
[[spring-batch-integration-configuration]]
|
||||
== Spring Batch Integration Configuration
|
||||
|
||||
Consider a case where someone needs to create a file `inbound-channel-adapter` to listen
|
||||
for CSV files in the provided directory, hand them off to a transformer
|
||||
(`FileMessageToJobRequest`), launch the job through the job launching gateway, and
|
||||
log the output of the `JobExecution` with the `logging-channel-adapter`.
|
||||
|
||||
[tabs]
|
||||
====
|
||||
Java::
|
||||
+
|
||||
The following example shows how that common case can be configured in Java:
|
||||
+
|
||||
.Java Configuration
|
||||
[source, java]
|
||||
----
|
||||
@Bean
|
||||
public FileMessageToJobRequest fileMessageToJobRequest() {
|
||||
FileMessageToJobRequest fileMessageToJobRequest = new FileMessageToJobRequest();
|
||||
fileMessageToJobRequest.setFileParameterName("input.file.name");
|
||||
fileMessageToJobRequest.setJob(personJob());
|
||||
return fileMessageToJobRequest;
|
||||
}
|
||||
|
||||
@Bean
|
||||
public JobLaunchingGateway jobLaunchingGateway() {
|
||||
TaskExecutorJobLauncher jobLauncher = new TaskExecutorJobLauncher();
|
||||
jobLauncher.setJobRepository(jobRepository);
|
||||
jobLauncher.setTaskExecutor(new SyncTaskExecutor());
|
||||
JobLaunchingGateway jobLaunchingGateway = new JobLaunchingGateway(jobLauncher);
|
||||
|
||||
return jobLaunchingGateway;
|
||||
}
|
||||
|
||||
@Bean
|
||||
public IntegrationFlow integrationFlow(JobLaunchingGateway jobLaunchingGateway) {
|
||||
return IntegrationFlow.from(Files.inboundAdapter(new File("/tmp/myfiles")).
|
||||
filter(new SimplePatternFileListFilter("*.csv")),
|
||||
c -> c.poller(Pollers.fixedRate(1000).maxMessagesPerPoll(1))).
|
||||
transform(fileMessageToJobRequest()).
|
||||
handle(jobLaunchingGateway).
|
||||
log(LoggingHandler.Level.WARN, "headers.id + ': ' + payload").
|
||||
get();
|
||||
}
|
||||
----
|
||||
|
||||
XML::
|
||||
+
|
||||
The following example shows how that common case can be configured in XML:
|
||||
+
|
||||
.XML Configuration
|
||||
[source, xml]
|
||||
----
|
||||
<int:channel id="inboundFileChannel"/>
|
||||
<int:channel id="outboundJobRequestChannel"/>
|
||||
<int:channel id="jobLaunchReplyChannel"/>
|
||||
|
||||
<int-file:inbound-channel-adapter id="filePoller"
|
||||
channel="inboundFileChannel"
|
||||
directory="file:/tmp/myfiles/"
|
||||
filename-pattern="*.csv">
|
||||
<int:poller fixed-rate="1000"/>
|
||||
</int-file:inbound-channel-adapter>
|
||||
|
||||
<int:transformer input-channel="inboundFileChannel"
|
||||
output-channel="outboundJobRequestChannel">
|
||||
<bean class="io.spring.sbi.FileMessageToJobRequest">
|
||||
<property name="job" ref="personJob"/>
|
||||
<property name="fileParameterName" value="input.file.name"/>
|
||||
</bean>
|
||||
</int:transformer>
|
||||
|
||||
<batch-int:job-launching-gateway request-channel="outboundJobRequestChannel"
|
||||
reply-channel="jobLaunchReplyChannel"/>
|
||||
|
||||
<int:logging-channel-adapter channel="jobLaunchReplyChannel"/>
|
||||
----
|
||||
====
|
||||
|
||||
|
||||
|
||||
|
||||
[[example-itemreader-configuration]]
|
||||
== Example ItemReader Configuration
|
||||
|
||||
Now that we are polling for files and launching jobs, we need to configure our Spring
|
||||
Batch `ItemReader` (for example) to use the files found at the location defined by the job
|
||||
parameter called "input.file.name", as the following bean configuration shows:
|
||||
|
||||
|
||||
[tabs]
|
||||
====
|
||||
Java::
|
||||
+
|
||||
The following Java example shows the necessary bean configuration:
|
||||
+
|
||||
.Java Configuration
|
||||
[source, java]
|
||||
----
|
||||
@Bean
|
||||
@StepScope
|
||||
public ItemReader sampleReader(@Value("#{jobParameters[input.file.name]}") String resource) {
|
||||
...
|
||||
FlatFileItemReader flatFileItemReader = new FlatFileItemReader();
|
||||
flatFileItemReader.setResource(new FileSystemResource(resource));
|
||||
...
|
||||
return flatFileItemReader;
|
||||
}
|
||||
----
|
||||
|
||||
XML::
|
||||
+
|
||||
The following XML example shows the necessary bean configuration:
|
||||
+
|
||||
.XML Configuration
|
||||
[source, xml]
|
||||
----
|
||||
<bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
|
||||
scope="step">
|
||||
<property name="resource" value="file://#{jobParameters['input.file.name']}"/>
|
||||
...
|
||||
</bean>
|
||||
----
|
||||
|
||||
====
|
||||
|
||||
The main points of interest in the preceding example are injecting the value of
|
||||
`#{jobParameters['input.file.name']}`
|
||||
as the Resource property value and setting the `ItemReader` bean
|
||||
to have step scope. Setting the bean to have step scope takes advantage of
|
||||
the late binding support, which allows access to the
|
||||
`jobParameters` variable.
|
||||
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
[[namespace-support]]
|
||||
= Namespace Support
|
||||
|
||||
Dedicated XML namespace support was added to Spring Batch Integration in version 1.3,
|
||||
with the aim to provide an easier configuration
|
||||
experience. To use the namespace, add the following
|
||||
namespace declarations to your Spring XML Application Context
|
||||
file:
|
||||
|
||||
[source, xml]
|
||||
----
|
||||
<beans xmlns="http://www.springframework.org/schema/beans"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xmlns:batch-int="http://www.springframework.org/schema/batch-integration"
|
||||
xsi:schemaLocation="
|
||||
http://www.springframework.org/schema/batch-integration
|
||||
https://www.springframework.org/schema/batch-integration/spring-batch-integration.xsd">
|
||||
|
||||
...
|
||||
|
||||
</beans>
|
||||
----
|
||||
|
||||
The following example shows a fully configured Spring XML application context file for Spring
|
||||
Batch Integration:
|
||||
|
||||
[source, xml]
|
||||
----
|
||||
<beans xmlns="http://www.springframework.org/schema/beans"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xmlns:int="http://www.springframework.org/schema/integration"
|
||||
xmlns:batch="http://www.springframework.org/schema/batch"
|
||||
xmlns:batch-int="http://www.springframework.org/schema/batch-integration"
|
||||
xsi:schemaLocation="
|
||||
http://www.springframework.org/schema/batch-integration
|
||||
https://www.springframework.org/schema/batch-integration/spring-batch-integration.xsd
|
||||
http://www.springframework.org/schema/batch
|
||||
https://www.springframework.org/schema/batch/spring-batch.xsd
|
||||
http://www.springframework.org/schema/beans
|
||||
https://www.springframework.org/schema/beans/spring-beans.xsd
|
||||
http://www.springframework.org/schema/integration
|
||||
https://www.springframework.org/schema/integration/spring-integration.xsd">
|
||||
|
||||
...
|
||||
|
||||
</beans>
|
||||
----
|
||||
|
||||
Appending version numbers to the referenced XSD file is also
|
||||
allowed. However, because a version-less declaration always uses the
|
||||
latest schema, we generally do not recommend appending the version
|
||||
number to the XSD name. Adding a version number
|
||||
could possibly create issues when updating the Spring Batch
|
||||
Integration dependencies, as they may require more recent versions
|
||||
of the XML schema.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user