99 lines
3.5 KiB
Plaintext
99 lines
3.5 KiB
Plaintext
= Partitioned Job
|
|
|
|
An example of the usage of the `DeployerPartitionHandler` and
|
|
`DeployerStepExecutionHandler` to partition a Spring Batch job.
|
|
|
|
== Requirements:
|
|
|
|
* Java 17 or Above
|
|
|
|
== Build:
|
|
|
|
[source,shell]
|
|
----
|
|
./mvnw clean install
|
|
----
|
|
|
|
== Execute:
|
|
|
|
[source,shell]
|
|
----
|
|
export SPRING_APPLICATION_JSON='{"spring.datasource.url":"jdbc:mariadb://localhost:3306/<your database>","spring.datasource.password":"<your password>","spring.datasource.username":"<your username>","spring.datasource.driverClassName":"org.mariadb.jdbc.Driver"}'
|
|
java -jar target/partitioned-batch-job-3.0.0.jar
|
|
----
|
|
|
|
NOTE: This example will use require a MySql RDBMS repository and currently uses the mariadb jdbc driver to connect.
|
|
You can changes this another driver based on your needs.
|
|
|
|
NOTE: Since this example uses the Spring Cloud Deployer Local to launch the partitions, you will need to establish the datasource settings using the SPRING_APPLICATION_JSON environment property as shown above.
|
|
|
|
== Dependencies:
|
|
|
|
A datasource (not in memory) must be configured based on normal Spring Boot conventions
|
|
(application.properties, application.yml, environment variables).
|
|
|
|
== Asynchronous remote partition task launch
|
|
Currently partitions are launched sequentially. To launch them asynchronously set the following environment variables:
|
|
|
|
* `spring.cloud.task.closecontextEnabled=true`
|
|
* `io.spring.asynchronous=true`
|
|
|
|
NOTE: We need to close the context since the use of ThreadPoolTaskExecutor leaves a thread active thus the app will not terminate.
|
|
To close the application appropriately, we will need to set `spring.cloud.task.closecontextEnabled`` to true.
|
|
|
|
== Running Partitioned sample on Kubernetes
|
|
The current sample is set up to run the partitioning using the `local` deployer, so to change this sample so that it work with the `kubernetes` deployer.
|
|
|
|
=== Changes to the application in order to partition on Kubernetes
|
|
. Replace the `spring-cloud-deployer-local` dependency with `spring-cloud-deployer-kubernetes` in the pom.xml. For example:
|
|
+
|
|
[source,xml]
|
|
----
|
|
# Replace
|
|
<dependency>
|
|
<groupId>org.springframework.cloud</groupId>
|
|
<artifactId>spring-cloud-deployer-local</artifactId>
|
|
<version>${spring-cloud-deployer}</version>
|
|
</dependency>
|
|
# With
|
|
<dependency>
|
|
<groupId>org.springframework.cloud</groupId>
|
|
<artifactId>spring-cloud-deployer-kubernetes</artifactId>
|
|
<version>${spring-cloud-deployer}</version>
|
|
</dependency>
|
|
----
|
|
|
|
. Add the following dependency so that we can use docker as a resource:
|
|
+
|
|
[source,xml]
|
|
----
|
|
<dependency>
|
|
<groupId>org.springframework.cloud</groupId>
|
|
<artifactId>spring-cloud-deployer-resource-docker</artifactId>
|
|
<version>${spring-cloud-deployer}</version>
|
|
</dependency>
|
|
----
|
|
. Replace the resource that is used to retrieve the jar from maven with the docker resource to obtain the docker image as follows:
|
|
+
|
|
[source,java]
|
|
----
|
|
// Replace
|
|
Resource resource = this.resourceLoader
|
|
.getResource("maven://io.spring.cloud:partitioned-batch-job:3.0.0-SNAPSHOT");
|
|
// With
|
|
Resource resource = new DockerResource("partitioned-batch-job:3.0.0-SNAPSHOT");
|
|
----
|
|
. Build a docker image using the following command:
|
|
+
|
|
[source,bash]
|
|
----
|
|
./mvnw spring-boot:build-image
|
|
----
|
|
|
|
=== Notes on when launching this application on Kubernetes
|
|
|
|
* Be sure to make sure the application has the proper permissions to launch another pod. For example in Spring Cloud Data Flow be sure to set the `deployment-service-account-name` ot `scdf-sa`.
|
|
* When using Spring Cloud Data Flow be sure to set the `entry-point-style` to `boot`.
|
|
|
|
|