SpringCloud/spring-cloud-task
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Support asynchronous launch of partitions.

Browse Source 
Resolves #785

Updated to allow user to set ThreadPoolTaskExecutor.

updated based on code review
This commit is contained in:
Glenn Renfro
2022-02-23 17:31:59 -05:00
parent 362d73650b
commit 2a4a066f5f
9 changed files with 378 additions and 134 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/src/main/asciidoc/batch.adoc
View File

@@ -62,7 +62,7 @@ most cloud infrastructures. The `DeployerPartitionHandler` and
Cloud Deployer.
To configure the `DeployerStepExecutionHandler`, you must provide a `Resource`
representing the Spring Boot über-jar to be executed, a `TaskLauncher`, and a
representing the Spring Boot über-jar to be executed, a `TaskLauncherHandler`, and a
`JobExplorer`. You can configure any environment properties as well as the max number of
workers to be executing at once, the interval to poll for the results (defaults to 10
seconds), and a timeout (defaults to -1 or no timeout). The following example shows how
@@ -135,6 +135,40 @@ NOTE: You can find a sample remote partition application in the samples module o
Spring Cloud Task project,
https://github.com/spring-cloud/spring-cloud-task/tree/master/spring-cloud-task-samples/partitioned-batch-job[here].
=== Asynchronously launch remote batch partitions
By default batch partitions are launched sequentially. However, in some cases this may affect performance as each launch will block until the resource (For example: provisioning a pod in Kubernetes) is provisioned.
In these cases you can provide a `ThreadPoolTaskExecutor` to the `DeployerPartitionHandler`. This will launch the remote batch partitions based on the configuration of the `ThreadPoolTaskExecutor`.
For example:
[source,java]
----
@Bean
public ThreadPoolTaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setThreadNamePrefix("default_task_executor_thread");
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.initialize();
return executor;
}
@Bean
public PartitionHandler partitionHandler(TaskLauncher taskLauncher, JobExplorer jobExplorer,
TaskRepository taskRepository, ThreadPoolTaskExecutor executor) throws Exception {
Resource resource = this.resourceLoader
.getResource("maven://io.spring.cloud:partitioned-batch-job:2.2.0.BUILD-SNAPSHOT");
DeployerPartitionHandler partitionHandler =
new DeployerPartitionHandler(taskLauncher, jobExplorer, resource,
"workerStep", taskRepository, executor);
...
}
----
NOTE: We need to close the context since the use of `ThreadPoolTaskExecutor` leaves a thread active thus the app will not terminate. To close the application appropriately, we will need to set `spring.cloud.task.closecontextEnabled` property to `true`.
=== Notes on Developing a Batch-partitioned application for the Kubernetes Platform
* When deploying partitioned apps on the Kubernetes platform, you must use the following

4
docs/src/main/asciidoc/features.adoc
View File

@@ -190,9 +190,7 @@ following property:
=== External Task Id
Spring Cloud Task lets you store an external task ID for each
`TaskExecution`. An example of this would be a task ID provided by
Cloud Foundry when a task is launched on the platform.
In order to configure your Task to use a generated `TaskExecutionId`, add the
`TaskExecution`. In order to configure your Task to use a generated `TaskExecutionId`, add the
following property:
`spring.cloud.task.external-execution-id=<externalTaskId>`