diff --git a/spring-batch-core/src/site/apt/index.apt b/spring-batch-core/src/site/apt/index.apt index fbbeddf8e..4c2e8e800 100644 --- a/spring-batch-core/src/site/apt/index.apt +++ b/spring-batch-core/src/site/apt/index.apt @@ -14,19 +14,17 @@ Overview of the Spring Batch Core dependencies to infrastructure indicated schematically. The figure above shows the central parts of the core domain and its - main touch points with the batch application develepor - (<<<*Configuration>>>). To launch a job there is a - <<>> interface and a facade for it that can be used to - simplify the launching for dumb clients like JMX or a command line. + main touch points with the batch application develepor (<<>> + and <<>>). To launch a job there is a <<>> + interface that can be used to simplify the launching for dumb + clients like JMX or a command line. - A <<>> is composed of a list of - <<>>s, each of which is executed in turn by the - <<>>, delegating to a <<>>. The - <<>> is a central strategy in the Spring Batch Core. - Implementations of <<>> are responsible for sharing - the work specified by the <<>> out, but in ways - that the configuration doesn't need to be aware of. For instance, - the same <<>> might be used in a simple + A <<>> is composed of a list of <<>>s, each of which is + executed in turn by the <<>>. The <<>> is a central + strategy in the Spring Batch Core. Implementations of <<>> + are responsible for sharing the work out, but in ways that the + configuration doesn't need to be aware of. For instance, the same + or very similar <<>> configuration might be used in a simple in-process sequential executor, or in a multi-threaded implementation, or one that delegates to remote calls to a distributed system. @@ -46,10 +44,10 @@ extended to include the datababase entities and identifier strategy. repository (usually a database). A new <<>> is also created to track the progress of this particular execution. - * If the job was previously launched the <<>> - has a flag indicating whether or not to continue and launch a new - execution (was this expected?). The decision is parameterised to - depend on whether or not the job failed last time it was executed. + * If the job was previously launched the <<>> has a flag + indicating whether or not to continue and launch a new execution + (was this expected?). The decision depends on whether or not the + job failed last time it was executed. If the there was a previous failure - maybe the operator has fixed some bad input and wants to run it again - then we might want to @@ -58,13 +56,14 @@ extended to include the datababase entities and identifier strategy. case a new <<>> is created and stored to monitor this execution of the <<>>. -Overview of the Spring Batch Simple Batch Execution Container +Overview of the Spring Batch Launch Environment - The diagram below provides an overview of the high level components, technical services, and basic operations required by a batch architecture. This architecture framework is a blueprint that has been proven through decades of implementations on the last several generations of platforms (COBOL/Mainframe, C++/Unix, and now Java/anywhere). The Simple Batch Execution Container provides a physical implementation of the layers, components and technical services commonly found in robust, maintainable systems used to address the creation of simple to complex batch applications, with the infrastructure and extensions to address very complex processing needs. The materials below will walk through the details of the diagram. + The diagram below provides an overview of the high level components, technical services, and basic operations required by a batch architecture. This architecture framework is a blueprint that has been proven through decades of implementations on the last several generations of platforms (COBOL/Mainframe, C++/Unix, and now Java/anywhere). Simple Batch provides a physical implementation of the layers, components and technical services commonly found in robust, maintainable systems used to address the creation of simple to complex batch applications, with the infrastructure and extensions to address very complex processing needs. The materials below will walk through the details of the diagram. -[images/simple-batch-execution-container.jpg] Simple Batch Execution Container high level flow and interaction of the architecture. +[images/simple-batch-execution-container.jpg] Simple Batch Launch Environment high level flow and interaction of the architecture. + +* Tiers - Tiers The application style is organized into four logical tiers, which include Run, Job, Application, and Data tiers. The primary goal for organizing an application according to the tiers is to embed what is known as "separation of concerns" within the system. Effective separation of concerns results in reducing the impact of change to the system. * <> The Run Tier is concerned with the scheduling and launching of the application. A vendor product is typically used in this tier to allow time-based and interdependent scheduling of batch jobs as well as providing parallel processing capabilities. @@ -73,42 +72,4 @@ Overview of the Spring Batch Simple Batch Execution Container * <> The Application Tier contains components required to execute the program. It contains specific modules that address the required batch functionality and enforces policies around a module execution (e.g., commit intervals, capture of statistics, etc.) - * <> The Data Tier provides the integration with the physical data sources that might include databases, files, or queues. <>: In some cases the Job tier can be completely missing and in other cases one Job Script can start several Batch Job instances. - -High Level Processing Flow - - The diagram above illustrates the flow and architecture components in a typical batch run execution. - - Standard interaction is described as follows: - - <<1.>> In the Run tier, a Scheduler starts a batch application by invoking a Job Script. The Scheduler identifies what batch process it wants to run by passing the name of the batch process and any required additional parameters to the Job Script. - - <<2.>> The Job Script initializes the program and executes any job specific scripts prior to calling the Batch Launcher. - - <<3.>> The Batch Launcher starts the Batch Execution Container based upon any environment settings established in the script. (NOTE: A Batch Execution Container is not a Java EE container) - - <<3.1>> The Batch Container starts and controls the batch execution. It initializes the Job execution environment with static configuration items such as database settings, logging levels and creates a Job based on the Job Configuration created by a Batch Developer. - - <<4>> Based on configuration provided by a Batch Developer, the Job sequentially executes steps after checking policies to ensure that each step should be started. The status of the job and step (start time, end time, status such as "started" or "completed") is stored at various points during the process. - - <<5.1>> In order to maintain data integrity, at the application tier, the Step acts as a controller to ensure that either an entire group of actions completes successfully or that none of the actions completes. This group of actions is referred to as a logical unit of work (LUW). The Step controls the overall execution of the Tasklet, ensuring that transaction are committed at the appropriate time, and restart and statistics information is stored appropriately. The first thing the Step is responsible for is the initialization of the data required to begin processing. The Step will interact with other architecture components, such as the Input Source, to setup the data required to be processed. - - <<5.1.1>> The Input Source provides services to access various data sources. It provides location transparency to the Batch Tasklet and hides the physical location details of the data. - - <<5.2>> Once the data is initialized by the Input Source, the Step will call into the Tasklet to begin processing. The Tasklet contains the business logic to define the LUW and the Step repeatedly calls the Tasklets LUW to finish the business function. The Step does this by first invoking the execute method on the Tasklet in order to acquire a single record/set of data for processing. - - <<5.2.1>> Before a record is returned to the Tasklet, it may be validated by any number of validation Frameworks that can be provided to an input source. A single record/set of data is gathered by interacting with the Input Source. - - <<5.3>> Once a record/set has been obtained, the step calls the module to begin processing. - - <<5.3.1>> The Tasklet executes its internal business logic by calling other Business Logic components as necessary. Based on the business service, it can requests or persists objects from the data access components. - - <<5.3.3>> Data Access components can be leveraged retrieve or persist domain objects. - - <<5.3.4>> Once the business logic has been executed, the resulting output record is written out by utilizing the Output Source interface. The Step will repeatedly call steps 4.2 \-> 4.4 for every record provided by the Input Source. - - <<5.4>> Once all of the records are processed, the Step calls the Tasklet to perform any clean up activities such as closing connections, exporting files, etc. - - <<5.4.1>> The Step is responsible for committing data associated with the remaining logical units of work as well as performing any finalization and administrative functions (e.g. closing database connections). - - Once the Step has completed finalization the control is passed back to the Job, where any necessary logging or clean up is executed for application termination and wrap-up -- provided there are no additional Steps to execute. + * <> The Data Tier provides the integration with the physical data sources that might include databases, files, or queues. <>: In some cases the Job tier can be completely missing and in other cases one job script can start several batch job instances. diff --git a/spring-batch-infrastructure/.springBeans b/spring-batch-infrastructure/.springBeans index 133aca434..344753b6b 100644 --- a/spring-batch-infrastructure/.springBeans +++ b/spring-batch-infrastructure/.springBeans @@ -1,7 +1,7 @@ 1 - + diff --git a/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java b/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java index d74cba027..600507195 100644 --- a/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java +++ b/spring-batch-samples/src/main/java/org/springframework/batch/sample/quartz/JobLauncherDetails.java @@ -41,7 +41,7 @@ public class JobLauncherDetails extends QuartzJobBean { /** * Special key in job data map for the name of a job to run. */ - private static final String JOB_NAME = "jobName"; + static final String JOB_NAME = "jobName"; private static Log log = LogFactory.getLog(JobLauncherDetails.class); @@ -93,7 +93,6 @@ public class JobLauncherDetails extends QuartzJobBean { private JobParameters getJobParametersFromJobMap(Map jobDataMap) { JobParametersBuilder builder = new JobParametersBuilder(); - JobParameters jobParameters = builder.toJobParameters(); for (Iterator iterator = jobDataMap.entrySet().iterator(); iterator.hasNext();) { Entry entry = (Entry) iterator.next(); @@ -116,7 +115,7 @@ public class JobLauncherDetails extends QuartzJobBean { } } - return jobParameters; + return builder.toJobParameters(); } diff --git a/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java b/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java new file mode 100644 index 000000000..ceeffa042 --- /dev/null +++ b/spring-batch-samples/src/test/java/org/springframework/batch/sample/quartz/JobLauncherDetailsTests.java @@ -0,0 +1,162 @@ +/* + * Copyright 2006-2007 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.springframework.batch.sample.quartz; + +import java.util.ArrayList; +import java.util.Date; +import java.util.List; + +import junit.framework.TestCase; + +import org.easymock.MockControl; +import org.quartz.Job; +import org.quartz.JobDetail; +import org.quartz.JobExecutionContext; +import org.quartz.Scheduler; +import org.quartz.SimpleTrigger; +import org.quartz.spi.TriggerFiredBundle; +import org.springframework.batch.core.JobExecution; +import org.springframework.batch.core.JobParameters; +import org.springframework.batch.core.JobSupport; +import org.springframework.batch.core.configuration.JobLocator; +import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; +import org.springframework.batch.core.repository.JobRestartException; +import org.springframework.batch.core.repository.NoSuchJobException; +import org.springframework.batch.execution.launch.JobLauncher; + +/** + * @author Dave Syer + * + */ +public class JobLauncherDetailsTests extends TestCase { + + private JobLauncherDetails details = new JobLauncherDetails(); + + private TriggerFiredBundle firedBundle; + + private List list = new ArrayList(); + + protected void setUp() throws Exception { + details.setJobLauncher(new JobLauncher() { + public JobExecution run(org.springframework.batch.core.Job job, JobParameters jobParameters) + throws JobExecutionAlreadyRunningException, JobRestartException { + list.add(jobParameters); + return null; + } + }); + details.setJobLocator(new JobLocator() { + public org.springframework.batch.core.Job getJob(String name) throws NoSuchJobException { + list.add(name); + return new JobSupport("foo"); + } + }); + } + + /** + * @return + * + */ + private JobExecutionContext createContext(JobDetail jobDetail) { + firedBundle = new TriggerFiredBundle(jobDetail, new SimpleTrigger(), null, false, new Date(), new Date(), new Date(), new Date()); + return new StubJobExecutionContext(); + } + + /** + * Test method for + * {@link org.springframework.batch.sample.quartz.JobLauncherDetails#executeInternal(org.quartz.JobExecutionContext)}. + */ + public void testExecuteWithNoJobParameters() { + JobDetail jobDetail = new JobDetail(); + JobExecutionContext context = createContext(jobDetail); + details.executeInternal(context); + assertEquals(2, list.size()); + JobParameters parameters = (JobParameters) list.get(1); + assertEquals(0, parameters.getParameters().size()); + } + + /** + * Test method for + * {@link org.springframework.batch.sample.quartz.JobLauncherDetails#executeInternal(org.quartz.JobExecutionContext)}. + */ + public void testExecuteWithJobName() { + JobDetail jobDetail = new JobDetail(); + jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); + JobExecutionContext context = createContext(jobDetail); + details.executeInternal(context); + assertEquals(2, list.size()); + assertEquals("FOO", list.get(0)); + } + + /** + * Test method for + * {@link org.springframework.batch.sample.quartz.JobLauncherDetails#executeInternal(org.quartz.JobExecutionContext)}. + */ + public void testExecuteWithSomeJobParameters() { + JobDetail jobDetail = new JobDetail(); + jobDetail.getJobDataMap().put("foo", "bar"); + JobExecutionContext context = createContext(jobDetail); + details.executeInternal(context); + assertEquals(2, list.size()); + JobParameters parameters = (JobParameters) list.get(1); + assertEquals(1, parameters.getParameters().size()); + } + + /** + * Test method for + * {@link org.springframework.batch.sample.quartz.JobLauncherDetails#executeInternal(org.quartz.JobExecutionContext)}. + */ + public void testExecuteWithJobNameAndParameters() { + JobDetail jobDetail = new JobDetail(); + jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); + jobDetail.getJobDataMap().put("foo", "bar"); + JobExecutionContext context = createContext(jobDetail); + details.executeInternal(context); + assertEquals(2, list.size()); + assertEquals("FOO", list.get(0)); + JobParameters parameters = (JobParameters) list.get(1); + assertEquals(1, parameters.getParameters().size()); + } + + /** + * Test method for + * {@link org.springframework.batch.sample.quartz.JobLauncherDetails#executeInternal(org.quartz.JobExecutionContext)}. + */ + public void testExecuteWithJobNameAndComplexParameters() { + JobDetail jobDetail = new JobDetail(); + jobDetail.getJobDataMap().put(JobLauncherDetails.JOB_NAME, "FOO"); + jobDetail.getJobDataMap().put("foo", this); + JobExecutionContext context = createContext(jobDetail); + details.executeInternal(context); + assertEquals(2, list.size()); + assertEquals("FOO", list.get(0)); + JobParameters parameters = (JobParameters) list.get(1); + // Silently ignore parameters that are not simple types + assertEquals(0, parameters.getParameters().size()); + } + + private final class StubJobExecutionContext extends JobExecutionContext { + /** + * @param scheduler + * @param firedBundle + * @param job + */ + private StubJobExecutionContext() { + super((Scheduler) MockControl.createNiceControl(Scheduler.class).getMock(), firedBundle, (Job) MockControl.createNiceControl(Job.class) + .getMock()); + } + } + +}