117 lines
6.4 KiB
Plaintext
117 lines
6.4 KiB
Plaintext
[[spring-batch-intro]]
|
|
= Spring Batch Introduction
|
|
|
|
Many applications within the enterprise domain require bulk processing to perform
|
|
business operations in mission-critical environments. These business operations include:
|
|
|
|
* Automated, complex processing of large volumes of information that is most efficiently
|
|
processed without user interaction. These operations typically include time-based events
|
|
(such as month-end calculations, notices, or correspondence).
|
|
* Periodic application of complex business rules processed repetitively across very large
|
|
data sets (for example, insurance benefit determination or rate adjustments).
|
|
* Integration of information that is received from internal and external systems that
|
|
typically requires formatting, validation, and processing in a transactional manner into
|
|
the system of record. Batch processing is used to process billions of transactions every
|
|
day for enterprises.
|
|
|
|
Spring Batch is a lightweight, comprehensive batch framework designed to enable the
|
|
development of robust batch applications that are vital for the daily operations of enterprise
|
|
systems. Spring Batch builds upon the characteristics of the Spring Framework that people
|
|
have come to expect (productivity, POJO-based development approach, and general ease of
|
|
use), while making it easy for developers to access and use more advanced enterprise
|
|
services when necessary. Spring Batch is not a scheduling framework. There are many good
|
|
enterprise schedulers (such as Quartz, Tivoli, Control-M, and others) available in both the
|
|
commercial and open source spaces. Spring Batch is intended to work in conjunction with a
|
|
scheduler rather than replace a scheduler.
|
|
|
|
Spring Batch provides reusable functions that are essential in processing large volumes
|
|
of records, including logging and tracing, transaction management, job processing statistics,
|
|
job restart, skip, and resource management. It also provides more advanced technical
|
|
services and features that enable extremely high-volume and high performance batch jobs
|
|
through optimization and partitioning techniques. You can use Spring Batch in both simple
|
|
use cases (such as reading a file into a database or running a stored procedure) and
|
|
complex, high volume use cases (such as moving high volumes of data between databases,
|
|
transforming it, and so on). High-volume batch jobs can use the framework in a
|
|
highly scalable manner to process significant volumes of information.
|
|
|
|
[[springBatchBackground]]
|
|
== Background
|
|
|
|
While open source software projects and associated communities have focused greater
|
|
attention on web-based and microservices-based architecture frameworks, there has been a
|
|
notable lack of focus on reusable architecture frameworks to accommodate Java-based batch
|
|
processing needs, despite continued needs to handle such processing within enterprise IT
|
|
environments. The lack of a standard, reusable batch architecture has resulted in the
|
|
proliferation of many one-off, in-house solutions developed within client enterprise IT
|
|
functions.
|
|
|
|
SpringSource (now VMware) and Accenture collaborated to change this. Accenture's
|
|
hands-on industry and technical experience in implementing batch architectures,
|
|
SpringSource's depth of technical experience, and Spring's proven programming model
|
|
together made a natural and powerful partnership to create high-quality, market-relevant
|
|
software aimed at filling an important gap in enterprise Java. Both companies worked with
|
|
a number of clients who were solving similar problems by developing Spring-based batch
|
|
architecture solutions. This input provided some useful additional detail and real-life
|
|
constraints that helped to ensure the solution can be applied to the real-world problems
|
|
posed by clients.
|
|
|
|
Accenture contributed previously proprietary batch processing architecture frameworks to
|
|
the Spring Batch project, along with committer resources to drive support, enhancements,
|
|
and the existing feature set. Accenture's contribution was based upon decades of
|
|
experience in building batch architectures with the last several generations of
|
|
platforms: COBOL on mainframes, C++ on Unix, and, now, Java anywhere.
|
|
|
|
The collaborative effort between Accenture and SpringSource aimed to promote the
|
|
standardization of software processing approaches, frameworks, and tools
|
|
enterprise users can consistently use when creating batch applications. Companies
|
|
and government agencies desiring to deliver standard, proven solutions to their
|
|
enterprise IT environments can benefit from Spring Batch.
|
|
|
|
[[springBatchUsageScenarios]]
|
|
== Usage Scenarios
|
|
|
|
A typical batch program generally:
|
|
|
|
* Reads a large number of records from a database, file, or queue.
|
|
* Processes the data in some fashion.
|
|
* Writes back data in a modified form.
|
|
|
|
Spring Batch automates this basic batch iteration, providing the capability to process
|
|
similar transactions as a set, typically in an offline environment without any user
|
|
interaction. Batch jobs are part of most IT projects, and Spring Batch is the only open
|
|
source framework that provides a robust, enterprise-scale solution.
|
|
|
|
[[business-scenarios]]
|
|
=== Business Scenarios
|
|
|
|
Spring Batch supports the following business scenarios:
|
|
|
|
* Commit batch process periodically.
|
|
* Concurrent batch processing: parallel processing of a job.
|
|
* Staged, enterprise message-driven processing.
|
|
* Massively parallel batch processing.
|
|
* Manual or scheduled restart after failure.
|
|
* Sequential processing of dependent steps (with extensions to workflow-driven batches).
|
|
* Partial processing: skip records (for example, on rollback).
|
|
* Whole-batch transaction, for cases with a small batch size or existing stored
|
|
procedures or scripts.
|
|
|
|
[[technical-objectives]]
|
|
=== Technical Objectives
|
|
|
|
Spring Batch has the following technical objectives:
|
|
|
|
* Let batch developers use the Spring programming model: Concentrate on business logic and
|
|
let the framework take care of the infrastructure.
|
|
* Provide clear separation of concerns between the infrastructure, the batch execution
|
|
environment, and the batch application.
|
|
* Provide common, core execution services as interfaces that all projects can implement.
|
|
* Provide simple and default implementations of the core execution interfaces that can be
|
|
used "`out of the box`".
|
|
* Make it easy to configure, customize, and extend services, by using the Spring framework
|
|
in all layers.
|
|
* All existing core services should be easy to replace or extend, without any impact to
|
|
the infrastructure layer.
|
|
* Provide a simple deployment model, with the architecture JARs completely separate from
|
|
the application, built by using Maven.
|