Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4269: Add Stream & Flux Support for Splitter

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4269

* Check if `outputChannel` is `ReactiveStreamsSubscribableChannel`
to let back-pressure splitting
* Build `Flux` or `Iterator` depending in the `reactive` state
* Allow to get a size of the `iterator` if it is possible,
for example `XPathMessageSplitter`

Add tests

Fix raw type and unused import

* Add JavaDocs to the `AbstractMessageSplitter#obtainSizeIfPossible()`
* Add asserts for the `sequenceSize` populataiton in the `XPathMessageSplitter`
* Document `Stream` & `Flux` support in the splitter

Minor doc polishing
This commit is contained in:
Artem Bilan
2017-05-25 14:21:40 -04:00
committed by Gary Russell
parent a800d9683a
commit f112ecbb7e
8 changed files with 371 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
src/reference/asciidoc/splitter.adoc
View File

@@ -43,17 +43,27 @@ The input argument might either be a `Message` or a simple POJO.
In the latter case, the splitter will receive the payload of the incoming message.
Since this decouples the code from the Spring Integration API and will typically be easier to test, it is the recommended approach.
*Splitter and Iterators*
*Iterators*
Starting with _version 4.1_, the `AbstractMessageSplitter` supports the `Iterator` type for the `value` to split.
Note, in the case of an `Iterator` (or `Iterable`), we don't have access to the number of underlying items and the `SEQUENCE_SIZE` header is set to `0`.
This means that the default `SequenceSizeReleaseStrategy` of an `<aggregator>` won't work and the group for the `CORRELATION_ID` from the `splitter` won't be released; it will remain as `incomplete`.
In this case you should use an appropriate custom `ReleaseStrategy` or rely on `send-partial-result-on-expiry` together with `group-timeout` or a `MessageGroupStoreReaper`.
Starting with _version 5.0_, the `AbstractMessageSplitter` provides `protected obtainSizeIfPossible()` methods to allow the determination of the size of the `Iterable` and `Iterator` objects if that is possible.
For example `XPathMessageSplitter` can determine the size of the underlying `NodeList` object.
An `Iterator` object is useful to avoid the need for building an entire collection in the memory before splitting.
For example, when underlying items are populated from some external system (e.g.
DataBase or FTP `MGET`) using iterations or streams.
*Stream and Flux*
Starting with _version 5.0_, the `AbstractMessageSplitter` supports the Java `Stream` and Reactive Streams `Publisher` types for the `value` to split.
In this case the target `Iterator` is built on their iteration functionality.
In addition, if Splitter's output channel is an instance of a `ReactiveStreamsSubscribableChannel`, the `AbstractMessageSplitter` produces a `Flux` result instead of an `Iterator` and the output channel is _subscribed_ to this `Flux` for back-pressure based splitting on downstream flow demand.
[[splitter-config]]
==== Configuring Splitter

35
src/reference/asciidoc/whats-new.adoc
View File

@@ -72,13 +72,6 @@ See <<transaction-synchronization>> for more information.
The aggregator expression-based `ReleaseStrategy` now evaluates the expression against the `MesageGroup` instead of just the collection of `Message<?>`.
See <<aggregator-spel>> for more information.
==== JMS Changes
Previously, Spring Integration JMS XML configuration used a default bean name `connectionFactory` for the JMS Connection Factory, allowing the property to be omitted from component definitions.
It has now been renamed to `jmsConnectionFactory`, which is the bean name used by Spring Boot to auto-configure the JMS Connection Factory bean.
If your application is relying on the previous behavior, rename your `connectionFactory` bean to `jmsConnectionFactory`, or specifically configure your components to use your bean using its current name.
==== Gateway Changes
The gateway now correctly sets the `errorChannel` header when the gateway method has a `void` return type and an error channel is provided.
@@ -89,6 +82,28 @@ The `RequestReplyExchanger` interface now has a `throws MessagingException` clau
See <<gateway-error-handling>> for more information.
==== Aggregator Performance Changes
Aggregators now use a `SimpleSequenceSizeReleaseStrategy` by default, which is more efficient, especially with large groups.
Empty groups are now scheduled for removal after `empty-group-min-timeout`.
See <<aggregator>> for more information.
==== Splitter Changes
The Splitter component now can handle and split Java `Stream` and Reactive Streams `Publisher` objects.
If the output channel is a `ReactiveStreamsSubscribableChannel`, the `AbstractMessageSplitter` builds a `Flux` for subsequent iteration instead of a regular `Iterator` independent of object being split.
In addition, `AbstractMessageSplitter` provides `protected obtainSizeIfPossible()` methods to allow the determination of the size of the `Iterable` and `Iterator` objects if that is possible.
See <<splitter>> for more information.
==== JMS Changes
Previously, Spring Integration JMS XML configuration used a default bean name `connectionFactory` for the JMS Connection Factory, allowing the property to be omitted from component definitions.
It has now been renamed to `jmsConnectionFactory`, which is the bean name used by Spring Boot to auto-configure the JMS Connection Factory bean.
If your application is relying on the previous behavior, rename your `connectionFactory` bean to `jmsConnectionFactory`, or specifically configure your components to use your bean using its current name.
See <<jms>> for more information.
==== Mail Changes
Some inconsistencies with rendering IMAP mail content have been resolved.
@@ -180,12 +195,6 @@ See <<content-type-conversion-outbound>> for more information.
The `DefaultHttpHeaderMapper.userDefinedHeaderPrefix` property is now an empty string by default instead of `X-`.
See <<http-header-mapping>> for more information.
==== Aggregator Performance Changes
Aggregators now use a `SimpleSequenceSizeReleaseStrategy` by default, which is more efficient, especially with large groups.
Empty groups are now scheduled for removal after `empty-group-min-timeout`.
See <<aggregator>> for more information.
==== MQTT Changes
Inbound messages are now mapped with headers `RECEIVED_TOPIC`, `RECEIVED_QOS` and `RECEIVED_RETAINED` to avoid inadvertent propagation to outbound messages when an application is relaying messages.