Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4137: Improve Aggregator Performance

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4137

Aggregators with `SequenceSizeReleaseStrategy` do not perform well with large groups
because of an O(n) linear search to reject (discard) duplicate sequences.

Change the aggregator to use a `SimpleSequenceSizeReleaseStrategy` by default, unless
`setReleasePartialSequences(true)`.

This avoids the linear search, which is not needed for most splitter -> ... -> aggregator
flows.

Log a warning if SSRS is used.

Polishing - PR Comments

* Remove `this.logger.isWarnEnabled()` since it is redundant when our log message is just string constant
This commit is contained in:
Gary Russell
2016-10-10 16:14:31 -04:00
committed by Artem Bilan
parent 8d94bd5e3e
commit 890ad63584
10 changed files with 129 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
src/reference/asciidoc/aggregator.adoc
View File

@@ -35,7 +35,7 @@ The Aggregation API consists of a number of classes:
* The interface `MessageGroupProcessor`, and its subclasses:`MethodInvokingAggregatingMessageGroupProcessor` and `ExpressionEvaluatingMessageGroupProcessor`
* The `ReleaseStrategy` interface and its default implementation `SequenceSizeReleaseStrategy`
* The `ReleaseStrategy` interface and its default implementation `SimpleSequenceSizeReleaseStrategy`
* The `CorrelationStrategy` interface and its default implementation `HeaderAttributeCorrelationStrategy`
@@ -85,7 +85,7 @@ public AbstractCorrelatingMessageHandler(MessageGroupProcessor processor, Messag
...
this.correlationStrategy = correlationStrategy == null ?
new HeaderAttributeCorrelationStrategy(IntegrationMessageHeaderAccessor.CORRELATION_ID) : correlationStrategy;
this.releaseStrategy = releaseStrategy == null ? new SequenceSizeReleaseStrategy() : releaseStrategy;
this.releaseStrategy = releaseStrategy == null ? new SimpleSequenceSizeReleaseStrategy() : releaseStrategy;
...
}
----
@@ -214,10 +214,13 @@ This can eventually cause out of memory conditions.
To avoid such situations, you should consider configuring a `MessageGroupStoreReaper` to remove the group metadata; the expiry parameters should be set to expire groups after it is not expected that late messages will arrive.
For information about configuring a reaper, see <<reaper>>.
Spring Integration provides an out-of-the box implementation for `ReleaseStrategy`, the `SequenceSizeReleaseStrategy`.
Spring Integration provides an out-of-the box implementation for `ReleaseStrategy`, the `SimpleSequenceSizeReleaseStrategy`.
This implementation consults the `SEQUENCE_NUMBER` and `SEQUENCE_SIZE` headers of each arriving message to decide when a message group is complete and ready to be aggregated.
As shown above, it is also the default strategy.
NOTE: Before _version 5.0_, the default release strategy was `SequenceSizeReleaseStrategy` which does not perform well with large groups.
With that strategy, duplicate sequence numbers are detected and rejected; this operation can be expensive.
===== CorrelationStrategy
The `CorrelationStrategy` interface is defined as follows:
@@ -263,7 +266,7 @@ Below you can see an example of an aggregator.
----
<channel id="inputChannel"/>
<int:aggregator id=""myAggregator" <1>
<int:aggregator id="myAggregator" <1>
auto-startup="true" <2>
input-channel="inputChannel" <3>
output-channel="outputChannel" <4>
@@ -680,7 +683,7 @@ Must be specified if this class will be used as an aggregator.
<2> An annotation indicating that this method shall be used as the release strategy of an aggregator.
If not present on any method, the aggregator will use the SequenceSizeReleaseStrategy.
If not present on any method, the aggregator will use the `SimpleSequenceSizeReleaseStrategy`.

3
src/reference/asciidoc/resequencer.adoc
View File

@@ -13,6 +13,9 @@ It simply releases them in the order of their `SEQUENCE_NUMBER` header values.
With respect to that, the user might opt to release all messages at once (after the whole sequence, according to the `SEQUENCE_SIZE`, has been released), or as soon as a valid sequence is available.
IMPORTANT: The resequencer is intended to resequence relatively short sequences of messages with small gaps.
If you have a large number of disjoint sequences with many gaps, you may experience performance issues.
==== Configuring a Resequencer
Configuring a resequencer requires only including the appropriate element in XML.

6
src/reference/asciidoc/whats-new.adoc
View File

@@ -65,6 +65,7 @@ See <<barrier>> for more information.
The AMQP outbound endpoints now support setting a delay expression for when using the RabbitMQ Delayed Message Exchange plugin.
See <<amqp-delay>> for more information.
The inbound endpoints now support the Spring AMQP `DirectMessageListenerContainer`.
See <<amqp-inbound-channel-adapter>> for more information.
@@ -72,3 +73,8 @@ See <<amqp-inbound-channel-adapter>> for more information.
The `DefaultHttpHeaderMapper.userDefinedHeaderPrefix` property is now an empty string by default instead of `X-`.
See <<http-header-mapping>> for more information.
==== Aggregator Performance Changes
Aggregators now use a `SimpleSequenceSizeReleaseStrategy` by default, which is more efficient, especially with large groups.
See <<aggregator>> for more information.