Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

GH-3446: Stream support in the MessageGroupStore

Browse Source 
Fixes https://github.com/spring-projects/spring-integration/issues/3446

* For better resources utilization provide a `Stream<Message<?>>` API
on the `MessageGroupStore`, `MessageGroup` and `MessageGroupQueue`
* Use this API in the `DelayHandler` when it reschedules persisted messages
This commit is contained in:
Artem Bilan
2021-01-19 18:40:10 -05:00
committed by Gary Russell
parent 51e240b761
commit f0f2c41ae3
12 changed files with 157 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
src/reference/asciidoc/message-store.adoc
View File

@@ -100,12 +100,9 @@ For this reason, you should either not perform such manipulation or set the `cop
[[message-group-factory]]
==== Using `MessageGroupFactory`
Starting with version 4.3, some `MessageGroupStore` implementations can be injected with a custom
`MessageGroupFactory` strategy to create and customize the `MessageGroup` instances used by the `MessageGroupStore`.
This defaults to a `SimpleMessageGroupFactory`, which produces `SimpleMessageGroup` instances based on the `GroupType.HASH_SET`
(`LinkedHashSet`) internal collection.
Other possible options are `SYNCHRONISED_SET` and `BLOCKING_QUEUE`, where the last one can be used to reinstate the
previous `SimpleMessageGroup` behavior.
Starting with version 4.3, some `MessageGroupStore` implementations can be injected with a custom `MessageGroupFactory` strategy to create and customize the `MessageGroup` instances used by the `MessageGroupStore`.
This defaults to a `SimpleMessageGroupFactory`, which produces `SimpleMessageGroup` instances based on the `GroupType.HASH_SET` (`LinkedHashSet`) internal collection.
Other possible options are `SYNCHRONISED_SET` and `BLOCKING_QUEUE`, where the last one can be used to reinstate the previous `SimpleMessageGroup` behavior.
Also the `PERSISTENT` option is available.
See the next section for more information.
Starting with version 5.0.1, the `LIST` option is also available for when the order and uniqueness of messages in the group does not matter.
@@ -113,16 +110,12 @@ Starting with version 5.0.1, the `LIST` option is also available for when the or
[[lazy-load-message-group]]
==== Persistent `MessageGroupStore` and Lazy-load
Starting with version 4.3, all persistent `MessageGroupStore` instances retrieve `MessageGroup` instances and their `messages`
from the store in the lazy-load manner.
In most cases, it is useful for the correlation `MessageHandler` instances (see <<./aggregator.adoc#aggregator,Aggregator>> and <<./resequencer.adoc#resequencer,Resequencer>>),
when it would add overhead to load entire the `MessageGroup` from the store on each correlation operation.
Starting with version 4.3, all persistent `MessageGroupStore` instances retrieve `MessageGroup` instances and their `messages` from the store in the lazy-load manner.
In most cases, it is useful for the correlation `MessageHandler` instances (see <<./aggregator.adoc#aggregator,Aggregator>> and <<./resequencer.adoc#resequencer,Resequencer>>), when it would add overhead to load entire the `MessageGroup` from the store on each correlation operation.
You can use the `AbstractMessageGroupStore.setLazyLoadMessageGroups(false)` option to switch off the lazy-load behavior from the configuration.
Our performance tests for lazy-load on MongoDB `MessageStore` (<<./mongodb.adoc#mongodb-message-store,MongoDB Message Store>>) and
`<aggregator>` (<<./aggregator.adoc#aggregator,Aggregator>>)
use a custom `release-strategy` similar to the following:
Our performance tests for lazy-load on MongoDB `MessageStore` (<<./mongodb.adoc#mongodb-message-store,MongoDB Message Store>>) and `<aggregator>` (<<./aggregator.adoc#aggregator,Aggregator>>) use a custom `release-strategy` similar to the following:
====
[source,xml]
@@ -149,3 +142,9 @@ ms % Task name
...
----
====
However starting with version 5.5, all the persistent `MessageGroupStore` implementations provide a `streamMessagesForGroup(Object groupId)` contract based on the target database streaming API.
This improves resources utilization when groups are very big in the store.
Internally in the framework this new API is used in the <<./delayer.adoc#delayer,Delayer>> (for example) when it reschedules persisted messages on startup.
A returned `Stream<Message<?>>` must be closed in the end of processing, e.g. via auto-close by the `try-with-resources`.
Whenever a `PersistentMessageGroup` is used, its `streamMessages()` delegates to the `MessageGroupStore.streamMessagesForGroup()`.

3
src/reference/asciidoc/whats-new.adoc
View File

@@ -18,6 +18,9 @@ If you are interested in more details, see the Issue Tracker tickets that were r
[[x5.5-general]]
=== General Changes
All the persistent `MessageGroupStore` implementation provide a `streamMessagesForGroup(Object groupId)` contract based on the target database streaming API.
See <<./message-store.adoc#message-store,Message Store>> for more information.
[[x5.5-amqp]]
==== AMQP Changes