INT-3387: MessageGroupStore Improvements

JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806

* Introduce
```
MessageGroupStore

void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.

* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality

`GroupType.PERSISTENT` and not lazy by default

PR Comments

Fix `JdbcMessageStoreTests` timing issues

Address PR comments

* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
This commit is contained in:
Artem Bilan
2016-04-12 13:38:21 -04:00
committed by Gary Russell
parent 4e4763d24f
commit 286c421c1a
34 changed files with 1085 additions and 505 deletions

View File

@@ -94,3 +94,40 @@ This defaults to a `SimpleMessageGroupFactory` which produces `SimpleMessageGrou
(`LinkedHashSet`) internal collection.
Other possible options are `SYNCHRONISED_SET` and `BLOCKING_QUEUE`, where the last one can be used to reinstate the
previous `SimpleMessageGroup` behavior.
Also the `PERSISTENT` option is available. See the next section for more information.
[[lazy-load-message-group]]
==== Persistence MessageGroupStore and Lazy-Load
Starting with _version 4.3_, all persistence `MessageGroupStore` s retrieve `MessageGroup` s and their `messages`
from the store with the _Lazy-Load_ manner.
In most cases it is useful for the Correlation `MessageHandler` s (<<aggregator>> and <<resequencer>>),
when it would be an overhead to load entire `MessageGroup` from the store on each correlation operation.
To switch off the lazy-load behavior the `AbstractMessageGroupStore.setLazyLoadMessageGroups(false)` option
can be used from the configuration.
Our performance tests for _lazy-load_ on MongoDB `MessageStore` (<<mongodb-message-store>>) and
`<aggregator>` (<<aggregator>>)
with custom `release-strategy` like:
[source,xml]
----
<int:aggregator input-channel="inputChannel"
output-channel="outputChannel"
message-store="mongoStore"
release-strategy-expression="size() == 1000"/>
----
demonstrate this results for 1000 simple messages:
....
StopWatch 'Lazy-Load Performance': running time (millis) = 38918
-----------------------------------------
ms % Task name
-----------------------------------------
02652 007% Lazy-Load
36266 093% Eager
....

View File

@@ -21,6 +21,11 @@ The `SimpleMessageGroupFactory` is provided for the `SimpleMessageGroup` with th
factory for the standard `MessageGroupStore` implementations.
See <<message-store>> for more information.
==== PersistentMessageGroup
The `PersistentMessageGroup`, - lazy-load proxy, - implementation is provided for persistent `MessageGroupStore` s,
which return this instance for the `getMessageGroup()` when their `lazyLoadMessageGroups` is `true` (defaults).
See <<message-store>> for more information.
[[x4.3-general]]
=== General Changes