The `RedisMessageGroupStoreTests.testMessageGroupUpdatedDateChangesWithEachAddedMessage()` can fail
sporadically because there is no guaranty the several calls to the `System.getCurrentTimeMillis()` return the same value.
* Fix `AbstractKeyValueMessageStore` to reuse `MessageGroup.timestamp` for the `lastModified` in case of a new group
https://build.spring.io/browse/INT-MJATS41-629
* Revert to SF and SA BUILD-SNAPSHOTs
* Upgrade to Spring Security 4.1
* Fix `RedisQueueMessageDrivenEndpointTests.testInt3442ProperlyStop()` for `atLeastOnce()` verification
because in between stop and the last `rightPush` there might be one more `pop` and push again because of `listening = false` state
* There is still sporadic `OutboundGatewayFunctionTests.testLazyContainerWithDest()`, so revise all the JMS tests for:
- proper `ApplicationContext.stop()` in the end of test
- `@DirtiesContext` if `SpringJUnit4ClassRunner` is in use
- `TaskScheduler.destroy()` if that
The JMS fix might relate to the https://jira.spring.io/browse/INT-4030, therefore consider this fix for cherry-picking into `4.2.x` (excluding `build.gradle` changes)
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3909
Un`@Ignore` the `ClientWebSocketContainerTests` to ensure that `Tomcat-8.0.33` has the fix for the closed WebSocket session race condition.
Fix `StompIntegrationTests` test data
Clean up queue channels before each test, because of some remained extra messages during the web-socket protocol interactions,
like socket close events or similar
Fix `RedisStoreInboundChannelAdapterIntegrationTests` race conditions
Since Jedis driver is async, we not always receive a reply from the `TX-sync` component immediately.
And loop checking the Redis for the `key` after some `Thread.sleep()`
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
Add `fixModifiers.gradle` to run after `check` task.
The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.
The `fixThis` has been ported from the SA, but without applying.
* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`
NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
JIRA: https://jira.spring.io/browse/INT-3939
Previously `RedisQueueInboundGateway` and `RedisQueueOutboundGateway` used different `RedisSerializer`s
for non-String objects.
* Change the default to the `JdkSerializationRedisSerializer` for the consistency in case of client/server scenarios.
The previous behavior with the `StringRedisSerializer` can be reinstated with `serializer` injection.
Doc Polishing.
JIRAs:
https://jira.spring.io/browse/INT-3830https://jira.spring.io/browse/INT-3523https://jira.spring.io/browse/INT-3846
* Fix the OOM condition, when we `release()` `UpperBound` independently of the previous `remove` result (https://jira.spring.io/browse/INT-3846)
* Fix "confuse" around `groupCapacity`, when we really didn't care about individual groups (https://jira.spring.io/browse/INT-3523)
* Add `upperBoundTimeout` to have a hook to wait some time for the empty slot in the store (https://jira.spring.io/browse/INT-3830)
* Fix some JavaDocs warnings
* Fix some typos
* Fix inconsistency in the `DelayHandler` around `removeMessageFromGroup` when `MS` is `SimpleMessageStore`
* Remove `UpperBound.release()` operation from `SimpleMessageStore.removeGroup()`.
The waiting process should worry about the new `UpperBound` instance.
Some other polishing
`tryAcquire` outside of the `lock`
Move `tryAcquire` within the `addMessageToGroup` outside of `lock`.
But do that only for groups which already exist.
For the new groups we have a fresh `UpperBound`, so no need to worry about dead lock and
we can obtain a permit immediately.
SimpleMessageGroup: BlockingQueue -> LinkedHashSet
`SimpleMessageStore`: use "unsynchonized" `SimpleMessageGroup`
Make some synchronization fixes according to the migration to the `LinkedHashSet`
Avoid extra `Collection`
`ResequencingMessageHandler`: compare `size()` of collections instead of `containsAll()`
Fix `ConcurrentModificationException` in the `AbstractKeyValueMessageStore`
Add `SimpleMessageStore.clearMessageGroup()`
Accept polishing and fix `RedisChannelMessageStoreTests`
`@Deprecated` `MessageGroupStore.removeMessageFromGroup()`
Fix some typos
Introduce `SimpleMessageGroupFactory`
Extract `MessageGroupFactory` and address PR comments
Polishing after rebase
JavaDocs and Reference Manual
Fix JavaDocs
JIRA: https://jira.spring.io/browse/INT-3924
* The `RedisQueueInboundGateway` and `RedisQueueMessageDrivenEndpoint` used `MICROSECONDS` for the wait for stop condition.
- Change to `MILLISECONDS`
* Both of them may pick up data from the Redis List even after the `stop()`.
- Add `rightPush()` to the `RedisQueueInboundGateway` to emulate `rollback`.
- Even if we have `rollback` there we may "steal" messages after the `context.stop()` causing unexpected race condition.
So change the logic between `stopTimeout` and `receiveTimeout` to `max`, thus we wait for stop more than for `rightPop`.
* Some attributes have been missed for the XML configuration.
- add `stop-timeout` and `recovery-interval` attributes to the XSD and parsers code.
- adjust `RedisQueueGatewayIntegrationTests` to use those new attributes.
* Some tests needed polishing:
- Rework `RedisStoreOutboundChannelAdapterIntegrationTests` to be based on `@ContextConfiguration` for better performance
- `RedisChannelParserTests` used to `mock` channels for the same Redis topic. Change of of them to different name to avoid `Dispatcher has no subscribers`
from the `testPubSubChannelUsage()` when the second `SubscribableRedisChannel` doesn't have subscribers.
* Polishing for `redis.adoc`
Address PR comments
* Rework logic to the `Math.min(this.stopTimeout, this.receiveTimeout)` on the adapter/gateway `stop()`
* Fix JavaDocs and `redis.adoc` to describe `stopTimeout` properly.
* Rework `RedisQueueGatewayIntegrationTests` to be based on the `randomUUID` for the queue name to send and receive.
Address PR comments.
https://jira.spring.io/browse/INT-3665
Fixes according Travis report
Introduce `...ExpressionString(String)` setter
Some further fixes and polishing
Address PR comments
This should be treated as experimental but with these changes it seems to build ok.
BUILD SUCCESSFUL
Total time: 5 mins 15.533 secs
It turns out that our sporadic Redis problems were fixed in a later version of Jedis.
The problem was that the `connection.subscribe()` exited immediately. When I started adding debug
logic, the problem went away (because I was using a newer versio of Jedis).
spring-data-redis 1.5.2 is updated to work with the 2.7.3 version of Jedis.
Polishing
JIRA: https://jira.spring.io/browse/INT-3749
Since some target application could inject there is `EvaluationContext` before, removal of
`setIntegrationEvaluationContext()` is breaking change.
This fix reverts all those setters.
JIRA: https://jira.spring.io/browse/INT-3749
Since `IntegrationEvaluationContextAware` isn't so "context-free" resource like `BeanFactory`
and `ApplicationContext`, but just a specific bean in the context, we can't follow with `BeanPostProcessor` logic - bad architecture by level of responsibility.
Therefore we should follow with standard Dependency Injection mechanism to retrieve `evaluationContext` for the particular component.
Since we can't rely on the `@Autowired` because SI can be used from the raw XML configuration,
we use utility method instead. The pattern to get the proper `integrationEvaluationContext` is:
```
@Override
protected void onInit() throws Exception {
super.onInit();
this.evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
}
```
JIRA: https://jira.spring.io/browse/INT-3642
Currently, `removeMessageFromGroup` rebuilds the group on every removal.
In every case where this method is used in the framework, the result is not used.
Add `removeMessagesFromGroup` that removes a collection of messages and returns no result.
INT-3642: Polishing - PR Comments
INT-3642: Polishing and Fix Group Metadata Size
Instead of waiting for a hard 1 second, intelligently wait
until the container is subscribed by sending/receiving a test message.
Also reduces the test run time from 20s to 200ms locally.
JIRA: https://jira.spring.io/browse/INT-3667
* fix `RedisLockRegistry.tryLock` memory leaks using 2 different thread local internal storages:
hard references for locked locks and weak references (optional) for others, weak references are used for lock obtaining optimization -
thread will get same `RedisLock` object for certain key before locking and after unlocking (if variable still exists)
* add `RedisLockRegistry.useWeakReferences` property for enable thread local weak references storage for unlocked locks, disabled by default
* fix `RedisLockRegistry$RedisLock.obtainLock` improper expire time update (expire time was updated on every attempt to get lock)
* update `RedisLockRegistry` tests
JIRA: https://jira.spring.io/browse/INT-3661
Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.
The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods
* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance
* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`
Fix `MongoDbMessageStoreClaimCheckIntegrationTests`
Addressing PR comments
JIRA: https://jira.spring.io/browse/INT-3628
Certain tasks in Spring Integration are long running. Indicate
so for the benefit of platforms such as WebLogic that can log warnings
for long running tasks otherwise.
Also fix conflicting versions of commons-io.
JIRA: https://jira.spring.io/browse/INT-3637
- Message Sources
- Message Handlers
Polishing; Final Review and PR Comments
- avoid second call to System.currentTimeMillis()
- fix elapsed time in handler metrics
- expose TrackableComponent when available
INT-3637: Add @IntergrationManagedResource
Prevents beans (channels, etc), which were previously picked
up via a proxy, from being exported by a standard context
MBeanExporter.
I looked at completely eliminating the MBeanExporterHelper,
which suppresses o.s.integration beans from being exported by
a standard MBeanExporter when there is an IMBE present, but I feel
this is too much of a breaking change. There are a number of
standard beans (such as WireTap) that are @ManagedResources and
these would disappear for users that don't have an IMBE.
That said, such beans previously disappear completely when there
*is* an IMBE so now they are now annotated with both so that
they are exported by at most one of the exporters.
Polishing; Use MBE.addExludedBean instead of DFA
INT-3639: JMX (AMQP/JMS) Channel Stats
JIRA: https://jira.spring.io/browse/INT-3639
Module channels inherit `ChannelSendMetrics`; add
`PollableChannelManagement` for polled channels.
INT-3638: JMX Initial Stats/Counts Settings
JIRA: https://jira.spring.io/browse/INT-3638
Add the ability to specify the initial settings for
'enableStats' and 'enableCounts' for MBeans that
support those statistics.
Polishing; PR Comments; Add Handler Metrics Test
Also found a problem using an inner MessageSource bean in an
inbound-channel-adapter - a BeanComponentDefinition was returned
instead of a BeanDefinition.
Add @DirtiesContext to JMX Tests
The new MonitorTests failed with InstanceAlreadyExistsException for the
errorChannel.
Add @DirtiesContext to all tests using the Spring test runner to avoid caching any
contexts after the test class completes.
INT-3637: JMX Support Negated Name Match Patterns
Note: with negated matches, order matters.
Polishing for `MonitorTests`
Fix `EnableIntegrationMBeanExport` JavaDocs
JIRA: https://jira.spring.io/browse/INT-3589, https://jira.spring.io/browse/INT-3624
* SF - 4.2. Fix Breaking changes for the `ApplicationEventPublisher`
* AMQP 1.5. Without issues
* Reactor - 2.0. Fix for the new `Stream` foundation
* jsonPath - 1.2.0. Fix for new `Predicate` abstraction. Add 'fail-fast error' to the `IntegrationRegistrar`
* Sshd - 0.13.0. Fix for new `VirtualFileSystemFactory` usage
* Spring Data - Fowler
* And others without issues
* Get rid of `reactor.util.StringUtils` usage
INT-3589: Address PR comments
Polishing
The "transaction" synchronization tests use a `QueueChannel`.
The payload is a live reference to the Redis object.
It is possible that the test for the returned size occurs after
the "transaction" commits.
Change the tests to extract the size() on the poller thread so
that the size is always captured before the "transaction" commits.
JIRA: https://jira.spring.io/browse/INT-3581
Move schemas to 4.2.
Add `selector-expression` to `<wire-tap/>`.
INT-3781: Fix What's New
Bump Namespace Version to 4.2
JIRA: https://jira.spring.io/browse/INT-3604
There is no test-case to cover the issue, because it isn't so easy to emulate the network glitch between `setIfAbsent` and `expire`
JIRA: https://jira.spring.io/browse/INT-3587
There is no ability to use own local LockRegistry to avoid overlapping of produced ReentrantLock objects, it can happens if used unique keys have same masked value for DefaultLockRegistry.
Add a constructor to allow the configuration of a lock registry with a differnt number of locks than the default.
Add RedisQueueOutboundGateway and RedisQueueInboundGateway
JIRA: https://jira.spring.io/browse/INT-3341
add test
fix format issue
change expectmessage to extractpayload, fix test potential bug
fix copyright year
fix format and naming issue
fix as Artem's comments
fix as Artem's comments
change boolean condition judgement
INT-3341: Polishing
Minor Doc Polishing
JIRA: https://jira.spring.io/browse/INT-3528
Use `min` and `max` methods instead of sorting an entire collection to get the minimum or maximum element.
Change other inline `Comparator<?>`s to the `final` fields
JIRA: https://jira.spring.io/browse/INT-3370
* Add `BoonJsonObjectMapper`
* Provide some tests
* Remove deprecations
* Polishing test according removed deprecations
* Change `JsonObjectMapper#populateJavaTypes` to get deal with `payload`, not its `class`,
since we can't (and Jackson's `TypeFactory`, too) determine generic type from `Collection`.
Use `iterators` instead to retrieve the type from the first item.
Conflicts:
build.gradle
spring-integration-amqp/src/test/java/org/springframework/integration/amqp/support/DefaultAmqpHeaderMapperTests.java
INT-3370: Polishing according PR comments
* Apply Gary's polishing to the Docs
* Fix `RecipientListRouter` JavaDoc warn
* Fix typo in test name for `UdpUnicastEndToEndTests`
* Fix `RedisQueueOutboundChannelAdapterTests` do not use Jackson 1.x
Final Polish
JIRA: https://jira.spring.io/browse/INT-3475
Change the phase for `SourcePollingChannelAdapterFactoryBean`, `AbstractPollingEndpoint`, `MessageProducerSupport`, `JmsMessageDrivenEndpoint`
to the `Integer.MAX_VALUE / 2`
JIRA: https://jira.spring.io/browse/INT-3442
Previously `RedisQueueMessageDrivenEndpoint.stop()` returned immediately.
It caused an issue when one more `message` might be read and processed to stopped app.
* Introduce `AbstractEndpoint#lifecycleCondition` and wait on it from `RedisQueueMessageDrivenEndpoint.doStop()`
and `signal()` it from `ListenerTask`.
* Since everything is done around `lifecycleLock` the `RedisQueueMessageDrivenEndpoint.stop()` waits for the proper
'last' message process.
* Add tiny `Thread.sleep(1)` to the `popMessageAndSend` cycle to free `lifecycleLock` for other Threads, e.g. `stop()`
INT-3442: Get rid of `lock` from listener cycle
* Introduce `stopTimeout` to minimize the `stop` thread blocking
* If the message is returned after that timeout it moved back to Redis List using `RPUSH`
INT-3442: Addressing PR comments
