JIRA: https://jira.spring.io/browse/INT-4126
Looks like `WATCH` doesn't provide good isolation when the call is performed concurrently at the same time from different Threads.
* Change locking algorithm as it is recommended by Redis `SET` command: http://redis.io/commands/set:
```
SET resource-name anystring NX EX max-lock-time
```
* Add `@Repeat(10)` to the `AggregatorWithRedisLocksTests#testDistributedAggregator()` since that helped to reproduce the issue
**Cherry-pick to 4.3.x & 4.2.x**
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now
Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule
The latest Checkstyle has a bug with local scope variables if they have the same names as property.
Revert some literals splitting
Fix some line length exceeding and code style
JIRA: https://jira.spring.io/browse/INT-4105
Fixes GH-1888
The lock in Redis can be expired in between `obtain()` calls.
So, even if we return a new lock instance, the old one must clear properly.
* Add `lock.unlock()` to the `obtain()` if the case of expiration in Redis.
* Add `warn` for the exception on the `lock.unlock()`. We don't care about error here and just proceed to a new instance.
**Chery-pick to 4.3.x**
Remove the lock reference from `weakThreadLocks` as well
Add `registry.setUseWeakReferences(true);` to test-case
* Add assert for `lock.unlock()` in case of expiration
* Add `UUID.randomUUID()` for the registry key to avoid cross-talking during concurrent builds, e.g. on the CI server
https://build.spring.io/browse/INT-MASTER-352
This is actually a fix for the
JIRA: https://jira.spring.io/browse/INT-4083
* Make most functional interfaces as `@FunctionalInterface`
* Convert some abstract classes to `@FunctionalInterface` with `default` methods
* Apply Lambda style implementation in some places
* Remove `Function` in favor of similar in Java 8
* Remove redundant code from `DefaultAmqpHeaderMapper` since we are already on Spring AMQP-2.0
* Add several ctors to the `ExpressionEvaluatingMessageListProcessor`
* Populate explicit `Boolean.class` `expectedType` from the `ExpressionEvaluatingReleaseStrategy`
JIRA: https://jira.spring.io/browse/INT-3913
* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
* Fix several typos in log messages. And some test on the matter as well
* Add comment to `AbstractPersistentAcceptOnceFileListFilter.rollback()` to clarify the reason of `rollingBack` variable
* Make `RemoteFileTemplate.StreamHolder` as `static` to avoid extra internal variable to outer class instance
* Replace `MessagingException` with `AbstractInboundFileSynchronizingMessageSource` in `init()` method of some components. It isn't Messaging yet in that phase
* Fix `SubscribableRedisChannel.MessageListenerDelegate` to handle `Object` not `String`, because with the `serializer` injection there is no guaranty that incoming is always `String`
* Move `JSch.setLogger(new JschLogger());` in the `DefaultSftpSessionFactory` to `static` block. It really should be done only once
* Remove `Assert.isTrue(this.port >= 0)` from the `DefaultSftpSessionFactory`. The subsequant `initJschSession()` convert it to default `22` port
* Change in the `JschProxyFactoryBean` `UnsupportedOperationException` to `IllegalArgumentException`. Wrong enum is wrong argument. That isn't a problem of operation
* Simplify `stop()` in the `CuratorFrameworkFactoryBean` and mark it as a `this.running = false`. Otherwise it wasn't able to be restarted
* Expose `leaderEventPublisher` in the `LeaderInitiatorFactoryBean` and fix `stop(Runnable callback)` with propagation `callback` to delegate.
Fix `SubscribableRedisChannelTests` for new `handleMessage(Object)` signature
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
Add `fixModifiers.gradle` to run after `check` task.
The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.
The `fixThis` has been ported from the SA, but without applying.
* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`
NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
JIRA: https://jira.spring.io/browse/INT-3939
Previously `RedisQueueInboundGateway` and `RedisQueueOutboundGateway` used different `RedisSerializer`s
for non-String objects.
* Change the default to the `JdkSerializationRedisSerializer` for the consistency in case of client/server scenarios.
The previous behavior with the `StringRedisSerializer` can be reinstated with `serializer` injection.
Doc Polishing.
JIRAs:
https://jira.spring.io/browse/INT-3830https://jira.spring.io/browse/INT-3523https://jira.spring.io/browse/INT-3846
* Fix the OOM condition, when we `release()` `UpperBound` independently of the previous `remove` result (https://jira.spring.io/browse/INT-3846)
* Fix "confuse" around `groupCapacity`, when we really didn't care about individual groups (https://jira.spring.io/browse/INT-3523)
* Add `upperBoundTimeout` to have a hook to wait some time for the empty slot in the store (https://jira.spring.io/browse/INT-3830)
* Fix some JavaDocs warnings
* Fix some typos
* Fix inconsistency in the `DelayHandler` around `removeMessageFromGroup` when `MS` is `SimpleMessageStore`
* Remove `UpperBound.release()` operation from `SimpleMessageStore.removeGroup()`.
The waiting process should worry about the new `UpperBound` instance.
Some other polishing
`tryAcquire` outside of the `lock`
Move `tryAcquire` within the `addMessageToGroup` outside of `lock`.
But do that only for groups which already exist.
For the new groups we have a fresh `UpperBound`, so no need to worry about dead lock and
we can obtain a permit immediately.
SimpleMessageGroup: BlockingQueue -> LinkedHashSet
`SimpleMessageStore`: use "unsynchonized" `SimpleMessageGroup`
Make some synchronization fixes according to the migration to the `LinkedHashSet`
Avoid extra `Collection`
`ResequencingMessageHandler`: compare `size()` of collections instead of `containsAll()`
Fix `ConcurrentModificationException` in the `AbstractKeyValueMessageStore`
Add `SimpleMessageStore.clearMessageGroup()`
Accept polishing and fix `RedisChannelMessageStoreTests`
`@Deprecated` `MessageGroupStore.removeMessageFromGroup()`
Fix some typos
Introduce `SimpleMessageGroupFactory`
Extract `MessageGroupFactory` and address PR comments
Polishing after rebase
JavaDocs and Reference Manual
Fix JavaDocs
JIRA: https://jira.spring.io/browse/INT-3924
* The `RedisQueueInboundGateway` and `RedisQueueMessageDrivenEndpoint` used `MICROSECONDS` for the wait for stop condition.
- Change to `MILLISECONDS`
* Both of them may pick up data from the Redis List even after the `stop()`.
- Add `rightPush()` to the `RedisQueueInboundGateway` to emulate `rollback`.
- Even if we have `rollback` there we may "steal" messages after the `context.stop()` causing unexpected race condition.
So change the logic between `stopTimeout` and `receiveTimeout` to `max`, thus we wait for stop more than for `rightPop`.
* Some attributes have been missed for the XML configuration.
- add `stop-timeout` and `recovery-interval` attributes to the XSD and parsers code.
- adjust `RedisQueueGatewayIntegrationTests` to use those new attributes.
* Some tests needed polishing:
- Rework `RedisStoreOutboundChannelAdapterIntegrationTests` to be based on `@ContextConfiguration` for better performance
- `RedisChannelParserTests` used to `mock` channels for the same Redis topic. Change of of them to different name to avoid `Dispatcher has no subscribers`
from the `testPubSubChannelUsage()` when the second `SubscribableRedisChannel` doesn't have subscribers.
* Polishing for `redis.adoc`
Address PR comments
* Rework logic to the `Math.min(this.stopTimeout, this.receiveTimeout)` on the adapter/gateway `stop()`
* Fix JavaDocs and `redis.adoc` to describe `stopTimeout` properly.
* Rework `RedisQueueGatewayIntegrationTests` to be based on the `randomUUID` for the queue name to send and receive.
Address PR comments.
https://jira.spring.io/browse/INT-3665
Fixes according Travis report
Introduce `...ExpressionString(String)` setter
Some further fixes and polishing
Address PR comments
JIRA: https://jira.spring.io/browse/INT-3749
Since some target application could inject there is `EvaluationContext` before, removal of
`setIntegrationEvaluationContext()` is breaking change.
This fix reverts all those setters.
JIRA: https://jira.spring.io/browse/INT-3749
Since `IntegrationEvaluationContextAware` isn't so "context-free" resource like `BeanFactory`
and `ApplicationContext`, but just a specific bean in the context, we can't follow with `BeanPostProcessor` logic - bad architecture by level of responsibility.
Therefore we should follow with standard Dependency Injection mechanism to retrieve `evaluationContext` for the particular component.
Since we can't rely on the `@Autowired` because SI can be used from the raw XML configuration,
we use utility method instead. The pattern to get the proper `integrationEvaluationContext` is:
```
@Override
protected void onInit() throws Exception {
super.onInit();
this.evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
}
```
JIRA: https://jira.spring.io/browse/INT-3667
* fix `RedisLockRegistry.tryLock` memory leaks using 2 different thread local internal storages:
hard references for locked locks and weak references (optional) for others, weak references are used for lock obtaining optimization -
thread will get same `RedisLock` object for certain key before locking and after unlocking (if variable still exists)
* add `RedisLockRegistry.useWeakReferences` property for enable thread local weak references storage for unlocked locks, disabled by default
* fix `RedisLockRegistry$RedisLock.obtainLock` improper expire time update (expire time was updated on every attempt to get lock)
* update `RedisLockRegistry` tests
JIRA: https://jira.spring.io/browse/INT-3628
Certain tasks in Spring Integration are long running. Indicate
so for the benefit of platforms such as WebLogic that can log warnings
for long running tasks otherwise.
Also fix conflicting versions of commons-io.
JIRA: https://jira.spring.io/browse/INT-3637
- Message Sources
- Message Handlers
Polishing; Final Review and PR Comments
- avoid second call to System.currentTimeMillis()
- fix elapsed time in handler metrics
- expose TrackableComponent when available
INT-3637: Add @IntergrationManagedResource
Prevents beans (channels, etc), which were previously picked
up via a proxy, from being exported by a standard context
MBeanExporter.
I looked at completely eliminating the MBeanExporterHelper,
which suppresses o.s.integration beans from being exported by
a standard MBeanExporter when there is an IMBE present, but I feel
this is too much of a breaking change. There are a number of
standard beans (such as WireTap) that are @ManagedResources and
these would disappear for users that don't have an IMBE.
That said, such beans previously disappear completely when there
*is* an IMBE so now they are now annotated with both so that
they are exported by at most one of the exporters.
Polishing; Use MBE.addExludedBean instead of DFA
INT-3639: JMX (AMQP/JMS) Channel Stats
JIRA: https://jira.spring.io/browse/INT-3639
Module channels inherit `ChannelSendMetrics`; add
`PollableChannelManagement` for polled channels.
INT-3638: JMX Initial Stats/Counts Settings
JIRA: https://jira.spring.io/browse/INT-3638
Add the ability to specify the initial settings for
'enableStats' and 'enableCounts' for MBeans that
support those statistics.
Polishing; PR Comments; Add Handler Metrics Test
Also found a problem using an inner MessageSource bean in an
inbound-channel-adapter - a BeanComponentDefinition was returned
instead of a BeanDefinition.
Add @DirtiesContext to JMX Tests
The new MonitorTests failed with InstanceAlreadyExistsException for the
errorChannel.
Add @DirtiesContext to all tests using the Spring test runner to avoid caching any
contexts after the test class completes.
INT-3637: JMX Support Negated Name Match Patterns
Note: with negated matches, order matters.
Polishing for `MonitorTests`
Fix `EnableIntegrationMBeanExport` JavaDocs
JIRA: https://jira.spring.io/browse/INT-3581
Move schemas to 4.2.
Add `selector-expression` to `<wire-tap/>`.
INT-3781: Fix What's New
Bump Namespace Version to 4.2
JIRA: https://jira.spring.io/browse/INT-3604
There is no test-case to cover the issue, because it isn't so easy to emulate the network glitch between `setIfAbsent` and `expire`
JIRA: https://jira.spring.io/browse/INT-3587
There is no ability to use own local LockRegistry to avoid overlapping of produced ReentrantLock objects, it can happens if used unique keys have same masked value for DefaultLockRegistry.
Add a constructor to allow the configuration of a lock registry with a differnt number of locks than the default.
Add RedisQueueOutboundGateway and RedisQueueInboundGateway
JIRA: https://jira.spring.io/browse/INT-3341
add test
fix format issue
change expectmessage to extractpayload, fix test potential bug
fix copyright year
fix format and naming issue
fix as Artem's comments
fix as Artem's comments
change boolean condition judgement
INT-3341: Polishing
Minor Doc Polishing
JIRA: https://jira.spring.io/browse/INT-3528
Use `min` and `max` methods instead of sorting an entire collection to get the minimum or maximum element.
Change other inline `Comparator<?>`s to the `final` fields
JIRA: https://jira.spring.io/browse/INT-3442
Previously `RedisQueueMessageDrivenEndpoint.stop()` returned immediately.
It caused an issue when one more `message` might be read and processed to stopped app.
* Introduce `AbstractEndpoint#lifecycleCondition` and wait on it from `RedisQueueMessageDrivenEndpoint.doStop()`
and `signal()` it from `ListenerTask`.
* Since everything is done around `lifecycleLock` the `RedisQueueMessageDrivenEndpoint.stop()` waits for the proper
'last' message process.
* Add tiny `Thread.sleep(1)` to the `popMessageAndSend` cycle to free `lifecycleLock` for other Threads, e.g. `stop()`
INT-3442: Get rid of `lock` from listener cycle
* Introduce `stopTimeout` to minimize the `stop` thread blocking
* If the message is returned after that timeout it moved back to Redis List using `RPUSH`
INT-3442: Addressing PR comments
JIRA: https://jira.spring.io/browse/INT-3379
Change enumerated attributes to a union with xsd:string
to allow placeholders.
JMS and XML deferred to 4.1 due to the amount of
changes needed to the parsers. See INT-3384.
JIRA: https://jira.spring.io/browse/INT-2738
Add a `getComponentType()` method for all handlers,
message producers, and message sources that implement
`NamedComponent` (see `SPCA.getComponentType()`).
Add rome as an optional dependency - STS complained
because AtomFeedHttpMessageConverter has a dependency
on it.
JIRA: https://jira.spring.io/browse/INT-3355
Previously, the RedisAvailableTests unconditionally
flushed Redis; this is not appropriate and could affect
other builds on the CI server (as well as wiping out the
developer's Redis DB).
Change the tests to clean up their own redis data.
Also, improve `RedisLockRegistry.listLocks()` to use mget.
JIRA: https://jira.spring.io/browse/INT-3352
Provid a global lock registry, initially for the correlating
message handlers.
INT-3252 Polishing
- Use a discrete key for each lock and use Redis EXPIRE
- Use an internal DefaultLockRegistry to improve latency
when threads are using the same registry instance.
INT-3352 More Polish
- Put localLock.unlock() in a finally block.
- Unlock localLock if acquiring the RedisLock fails.
INT-3352 Polishing - PR Comments
- PR Comments
- Improve equals()
INT-3352 Add Aggregator Integration Test
- move lock registry to the `util` package
- Add an aggregator integration test
INT-3352 Add Distributed Aggregator Test
- Simulates a distributed environment with a shared message
store and distinct lock registries.
INT-3352 More PR Comments
INT-3352 Add Multiple Registry Tests
- Add tests to verify expected behavior when different registries are used.
- Polish javadocs.
