JIRA: https://jira.spring.io/browse/INT-4498
* Use `FileCopyUtils` instead of `StreamUtils` in test-cases when we deal
with `InputStream` s and files resources.
* Fix reflection-based tests with AOP to use an `ArrayList` access
instead of `LinkedList` since an optimization in SF:
6d6aa72e8f
**Cherry-pick to 5.0.x and 4.3.x**
* Destroy RmiInboundGateway.RmiServiceExporter
The internal instance `RmiServiceExporter` of the `RmiInboundGateway`
has to be destroyed together with the outer instance to unbind `service`
from the RMI registry
* Perform some polishing for the `RmiInboundGateway` and optimize
a `AbstractInboundGatewayParser` for late channels binding
* * Polishing some tests
* Expose getters for channels on the `MessagingGatewaySupport`
* Fix RMI tests to use random port
* Fix LockRegLeaderIn for interrupted Thread.sleep
https://build.spring.io/browse/INT-FATS5IC-517
If current thread is interrupted, the `Thread.sleep()` interrupts
immediately.
In the catch block of the main loop in the `LockRegistryLeaderInitiator`
we have such a dangerous `sleep()` and don't restart election in this
candidate any more
* Move `Thread.sleep()` to else after checking the current thread for
interrupted state
* Remove `LongRunningIntegrationTest` rule from the
`RedisLockRegistryLeaderInitiatorTests` since it now works much faster
after proper `busy-wait` handling
**Cherry-pick to master**
* Ignore interruption on the sleep i catch and move on with loop
https://build.spring.io/browse/INT-FATS5IC-509
Looks like `busyWait` period as `5 secs` is too aggressive and may be
two or more cycles pass until the election happens.
At this time the waiting for the `granted` latch in `10 secs`
may expire already, therefore we fail in the assertion
* Decrease `busyWait` to the `1 secs` to let the election cycle to pass
quickly
* Increase latch wait time to the `20 secs`
**Cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-SI50X-JOB1-56
We can't wait for the latch in the interruptable code flow;
we can't have a round-robing election guarantees.
* Add `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)`
to the `LockRegistryLeaderInitiator` when we restart the main task
* Remove latches waiting and thread shifting from the
`RedisLockRegistryLeaderInitiatorTests`
* Use long `busyWaitMillis` for yielding initiator to let the second
candidate to be elected
**Cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-FATS5IC-501/
When we try to wait for the `Latch` in the interruptable code flow,
it is a fact that we step away from the waiting and end up with the
race condition downstream.
* Wrap `Latch` in the interruptable `publishOnRevoked()` code to the
`Executor.execute()`
* Remove `deleteTimeoutMillis` option from the `RedisLockRegistry`
since it doesn't make sense in the interruptable code.
* Add `RedisLockRegistry.setExecutor()` to allow to inject an external
`Executor`
* Add more debug logging into the `LockRegistryLeaderInitiator`
**Cherry-pick to 5.0.x**
* Use Lettuce Redis client; Redis module fixes
The Lettuce client is based on the Netty and more stable, than Jedis
therefore we get a performance improvement for tests
(it saves us at about 30 seconds).
Also this client doesn't fail for me on Windows sporadically (very often)
with the `ConnectionClosedException`
* After switching to the Netty-based client, we expose the interrupted
Thread issue in the `LockRegistryLeaderInitiator`.
If we interrupted (expected behavior), we try to unlock calling
`RedisLockRegistry`, but Netty client reject our request because the
thread is interrupted, therefore we never delete the lock when we yield
our leadership.
Fix the issue with shifting a `RedisTemplate.delete()` operation to the
`ExecutorService` when the current thread is interrupted
* Allow to configure such an `ExecutorService` and timeout to wait for
the `submit()` result
* Refactor `RedisAvailableRule` and all the Redis tests do not expose
the target `RedisConnectionFactory` implementation.
* Make all the test-cases based on the `connectionFactory` created by
the `RedisAvailableTests.setupConnectionFactory()`
* Tweak some unnecessary timeouts and sleeps for better tests task
throughput
* Add a `Log4j2LevelAdjuster` into the `RedisLockRegistryLeaderInitiatorTests`
JIRA: https://jira.spring.io/browse/INT-4390
Phase I: flow context/registration cycle
- extract interfaces
- rename implementations to `Standard...`
Phase II - dsl<->dsl.channel tangles
Move the channel specs to dsl.
Fix missing refactorings
(Not related to DSL) - rename core `event` package to `events`
- avoid collision with event module package
- fix tangle caused by `MessageGroupExpiredEvent`.
Phase III - dsl<->config.dsl
- move classes from config.dsl to dsl
(Not related to DSL) - fix many tangles caused by graph being a sub-package of support.management
- make `graph` a top-level package
(Not related to DSL) - move `IntegrationManagementConfigurer` from
`management` to `config` - tangle between core and management
* Polishing Copyrights, diamonds, some JavaDocs and What's New
JIRA: https://jira.spring.io/browse/INT-4407
To make a `Log4j2LevelAdjuster` more builder friendly, add a `merge`
option to the overloaded `classes()` and `categories()` methods.
This way the existing (or previously configured) `categories` and
`classes` are merged with the provided values and default
`org.springframework.integration` category can be reuse.
Note: this behavior can be reconsidered as default one in the `5.1`
https://build.spring.io/browse/INT-FATS5IC-400
When we run several concurrent builds (e.g. CI) and use the same shared
Redis server we may end up with the case when one process reads data
populated by another because we use the same key (groupId in our case)
* Fix `RedisMessageGroupStoreTests` to use `UUID.randomUUID()`
for the `groupId`
**Cherry-pick to 4.3.x**
* INT-4376: Upgage to Log4J 2
JIRA: https://jira.spring.io/browse/INT-4376
* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2
* Polishing after testing
Resolves: /spring-projects/spring-integration#2268
* To indicate the source the Redis message in the `RedisInboundChannelAdapter`
populate the `RedisHeaders.MESSAGE_SOURCE` header to the messages to produce
* Fix the `SimpleMessageConverter` to populate the provided `MessageHeaders`
to the message to produce
* Add `toMessage(T object, @Nullable Map<String, Object> headers)`
to the `InboundMessageMapper` to propagate additional header
to the message to create
* Rework all the out-of-the-box `InboundMessageMapper` implementations
to properly propagate additional headers via `toMessage()`
from the `SimpleMessageConverter`
* Provide optimizations in the `InboundMessageMapper` implementations
do not re-create messages
* Refactor `MutableMessage.toString()` to align with the `GenericMessage`
* Add more `@Nullable` to method arguments
* Increase latch wait timeout in the `EndpointParserTests`
* Address `redis.adoc` PR comment
Fixes: spring-projects/spring-integration#2071
* Clean up `build.gradle` for redundant excludes
* Add `javax.mail` dependency to WS module to avoid WARN about missed providers
* Refactoring for the XMPP module according changes in the latest Smack
* Polishing for the `BackToBackAdapterTests` to avoid extra wait for `null`
on the channel and some race conditions when client is closed during by the
`stop()` during publishing
Polishing `UriVariableTests` according the latest S-WS B-S
Revert excludes removal
JIRA: https://jira.spring.io/browse/INT-4357
Updated based on code review.
Updated adocs
*Polishing code style, typos
* Add protection from NPE when `LockRegistryLeaderInitiator` isn't
supplied with the `leaderEventPublisher`
* Add `What's New` note
JIRA: https://jira.springsource.org/browse/INT-4343
* Add a `Executor` setter named `setTaskExecutor` in
`RedisInboundChannelAdapter` to set the container's task executor.
* Add a `task-executor` attribute to
`<int-redis:inbound-channel-adapter>`.
* Update RedisInboundChannelAdapterParser configuration validation
tests.
* Update redis.adoc and whats-new.adoc.
* Polishing according PR comments
* Improve `RedisQueueGatewayIntegrationTests` performance
Fixesspring-projects/spring-integration#2213
JIRA: https://jira.spring.io/browse/INT-4123
Fully different `MessageStore`s can be configured for the same shared
Key-Value data-base.
Since the retrieval logic is based on the keys, that may cause the
unexpected messages expiration via `MessageGroupStoreReaper`.
* To distinguish store instances on the shared store add `prefix`
option to the `AbstractKeyValueMessageStore`
* Deprecate the `GemfireMessageStore` `Cache`-based configuration - `setIgnoreJta()` and `afterPropertiesSet()`.
The `GemfireMessageStore` relies only on an externally configured `Region`.
**Cherry-pick to 4.3.x**
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4341Fixesspring-projects/spring-integration#2228
Add to the `RedisQueueInboundGateway` and `RedisQueueMessageDrivenEndpoint`support
for the `receiveTimeout` of 0 and block indefinitely
Added some simple tests to check if the receive timeout can be set to 0
**Cherry-pick to 4.3.x**
https://build.spring.io/browse/INT-AT42SIO-595/
To avoid clashes with other parallel builds on CI for different versions
on the same shared Redis make a testing `groupId` as unique using
`UUID.randomUUID()`
**Cherry-pick to 4.3.x & 4.2.x**
JIRA: https://jira.spring.io/browse/INT-4290
See CVE-2017-4995
To disallow deserialization of unknown classes,
the `JacksonJsonUtils#messagingAwareMapper()` can now be supplied
with the `trustedPackages`.
The default list is:
```
java.util
java.lang
org.springframework.messaging.support
org.springframework.integration.support
org.springframework.integration.message
org.springframework.integration.store
```
Can be configured with `*` (asterisk) with meaning trust all
**Cherry-pick to 4.3.x**
Polishing according PR comments
JIRA: https://jira.spring.io/browse/INT-4267Fixes: spring-projects/spring-integration#2110
The documentation clearly point that we can simply use JSON (de)serialization
with the `RedisMessageStore`, but actually it isn't so easy
* Fix `MessageGroupMetadata`, `MessageHolder`, `MessageMetadata` for Jackson
deserialization compatibility
* Add `MessageHeaders`-based ctor to the `AdviceMessage`
* Add `MessageHeadersJacksonSerializer` to serialize `MessageHeaders`
to the `HashMap` for easier deserialization afterwards
* Add deserializer implementations for all `Message` types
* Add convenient `JsonObjectMapperProvider#jacksonMessageAwareMapper()`
factory method to build `ObjectMapper` supplied with mentioned above
(de)serializers
**Cherry-pick to 4.3.10 without `MessageHolder` and `MessageMetadata`**
Address PR comments and document the feature
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4258
Since Redis `ZADD` command doesn't perform incrementation by default,
it should be consistent in the `RedisStoreWritingMessageHandler` as well
* Make a default behavior of the `RedisStoreWritingMessageHandler`
regarding `INCR` option as `false`
* Provide some performance refactoring for the
`RedisStoreWritingMessageHandler` moving the code to Java 8 style
and using `FunctionExpression`
* Fix tests according to the changed default behavior
* Add `setZsetIncrementExpression` and String-based setters for other
expressions on the `RedisStoreWritingMessageHandler`
* Add `zset-increment-expression` option for XML configuration
* Add `What's New` note on the matter
Polishing
JIRA: https://jira.spring.io/browse/INT-4248
To avoid unexpected double locking behavior in the cluster,
remove the local cache functionality.
Now with the new `clientId` property, the `expire` for the record in
store is always update on each lock operation
JIRA: https://jira.spring.io/browse/INT-4243
Remove newline char from key exchange.
Fix CR and Add Windows Delete Diagnostics
Fix For Windows
More diagnostics.
It appears the new server doesn't close the file when the session is closed.
In the streaming test, consume the stream before closing the session.
Polishing - PR Comments
* More polishing according PR comments
JIRA: https://jira.spring.io/browse/INT-4219
- Use `AdditionalAnswers.returnFirstArgument` instead of lambda to return first arg
- Enable CheckStyle `AvoidStaticImport` for tests
- Fix static import violations in tests
JIRA: https://jira.spring.io/browse/INT-4176
The `DelayerHandlerRescheduleIntegrationTests` doesn't close `context` in the end of test.
That causes clashes for `beanFactoryResolver` in other tests.
When those tests are fail, we come into condition when new processes can't be started/stopped because of effect of non-stopped context
* Fix all the `DelayerHandlerRescheduleIntegrationTests` to stop `context` in the end of test
* Remove unused files in the `gemfire/store` test package
* Add current date value to the `CacheServerProcess` name to avoid clashes with other ran on the same host
* Add NPE check for `region` in the `GemfireMetadataStoreTests`
* Extract `@BeforeClass/@AfterClass` for the `CacheWritingMessageHandlerTests` to start only one Gemfire cache per class
**Cherry-pick to 4.3.x & 4.2.x**
Polishing - PR Comments and Closeable Warnings
Eclipse emits bogus warnings with exceptions in lambdas.
Even though the lambda might run on another thread, elipse thinks it could
cause the context to not be closed.
SPR-14854: MessageChannel is now a @FunctionalInterface
* Additional Lambda polishing and some code style fixes
JIRA: https://jira.spring.io/browse/INT-832
When we scan directory recursively for files (e.g. `WatchServiceDirectoryScanner`), it can be useful to get access to the relative path from the `Message`, e.g. on the `FileWritingMessageHandler` side to restore the original structure
* Add a `FileHeaders.FILENAME` into the outbound `Message` from the `FileReadingMessageSource`
* The result of that header is like a removal of leading `this.directory.getAbsolutePath()` in the target `File.getAbsolutePath()`.
In case of not recursion we get just only regular file name.
* Introduce `FileHeaders.RELATIVE_PATH`
* Populated `FileHeaders.RELATIVE_PATH`, `FileHeaders.FILENAME`, `FileHeaders.ORIGINAL_FILE` in the `FileReadingMessageSource`
* File `FileTailingMessageProducerSupport` to populate `FileHeaders` properly
* Introduce ctor for the `LastModifiedFileListFilter` for better Java configuration experience
* Add docs for changes
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4126
Looks like `WATCH` doesn't provide good isolation when the call is performed concurrently at the same time from different Threads.
* Change locking algorithm as it is recommended by Redis `SET` command: http://redis.io/commands/set:
```
SET resource-name anystring NX EX max-lock-time
```
* Add `@Repeat(10)` to the `AggregatorWithRedisLocksTests#testDistributedAggregator()` since that helped to reproduce the issue
**Cherry-pick to 4.3.x & 4.2.x**
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now
Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule
The latest Checkstyle has a bug with local scope variables if they have the same names as property.
Revert some literals splitting
Fix some line length exceeding and code style
JIRA: https://jira.spring.io/browse/INT-4105
Fixes GH-1888
The lock in Redis can be expired in between `obtain()` calls.
So, even if we return a new lock instance, the old one must clear properly.
* Add `lock.unlock()` to the `obtain()` if the case of expiration in Redis.
* Add `warn` for the exception on the `lock.unlock()`. We don't care about error here and just proceed to a new instance.
**Chery-pick to 4.3.x**
Remove the lock reference from `weakThreadLocks` as well
Add `registry.setUseWeakReferences(true);` to test-case
* Add assert for `lock.unlock()` in case of expiration
* Add `UUID.randomUUID()` for the registry key to avoid cross-talking during concurrent builds, e.g. on the CI server
https://build.spring.io/browse/INT-MASTER-352
This is actually a fix for the
JIRA: https://jira.spring.io/browse/INT-4083
* Fix several typos in log messages. And some test on the matter as well
* Add comment to `AbstractPersistentAcceptOnceFileListFilter.rollback()` to clarify the reason of `rollingBack` variable
* Make `RemoteFileTemplate.StreamHolder` as `static` to avoid extra internal variable to outer class instance
* Replace `MessagingException` with `AbstractInboundFileSynchronizingMessageSource` in `init()` method of some components. It isn't Messaging yet in that phase
* Fix `SubscribableRedisChannel.MessageListenerDelegate` to handle `Object` not `String`, because with the `serializer` injection there is no guaranty that incoming is always `String`
* Move `JSch.setLogger(new JschLogger());` in the `DefaultSftpSessionFactory` to `static` block. It really should be done only once
* Remove `Assert.isTrue(this.port >= 0)` from the `DefaultSftpSessionFactory`. The subsequant `initJschSession()` convert it to default `22` port
* Change in the `JschProxyFactoryBean` `UnsupportedOperationException` to `IllegalArgumentException`. Wrong enum is wrong argument. That isn't a problem of operation
* Simplify `stop()` in the `CuratorFrameworkFactoryBean` and mark it as a `this.running = false`. Otherwise it wasn't able to be restarted
* Expose `leaderEventPublisher` in the `LeaderInitiatorFactoryBean` and fix `stop(Runnable callback)` with propagation `callback` to delegate.
Fix `SubscribableRedisChannelTests` for new `handleMessage(Object)` signature
JIRA: https://jira.spring.io/browse/INT-1463
An existing `QueueChannel#getRemainingCapacity()` is based on the `BlockingQueue#getRemainingCapacity()` which is always `Integer.MAX_VALUE`
and doesn't reflect reality for provided `capacity`
* Fix `PriorityChannel` to return `upperBound.availablePermits()` for `getRemainingCapacity()`
* Add test for `PriorityChannel` on the matter
* Also increase `reply-timeout` for `<int-redis:queue-outbound-gateway>` in `RedisQueueGatewayIntegrationTests-context.xml` from 1 sec to 10 secs
The `RedisMessageGroupStoreTests.testMessageGroupUpdatedDateChangesWithEachAddedMessage()` can fail
sporadically because there is no guaranty the several calls to the `System.getCurrentTimeMillis()` return the same value.
* Fix `AbstractKeyValueMessageStore` to reuse `MessageGroup.timestamp` for the `lastModified` in case of a new group
https://build.spring.io/browse/INT-MJATS41-629
* Revert to SF and SA BUILD-SNAPSHOTs
* Upgrade to Spring Security 4.1
* Fix `RedisQueueMessageDrivenEndpointTests.testInt3442ProperlyStop()` for `atLeastOnce()` verification
because in between stop and the last `rightPush` there might be one more `pop` and push again because of `listening = false` state
* There is still sporadic `OutboundGatewayFunctionTests.testLazyContainerWithDest()`, so revise all the JMS tests for:
- proper `ApplicationContext.stop()` in the end of test
- `@DirtiesContext` if `SpringJUnit4ClassRunner` is in use
- `TaskScheduler.destroy()` if that
The JMS fix might relate to the https://jira.spring.io/browse/INT-4030, therefore consider this fix for cherry-picking into `4.2.x` (excluding `build.gradle` changes)
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3909
Un`@Ignore` the `ClientWebSocketContainerTests` to ensure that `Tomcat-8.0.33` has the fix for the closed WebSocket session race condition.
Fix `StompIntegrationTests` test data
Clean up queue channels before each test, because of some remained extra messages during the web-socket protocol interactions,
like socket close events or similar
Fix `RedisStoreInboundChannelAdapterIntegrationTests` race conditions
Since Jedis driver is async, we not always receive a reply from the `TX-sync` component immediately.
And loop checking the Redis for the `key` after some `Thread.sleep()`
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
