Fixes: #9561
PR: https://github.com/spring-projects/spring-integration/pull/9561
The `DelayHandler.DelayedMessageWrapper` cannot be deserialized when using `RedisMessageStore` and JSON serialization:
```
org.springframework.data.redis.serializer.SerializationException: Could not read JSON:Cannot construct instance of `org.springframework.integration.handler.DelayHandler$DelayedMessageWrapper` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: UNKNOWN; byte offset: #UNKNOWN]
at org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer.deserialize(GenericJackson2JsonRedisSerializer.java:311)
```
* More code clean up and refactoring in the test
**Auto-cherry-pick to `6.3.x` & `6.2.x`**
Although `RenewableLockRegistry` provides a renew interface, it is inconvenient for users.
Developers hope to have a lock that can be automatically renewed.
On the one hand, it can avoid subsequent failures caused by locks that will not expire when abnormal exits,
and on the other hand, it can avoid unlock failures caused by lock expired.
* Add `RenewableLockRegistry.setRenewalTaskScheduler()` and when it is set, schedule a `renew()` script periodically
when lock is acquired from Redis with `1/3` of `expireAfter`
* Test and document the feature
Even if `Executors.newSingleThreadExecutor()` returns a `FinalizableDelegatedExecutorService`,
an instance is kept in the memory until JVM exists.
That may lead to memory leak since we have a lot of threads in memory.
**Auto-cherry-pick to `6.3.x` & `6.2.x`**
Fixes: #9291
* Modify `unlock()` method of `JdbcLock`: if the lock ownership can not be removed due to data expiration, a `ConcurrentModificationException` should be thrown.
* Modify `unlock()` method of `RedisLock`: if the lock ownership can not be removed due to data expiration, a `ConcurrentModificationException` should be thrown.
* Maintain test cases
* Document the new behavior
Fixes: #8869
Do not set the `redisMessageListenerContainer`'s executor in the setter method
since the Redis Message Listener container is not initialised during configuration due to lazy loading
**Cherry-pick to `6.2.x` & `6.0.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/8773
The https://github.com/spring-projects/spring-integration/issues/8732 introduced a filtering for messages in group.
So, plain `removeMessage()` doesn't work any more if message is connected to some group yet.
Therefore, `DelayHandler` is failing.
* Introduce `getMessageFromGroup()` and `removeMessageFromGroupById()` into `MessageGroupStore` API
and implement it respectively in all the stores
* Remove `@LongRunningTest` from delayer integration tests and adjust its config to delay not for a long
**Cherry-pick to `6.1.x`**
* Fix KV Stores for same message in multiple groups
If same message is stored into different groups with the same KV store,
the removal of one group would lead to removal the message for the other one.
* Improve KV Store to save message with the key including a group id
* Respectively, refine the removal API to include group id into keys
* Also change the `MESSAGE_GROUP_KEY_PREFIX` for group records to `GROUP_OF_MESSAGES_`
since the `MESSAGE_` prefix includes group records as well for various operations
based on key pattern
* * Use `[^GROUP_]` in the pattern for messages count
**Cherry-pick to `6.1.x`**
* * Fix MongoDB MS for message removal logic
* * Bring back `GROUP_OF_MESSAGES_` prefix
to avoid complex regexp and don't bother for edge cases,
where even that regexp may fail
* GH-8704: Add global property for `defaultTimeout`
Fixes https://github.com/spring-projects/spring-integration/issues/8704
The default timeout for requests and replies in the integration endpoints
is 30 seconds to avoid indefinite blocking in threads.
Sometime those 30 seconds is not enough.
* Introduce a `spring.integration.endpoints.defaultTimeout` global property
to allow overriding all the timeouts to desired value.
The negative number indicates an indefinite waiting time: similar to what
was there before introducing 30 seconds by default
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8642: Revise executors in the project
Fixes https://github.com/spring-projects/spring-integration/issues/8642
* Rework some `Executors.newSingleThreadExecutor()` to `ExecutorServiceAdapter(new SimpleAsyncTaskExecutor())`
* Expose `TaskExecutor` setters; deprecate `ExecutorService`-based
* Some other code clean up in the effected classes: `LogAccessor`, no `synchronized` in critical blocks
* Give a meaningful prefix for default threads in the context of components, e.g. `SubscribableRedisChannel` - `getBeanName() + "-"`
* * Fix `PostgresChannelMessageTableSubscriberTests` for
`PostgresSubscribableChannel` initialization to let it create
its internal `Executor`
* Use an `AsyncTaskExecutor` injection instead of `ExecutorServiceAdapter` wrapping
* Fix `LockRegistryLeaderInitiatorTests` for `taskExecutor` injection
* Bring back `LockRegistryLeaderInitiator.setExecutorService()`
as an accident after property auto-renaming
Fix race condition, when methods `RedisLockRegistry#expireUnusedOlderThan` and `RedisLockRegistry#obtain` are executed successively.
It's possible to delete the lock from `RedisLockRegistry#expireUnusedOlderThan` method, when lock is created but is not acquired (`RedisLock#getLockedAt = 0`)
It can lead to the situation, when `RedisLockRegistry#obtain` returns multiple locks with the same redis-key, which shouldn't happen at all.
* Skip locks from expiration when their `lockedAt == 0` - new, not acquired yet.
**Cherry-pick to `6.0.x` & `5.5.x`**
Looks like sometime Spring Framework XML parser can properly determine the target factory method to chose,
but sometimes it does that in a wrong order.
* Add `type="java.lang.Class"` to the ctor args everywhere for `class="org.mockito.Mockito" factory-method="mock"`
`<bean>` definitions
Fixes https://github.com/spring-projects/spring-integration/issues/3686
* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
* GH-3888: Fix NPE in the `RedisLockRegistry.destroy()`
Fixed https://github.com/spring-projects/spring-integration/issues/3888
When `RedisLockType.SPIN_LOCK` (default), the `RedisLockRegistry.destroy()`
causes an NPE on the `redisMessageListenerContainer` since pub-sub is not used
in a busy-spin mode
* Check for `redisMessageListenerContainer` before calling its `destroy()`
**Cherry-pick to 5.5.x**
* * Reset properties in the `RedisLockRegistry` after `destroy()`
The `GenericJackson2JsonRedisSerializer` now resolves the target type
before calling `ObjectMapper.readValue()` causing a `ClassCast` in the
`StdNodeBasedDeserializer.deserializeWithType()`
* Override `MessageJacksonDeserializer.deserializeWithType()` with delegating
to the plain `deserialize()` ignoring the `TypeDeserializer`
Related to https://github.com/spring-projects/spring-data-redis/issues/2322
Fixes https://github.com/spring-projects/spring-integration/issues/3840
* Create a new abstraction `RedisTest` for Testcontainers-based tests
* Move existing common utility methods to the new interface
* Migrate all existing Redis tests to use this new `RedisTest` interface
* Migrate all existing Redis tests to JUnit5
* Make all existing Redis tests confirming to JUnit5 standards, such as
default methods and classes visibility instead of public
* Add a dependency for parametrized tests in JUnit5
* Improve assertions readability across tests, for instance:
assertThat(thing.size()).isEqualTo(2) -> assertThat(thing).hasSize(2)
* Add real assertions to
`RedisLockRegistryTests.twoRedisLockRegistryTest` (earlier it did not
have assertions at all)
* Reformat, rearrange and cleanup the code
* Fix a couple of small changes after the code review:
* Change base interface name to be consistent with other similar places
* Typo in javadocs
* Small tests readability improvement regarding assertions
* Add `opens java.util` to `spring-integration-redis`
to satisfy a `ReactiveRedisStreamMessageHandlerTests.testMessageWithListPayload()`
requirements
The logic of the test does not reflect the state of the
`RedisLockRegistry` around `ulinkAvailable` property:
we don't check the Redis version anymore, but try to call
`UNLINK` command.
Then we reset `ulinkAvailable` to `false` to call regular `DEL`
for the rest of `RedisLockRegistry` life
The `RedisAvailableRule` marks `testUlink()` as passed even without
calling it.
We are not going to look into fixing the rule since we are migrating
to Testcontainers: https://github.com/spring-projects/spring-integration/issues/3840
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException` does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
Fixes https://github.com/spring-projects/spring-integration/issues/3805
The Redis Pub-Sub doesn't work in all the environment, therefore there has to
be a choice to use old busy-spin algorithm
* Change to select between spinLock method and pub-sub method
* Make spinLock as a default one to let the `RedisLockRegistry` work everywhere
* Fix javadoc, convention, lazy init
* Fix javadoc, convention
* Code clean up and docs for `RedisLockType` feature
**Cherry-pick to 5.5.x**
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
Fixes https://github.com/spring-projects/spring-integration/issues/3700
Fixed a case where lock acquisition attempt was abandoned earlier than expected time.
If a vm uses `tryLock()` and another vm unlock it, it will wake up with an unlock message and try to acquire the lock,
but if it fails(the other vm obtain lock), vm will return false.
* Add `org.springframework.integration.history` to trusted default packaged of the `JacksonJsonUtils`
* Add `@JsonCreator` to `MessageHistory` `private` ctor to let it to be created automatically by Jackson
* Add `equals()` and `hashCode()` into `MessageHistory` for the proper `Message` comparison
* Add `MessageHistory` into headers for testing with Redis JSON (de)serialization
**Cherry-pick to `5.4.x` & `5.3.x`**
Related to https://build.spring.io/browse/INT-SONAR-3564
When we reset shared Redis connection between tests,
it may lead to a dead lock on some async command in process
when the next test initiate a new connection
* Remove `resetConnection()` from the `ReactiveRedisStreamMessageProducerTests` `@After` -
looks like it doesn't effect anything in a whole test suite
Fixes https://github.com/spring-projects/spring-integration/issues/3439
The latest Spring Data Redis has introduced an `onErrorResume` function
option for the `StreamReceiver` and this one is now recommended way
to handle errors in the `Flux` from this receiver
* Expose all the `StreamReceiver.StreamReceiverOptionsBuilder` option
onto the `ReactiveRedisStreamMessageProducer`, including `onErrorResume`
* Have a default function as it was before - send into an error channel
supporting (n)ack in the failed message based on the failed record
* Make new setters mutually exclusive with an explicit `StreamReceiver.StreamReceiverOptions`
* Doc polishing
Related to https://build.spring.io/browse/INT-MASTERSPRING40-JOB1-1271
* Fix race condition with subscription in the `FluxMessageChannel`:
rely on the `doOnRequest()` to start producing from upstream publishers
* Remove `SourcePollingChannelAdapterTests` since
`Class.getDeclaredConstructor().newInstance()` doesn't rethrow an exception as is
but wrap it into the `InvocationTargetException`.
This is probably the main reason to deprecate a regular `Class.newInstance()`
* Use `LettuceConnectionFactory.setEagerInitialization(true)` in the `RedisAvailableRule`
to avoid possible dead lock with lazy init
* Adjust Redis tests for new `RedisAvailableRule` behavior
* Use 'onErrorContinue' to continue receiving messages from Stream
* Send an `ErrorMessage` to the provided `errorChannel` (if any)
* And `@Nullable` to some `MessageProducerSupport` API which definitely
may accept `null`
* Extract common `buildMessageFromRecord()` method in the `ReactiveRedisStreamMessageProducer`,
so all the headers from the stream `Record` are carried to the message independently of the
record state - normal send or error sending
The R2DBC channel adapters have wrong componentType
* Fix R2DBC component type to not mention `reactive-`: they are reactive by R2DBC definition
The `ReactiveRedisStreamMessageProducer` calls ack() explicitly
when populates `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header,
but must provide an implementation of `SimpleAcknowledgment` instead
* Fix `ReactiveRedisStreamMessageProducer` to populate the proper
`IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header and in the proper moment
* Fix `ReactiveRedisStreamMessageProducerTests` to verify that `autoAck=false` works as expected
together with the `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header functionality
* Remove redundant `@SuppressWarnings("deprecation")`
* Add `Duration.ofSeconds(10)` to `StepVerifier.verify()`
to avoid infinite wait and lose failing text context on the hang CI build
