Commit Graph

343 Commits

Author SHA1 Message Date
Artem Bilan
69efbd3585 More Redis fixes for latest Spring Data 2022-03-18 12:44:31 -04:00
Artem Bilan
50cf3b1aa4 Various fixes after upgrades 2022-03-18 12:05:04 -04:00
Artem Bilan
70587f5e2f Upgrade some deps; fix compatibility with them
* Upgrade to the latest Spring portfolio stack;
fix tests and effected classes
* The move to the latest Micrometer does not affect code base
2022-03-08 10:08:36 -05:00
Unseok Kim
291fe34610 GH-3716 Fix wait for init redisMsgListenContainer
Fixes https://github.com/spring-projects/spring-integration/issues/3716

If the `redisMessageListenerContainer` is starting, waiting for it to complete without doing `subscribeUnlock()`

* Introduce `isRunningRedisMessageListenerContainer` state since the `running` in the `RedisMessageListenerContainer` is set in the beginning of the `start()` misleading on the concurrent calls to the `RedisLockRegistry`

**Cherry-pick to `5.5.x`**
2022-02-08 10:08:23 -05:00
Artem Bilan
a80b22638d Start 6.0 version
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
2022-01-18 14:38:50 -05:00
Unseok Kim
6c769cab37 GH-3700: Fix Redis lock retry until expiration
Fixes https://github.com/spring-projects/spring-integration/issues/3700

Fixed a case where lock acquisition attempt was abandoned earlier than expected time.
If a vm uses `tryLock()` and another vm unlock it, it will wake up with an unlock message and try to acquire the lock, 
but if it fails(the other vm obtain lock), vm will return false.
2022-01-04 09:16:10 -05:00
Artem Bilan
bb9dd89286 Fix RedisLockRegistry Sonar smells
* Remove redundant `subscribeLock()` method
* Rework `lock()` and `lockInterruptibly()` logic in favor of `while (true) {`
to avoid "empty `while()`" smell
2021-12-16 13:49:15 -05:00
Artem Bilan
30bb8467b4 Fix Checkstyle violation 2021-12-14 21:12:41 -05:00
Artem Bilan
2d752894cd Fix UNLINK/DEL logic in the RedisLockRegistry
See review: https://github.com/spring-projects/spring-integration/pull/3691#issuecomment-994124360
2021-12-14 19:50:49 -05:00
unseok kim
3c32daa25a GH-3690: Use pub-sub in RedisLockReg for unlocks
Fixes https://github.com/spring-projects/spring-integration/issues/3690

Replace spinLock with pub-sub

rename and add comment

* change remove timing for 'notifyMap'
* use unlink
* remove DUMMY_CALLABLE

* Clean up code style
2021-12-14 12:05:57 -05:00
Unseok Kim
db611028da GH-3672: Clean up Jdbc & ZK LockRegistry caches
Fixes https://github.com/spring-projects/spring-integration/issues/3672

* Clean up `JdbcLockRegistry`, `ZookeeperLockRegistry` cache automatically 
* setCapacity(int capacity) to cacheCapacity(int capacity)
* field rename `capacity`to `cacheCapacity`, add static
2021-11-11 16:36:27 -05:00
Unseok Kim
3c812440cf Change RedisLockRegistry cache to 100K
1_000_000 of cache entries take too much memory
2021-11-11 10:54:56 -05:00
unseok kim
e51513230a GH-3655: Add automatically delete for Redis Locks
Fixes https://github.com/spring-projects/spring-integration/issues/3655

* support automatically clean up cache
* RedisLockRegistry.capacity desc
2021-11-10 14:24:53 -05:00
Artem Bilan
fe57fd281c Checkstyle changes
* Upgrade to Checkstyle 9.0
* apply some JavaDocs rules
* Fix JavaDocs rules violations
* Some other minor clean up in the affected classes
2021-09-28 11:55:25 -04:00
trungPa
9e512186ed GH-3549: Fix minor SonarQube smells
Fixes https://github.com/spring-projects/spring-integration/issues/3549
2021-07-23 11:07:39 -04:00
Artem Bilan
af9e69c251 Make MessageHistory JSON-serializable
* Add `org.springframework.integration.history` to trusted default packaged of the `JacksonJsonUtils`
* Add `@JsonCreator` to `MessageHistory` `private` ctor to let it to be created automatically by Jackson
* Add `equals()` and `hashCode()` into `MessageHistory` for the proper `Message` comparison
* Add  `MessageHistory` into headers for testing with Redis JSON (de)serialization

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-07-19 12:37:05 -04:00
trungPa
9c718c37a6 GH-3424: Refactor to use logging methods from LogAccessor
Fixes https://github.com/spring-projects/spring-integration/issues/3424

* Use `LogMessage.format()` for lazily formatting

* Fix some logging statements in `JdbcChannelMessageStore`
2021-07-19 11:30:01 -04:00
Artem Bilan
dfd577596e Increase expiration time for RedisLock in test 2021-06-17 11:41:24 -04:00
Artem Bilan
1a288cd527 Remove resetConnection from Redis Reactive test
Related to https://build.spring.io/browse/INT-SONAR-3564

When we reset shared Redis connection between tests,
it may lead to a dead lock on some async command in process
when the next test initiate a new connection

* Remove `resetConnection()` from the `ReactiveRedisStreamMessageProducerTests` `@After` -
looks like it doesn't effect anything in a whole test suite
2021-04-14 10:17:13 -04:00
Artem Bilan
6ff5eac86c Increase timeouts for RedisLockLeaderInitTests
* `setBusyWaitMillis(10000)` before yielding to let another candidate to lock
* Add `Thread.sleep(100);` before yielding
2021-01-22 11:22:41 -05:00
Artem Bilan
8763c95622 GH-3469 New Sonar smell & volatile busyWaitMillis
Fixes https://github.com/spring-projects/spring-integration/issues/3469

* Fix new Sonar smells
* Mark `LockRegistryLeaderInitiator.busyWaitMillis` as `volatile`
so runtime changes (e.g. `RedisLockRegistryLeaderInitiatorTests`)
will make an immediate effect
2021-01-21 14:55:14 -05:00
Gary Russell
14fe3093f2 Ignore Flaky Test 2021-01-20 15:22:18 -05:00
Artem Bilan
51e240b761 GH-3439: Revise error handling in RedisStreamMP
Fixes https://github.com/spring-projects/spring-integration/issues/3439

The latest Spring Data Redis has introduced an `onErrorResume` function
option for the `StreamReceiver` and this one is now recommended way
to handle errors in the `Flux` from this receiver

* Expose all the `StreamReceiver.StreamReceiverOptionsBuilder` option
onto the `ReactiveRedisStreamMessageProducer`, including `onErrorResume`
* Have a default function as it was before - send into an error channel
supporting (n)ack in the failed message based on the failed record
* Make new setters mutually exclusive with an explicit `StreamReceiver.StreamReceiverOptions`

* Doc polishing
2021-01-20 13:36:35 -05:00
Artem Bilan
4909f030a1 GH-3434: Optimize unlinkAvailable in Redis
Fixes https://github.com/spring-projects/spring-integration/issues/3434

Each `RedisLock` has their own `unlinkAvailable` property
therefore we try to `unlink` on every unlock call and also log a WARN
about unavailability of this command in Redis, plus exception

* Move `unlinkAvailable` property to the `RedisLockRegistry` level
to have a check only once on a first `unlock()` call
* Move the whole exception logging onto the DEBUG level, leaving the WARN
only with the `ex.getMessage()`
* Optimize logging the same way in the `RedisMessageStore`

Cherry-pick to `5.3.x & 5.2.x`
2020-12-01 14:50:51 -05:00
Artem Bilan
efcc01804c Some miscellaneous fixes
Related to https://build.spring.io/browse/INT-MASTERSPRING40-JOB1-1271

* Fix race condition with subscription in the `FluxMessageChannel`:
rely on the `doOnRequest()` to start producing from upstream publishers
* Remove `SourcePollingChannelAdapterTests` since
`Class.getDeclaredConstructor().newInstance()` doesn't rethrow an exception as is
but wrap it into the `InvocationTargetException`.
This is probably the main reason to deprecate a regular `Class.newInstance()`
* Use `LettuceConnectionFactory.setEagerInitialization(true)` in the `RedisAvailableRule`
to avoid possible dead lock with lazy init
* Adjust Redis tests for new `RedisAvailableRule` behavior
2020-11-25 12:13:33 -05:00
Artem Bilan
7eb07344cc GH-3428: Make Kotlin dependency optional again
Fixes https://github.com/spring-projects/spring-integration/issues/3428

The Kotlin Gradle plugin starting with Kotlin `1.4` adds a standard
Kotlin lib into `compile` scope fully ignoring an `optional` variant.
See more info here: https://kotlinlang.org/docs/reference/using-gradle.html#dependency-on-the-standard-library

* Add `kotlin.stdlib.default.dependency=false` into `gradle.properties`
to rely only on the explicit dependency definitions in the project config
* Remove explicit Kotlin deps from test scope in the target modules since
it is declared globally for all the modules
* Fix author emails in the `publish-maven.gradle` to their VMware variants
* Remove usage of `org.jetbrains.annotations` in favor of `@Nullable`
from Spring Framework
2020-11-11 11:04:24 -05:00
Artem Bilan
0331933c16 Rely on MProducerSupport.active for Flux (#3423)
* Rely on `MProducerSupport.active` for `Flux`

* Fix `MessageProducerSupport` to extract an `active` flag and set it before
`isRunning` - the `Flux` subscription relies on the `takeWhile()`
where in case of `autoStartup = false` we will never start consume because
it is set to `true` already after `doStart()`
* Refactor all the `MessageProducerSupport` implementation with similar
`active` state to use already one from the super class

**Cherry-pick to 5.3.x**

* * Remove `MessageProducerSupport.setActive()`
to not let to mutate it from the implementations
* Set `active` to `false` in the `destroy()`
* Clean up and fix typos in the affected `JmsMessageDrivenEndpoint`

* * Pull `active` flag down to the `AbstractEndpoint`
* Set `active = true` in the `start()` before calling `doStart()`
* Do same for `active = false` in the `stop()`
* Clean up `AbstractEndpoint` impls to not call `doStart/doStop` for nothing
* Refactor endpoints to rely on the `active` state from the `AbstractEndpoint`
not their own
2020-11-06 13:51:03 -05:00
Artem Bilan
ac6a0982e5 Fix deprecations from Spring's StringUtils
* Use `LogAccessor` in the `XmlValidatingMessageSelector`
2020-10-28 16:33:51 -04:00
rohan mukesh
51ebf40a5f Add Docs for Redis Stream Channel adapters
* Fix JavaDoc for the `ReactiveRedisStreamMessageProducer.setExtractPayload()`
2020-10-28 13:43:15 -04:00
rohan mukesh
b1a383060d Add Error Handling to RedisStreamMessageProducer
* Use 'onErrorContinue' to continue receiving messages from Stream
* Send an `ErrorMessage` to the provided `errorChannel` (if any)
* And `@Nullable` to some `MessageProducerSupport` API which definitely
may accept `null`
* Extract common `buildMessageFromRecord()` method in the `ReactiveRedisStreamMessageProducer`,
so all the headers from the stream `Record` are carried to the message independently of the
 record state - normal send or error sending
2020-10-14 13:36:01 -04:00
Artem Bilan
5b74db8417 Upgrade dependencies; improve Redis Streams tests 2020-10-13 16:58:54 -04:00
Artem Bilan
20447df7a2 USe different consumer group for no ack test 2020-10-08 13:52:53 -04:00
Artem Bilan
1df4f92cfd Stop channel adapter and reset conn between tests 2020-10-08 13:27:13 -04:00
Artem Bilan
04296664e3 One more attempt for Redis Streams test 2020-10-08 13:07:48 -04:00
Artem Bilan
d020d7abfb Increase consume timeout for Redis Streams test 2020-10-08 11:45:56 -04:00
Artem Bilan
6c2498de34 AFTER_EACH_TEST_METHOD for Redis Streams tests 2020-10-08 11:26:49 -04:00
Artem Bilan
dbbd855fb8 More Redis Streams tests tweaks 2020-10-08 11:04:30 -04:00
Artem Bilan
68379b6fb1 Some Redis Streams tests improvements 2020-10-08 10:35:26 -04:00
rohan mukesh
6d6f53fa87 Fix r2bdc componentType; Redis Streams ack issue
The R2DBC channel adapters have wrong componentType

* Fix R2DBC component type to not mention `reactive-`: they are reactive by R2DBC definition

The `ReactiveRedisStreamMessageProducer` calls ack() explicitly
when populates `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header,
but must provide an implementation of `SimpleAcknowledgment` instead

* Fix `ReactiveRedisStreamMessageProducer` to populate the proper
`IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header and in the proper moment
* Fix `ReactiveRedisStreamMessageProducerTests` to verify that `autoAck=false` works as expected
together with the `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header functionality
2020-10-07 17:06:49 -04:00
Artem Bilan
c7ff99a4e8 Use LogAccessor from SF
* Change main classes to use a `LogAccessor` API to simplify code flow
* Fix tests according `LogAccessor` property
* Fix some Sonar smells
2020-10-06 13:56:50 -04:00
Artem Bilan
3c53a53b06 Fix Redis test to not initialize connection twice
Possible cause for: https://build.spring.io/browse/INT-MASTERSPRING40-1181
2020-08-27 10:53:18 -04:00
Artem Bilan
e5d859f1c7 Refinement for Redis Stream tests 2020-08-26 14:07:38 -04:00
Artem Bilan
3fb6567a1f Fix new Sonar smells 2020-08-21 11:37:43 -04:00
Artem Bilan
750d721437 Fix some Sonar smells 2020-08-20 16:08:18 -04:00
Attoumane
3cb13af813 More tests for Redis Stream support
Related to https://github.com/spring-projects/spring-integration/issues/3226
2020-08-11 09:30:09 -04:00
Artem Bilan
a10e317696 Fix unused import
* Fix new Sonar smells
2020-08-10 14:39:26 -04:00
Artem Bilan
9d557426b5 * Fix new Sonar smells
* Remove redundant `@SuppressWarnings("deprecation")`
* Add `Duration.ofSeconds(10)` to `StepVerifier.verify()`
to avoid infinite wait and lose failing text context on the hang CI build
2020-08-08 12:29:24 -04:00
Artem Bilan
65ad76232e Fix Checkstyle issues; upgrade some dependencies 2020-08-07 16:40:49 -04:00
Attoumane Ahamadi
4ae6b52c00 GH-3226: Add ReactiveRedisStreamMessageProducer
Fixes https://github.com/spring-projects/spring-integration/issues/3226

* Implement a `ReactiveRedisStreamMessageProducer` to consume Redis streams
* Add support for consumer groups and group auto-creation
* Add `@author` to `RedisHeaders`
* Rename test methods in the `ReactiveRedisStreamMessageHandlerTests` to project code style
* Implement `hashCode()` & `equals()` for `Person` & `Address` testing models
* Fix `ReactiveRedisStreamMessageProducerTests` not creating bean for `ConnectionFactory`.
Otherwise it caused to create one more not controlled `RedisClient`
* Code style clean up in the `ReactiveRedisStreamMessageProducer`
* Implement a group creation logic as a reactive stream deferring the call until
a subscription happens on the final `messageFlux`
* Move a common code for message building as the last `map()` operator in the final `Flux`
* Remove an `IntegrationFlow` definition in the `ReactiveRedisStreamMessageProducerTests`
as redundant
2020-08-07 15:23:20 -04:00
Gary Russell
1beb854fb4 Remove Legacy Metrics
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
  implement `ManageableLifecycle`. Register an additional MBean for
  polled endpoints to control the lifecycle.

* Polishing

- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
  to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
2020-08-07 12:56:57 -04:00