Commit Graph

325 Commits

Author SHA1 Message Date
Artem Bilan
1a288cd527 Remove resetConnection from Redis Reactive test
Related to https://build.spring.io/browse/INT-SONAR-3564

When we reset shared Redis connection between tests,
it may lead to a dead lock on some async command in process
when the next test initiate a new connection

* Remove `resetConnection()` from the `ReactiveRedisStreamMessageProducerTests` `@After` -
looks like it doesn't effect anything in a whole test suite
2021-04-14 10:17:13 -04:00
Artem Bilan
6ff5eac86c Increase timeouts for RedisLockLeaderInitTests
* `setBusyWaitMillis(10000)` before yielding to let another candidate to lock
* Add `Thread.sleep(100);` before yielding
2021-01-22 11:22:41 -05:00
Artem Bilan
8763c95622 GH-3469 New Sonar smell & volatile busyWaitMillis
Fixes https://github.com/spring-projects/spring-integration/issues/3469

* Fix new Sonar smells
* Mark `LockRegistryLeaderInitiator.busyWaitMillis` as `volatile`
so runtime changes (e.g. `RedisLockRegistryLeaderInitiatorTests`)
will make an immediate effect
2021-01-21 14:55:14 -05:00
Gary Russell
14fe3093f2 Ignore Flaky Test 2021-01-20 15:22:18 -05:00
Artem Bilan
51e240b761 GH-3439: Revise error handling in RedisStreamMP
Fixes https://github.com/spring-projects/spring-integration/issues/3439

The latest Spring Data Redis has introduced an `onErrorResume` function
option for the `StreamReceiver` and this one is now recommended way
to handle errors in the `Flux` from this receiver

* Expose all the `StreamReceiver.StreamReceiverOptionsBuilder` option
onto the `ReactiveRedisStreamMessageProducer`, including `onErrorResume`
* Have a default function as it was before - send into an error channel
supporting (n)ack in the failed message based on the failed record
* Make new setters mutually exclusive with an explicit `StreamReceiver.StreamReceiverOptions`

* Doc polishing
2021-01-20 13:36:35 -05:00
Artem Bilan
4909f030a1 GH-3434: Optimize unlinkAvailable in Redis
Fixes https://github.com/spring-projects/spring-integration/issues/3434

Each `RedisLock` has their own `unlinkAvailable` property
therefore we try to `unlink` on every unlock call and also log a WARN
about unavailability of this command in Redis, plus exception

* Move `unlinkAvailable` property to the `RedisLockRegistry` level
to have a check only once on a first `unlock()` call
* Move the whole exception logging onto the DEBUG level, leaving the WARN
only with the `ex.getMessage()`
* Optimize logging the same way in the `RedisMessageStore`

Cherry-pick to `5.3.x & 5.2.x`
2020-12-01 14:50:51 -05:00
Artem Bilan
efcc01804c Some miscellaneous fixes
Related to https://build.spring.io/browse/INT-MASTERSPRING40-JOB1-1271

* Fix race condition with subscription in the `FluxMessageChannel`:
rely on the `doOnRequest()` to start producing from upstream publishers
* Remove `SourcePollingChannelAdapterTests` since
`Class.getDeclaredConstructor().newInstance()` doesn't rethrow an exception as is
but wrap it into the `InvocationTargetException`.
This is probably the main reason to deprecate a regular `Class.newInstance()`
* Use `LettuceConnectionFactory.setEagerInitialization(true)` in the `RedisAvailableRule`
to avoid possible dead lock with lazy init
* Adjust Redis tests for new `RedisAvailableRule` behavior
2020-11-25 12:13:33 -05:00
Artem Bilan
7eb07344cc GH-3428: Make Kotlin dependency optional again
Fixes https://github.com/spring-projects/spring-integration/issues/3428

The Kotlin Gradle plugin starting with Kotlin `1.4` adds a standard
Kotlin lib into `compile` scope fully ignoring an `optional` variant.
See more info here: https://kotlinlang.org/docs/reference/using-gradle.html#dependency-on-the-standard-library

* Add `kotlin.stdlib.default.dependency=false` into `gradle.properties`
to rely only on the explicit dependency definitions in the project config
* Remove explicit Kotlin deps from test scope in the target modules since
it is declared globally for all the modules
* Fix author emails in the `publish-maven.gradle` to their VMware variants
* Remove usage of `org.jetbrains.annotations` in favor of `@Nullable`
from Spring Framework
2020-11-11 11:04:24 -05:00
Artem Bilan
0331933c16 Rely on MProducerSupport.active for Flux (#3423)
* Rely on `MProducerSupport.active` for `Flux`

* Fix `MessageProducerSupport` to extract an `active` flag and set it before
`isRunning` - the `Flux` subscription relies on the `takeWhile()`
where in case of `autoStartup = false` we will never start consume because
it is set to `true` already after `doStart()`
* Refactor all the `MessageProducerSupport` implementation with similar
`active` state to use already one from the super class

**Cherry-pick to 5.3.x**

* * Remove `MessageProducerSupport.setActive()`
to not let to mutate it from the implementations
* Set `active` to `false` in the `destroy()`
* Clean up and fix typos in the affected `JmsMessageDrivenEndpoint`

* * Pull `active` flag down to the `AbstractEndpoint`
* Set `active = true` in the `start()` before calling `doStart()`
* Do same for `active = false` in the `stop()`
* Clean up `AbstractEndpoint` impls to not call `doStart/doStop` for nothing
* Refactor endpoints to rely on the `active` state from the `AbstractEndpoint`
not their own
2020-11-06 13:51:03 -05:00
Artem Bilan
ac6a0982e5 Fix deprecations from Spring's StringUtils
* Use `LogAccessor` in the `XmlValidatingMessageSelector`
2020-10-28 16:33:51 -04:00
rohan mukesh
51ebf40a5f Add Docs for Redis Stream Channel adapters
* Fix JavaDoc for the `ReactiveRedisStreamMessageProducer.setExtractPayload()`
2020-10-28 13:43:15 -04:00
rohan mukesh
b1a383060d Add Error Handling to RedisStreamMessageProducer
* Use 'onErrorContinue' to continue receiving messages from Stream
* Send an `ErrorMessage` to the provided `errorChannel` (if any)
* And `@Nullable` to some `MessageProducerSupport` API which definitely
may accept `null`
* Extract common `buildMessageFromRecord()` method in the `ReactiveRedisStreamMessageProducer`,
so all the headers from the stream `Record` are carried to the message independently of the
 record state - normal send or error sending
2020-10-14 13:36:01 -04:00
Artem Bilan
5b74db8417 Upgrade dependencies; improve Redis Streams tests 2020-10-13 16:58:54 -04:00
Artem Bilan
20447df7a2 USe different consumer group for no ack test 2020-10-08 13:52:53 -04:00
Artem Bilan
1df4f92cfd Stop channel adapter and reset conn between tests 2020-10-08 13:27:13 -04:00
Artem Bilan
04296664e3 One more attempt for Redis Streams test 2020-10-08 13:07:48 -04:00
Artem Bilan
d020d7abfb Increase consume timeout for Redis Streams test 2020-10-08 11:45:56 -04:00
Artem Bilan
6c2498de34 AFTER_EACH_TEST_METHOD for Redis Streams tests 2020-10-08 11:26:49 -04:00
Artem Bilan
dbbd855fb8 More Redis Streams tests tweaks 2020-10-08 11:04:30 -04:00
Artem Bilan
68379b6fb1 Some Redis Streams tests improvements 2020-10-08 10:35:26 -04:00
rohan mukesh
6d6f53fa87 Fix r2bdc componentType; Redis Streams ack issue
The R2DBC channel adapters have wrong componentType

* Fix R2DBC component type to not mention `reactive-`: they are reactive by R2DBC definition

The `ReactiveRedisStreamMessageProducer` calls ack() explicitly
when populates `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header,
but must provide an implementation of `SimpleAcknowledgment` instead

* Fix `ReactiveRedisStreamMessageProducer` to populate the proper
`IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header and in the proper moment
* Fix `ReactiveRedisStreamMessageProducerTests` to verify that `autoAck=false` works as expected
together with the `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header functionality
2020-10-07 17:06:49 -04:00
Artem Bilan
c7ff99a4e8 Use LogAccessor from SF
* Change main classes to use a `LogAccessor` API to simplify code flow
* Fix tests according `LogAccessor` property
* Fix some Sonar smells
2020-10-06 13:56:50 -04:00
Artem Bilan
3c53a53b06 Fix Redis test to not initialize connection twice
Possible cause for: https://build.spring.io/browse/INT-MASTERSPRING40-1181
2020-08-27 10:53:18 -04:00
Artem Bilan
e5d859f1c7 Refinement for Redis Stream tests 2020-08-26 14:07:38 -04:00
Artem Bilan
3fb6567a1f Fix new Sonar smells 2020-08-21 11:37:43 -04:00
Artem Bilan
750d721437 Fix some Sonar smells 2020-08-20 16:08:18 -04:00
Attoumane
3cb13af813 More tests for Redis Stream support
Related to https://github.com/spring-projects/spring-integration/issues/3226
2020-08-11 09:30:09 -04:00
Artem Bilan
a10e317696 Fix unused import
* Fix new Sonar smells
2020-08-10 14:39:26 -04:00
Artem Bilan
9d557426b5 * Fix new Sonar smells
* Remove redundant `@SuppressWarnings("deprecation")`
* Add `Duration.ofSeconds(10)` to `StepVerifier.verify()`
to avoid infinite wait and lose failing text context on the hang CI build
2020-08-08 12:29:24 -04:00
Artem Bilan
65ad76232e Fix Checkstyle issues; upgrade some dependencies 2020-08-07 16:40:49 -04:00
Attoumane Ahamadi
4ae6b52c00 GH-3226: Add ReactiveRedisStreamMessageProducer
Fixes https://github.com/spring-projects/spring-integration/issues/3226

* Implement a `ReactiveRedisStreamMessageProducer` to consume Redis streams
* Add support for consumer groups and group auto-creation
* Add `@author` to `RedisHeaders`
* Rename test methods in the `ReactiveRedisStreamMessageHandlerTests` to project code style
* Implement `hashCode()` & `equals()` for `Person` & `Address` testing models
* Fix `ReactiveRedisStreamMessageProducerTests` not creating bean for `ConnectionFactory`.
Otherwise it caused to create one more not controlled `RedisClient`
* Code style clean up in the `ReactiveRedisStreamMessageProducer`
* Implement a group creation logic as a reactive stream deferring the call until
a subscription happens on the final `messageFlux`
* Move a common code for message building as the last `map()` operator in the final `Flux`
* Remove an `IntegrationFlow` definition in the `ReactiveRedisStreamMessageProducerTests`
as redundant
2020-08-07 15:23:20 -04:00
Gary Russell
1beb854fb4 Remove Legacy Metrics
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
  implement `ManageableLifecycle`. Register an additional MBean for
  polled endpoints to control the lifecycle.

* Polishing

- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
  to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
2020-08-07 12:56:57 -04:00
Artem Bilan
381a071287 Clean up RedisLockRegistryLeaderInitiatorTests
Related to https://build.spring.io/browse/INT-MASTERSPRING40-1129

When there is no other candidates in the cluster, the current `yield()`
will give a leadership to the current active initiator back
Therefore a `revoke()` might not be called at all

* Fir the `RedisLockRegistryLeaderInitiatorTests` to check revoking
after `stop()` instead of `yield()`
2020-07-06 11:30:43 -04:00
Attoumane AHAMADI
afa79d868b GH-3226: Redis Stream Outbound Channel Adapter
Fixes https://github.com/spring-projects/spring-integration/issues/3226

* Redis stream message handler support.
* This is the outbound part publishing message to the actual stream using ReactiveStreamOperations

* Addition of more test cases with one using `MessageChannel`.

* Improvements after PR review.

* Removed failed test reading List from a Stream
* Code style clean up
* Remove `rawtypes` usage
* Remove redundant inner classes for test model
* Add `What's New` note
2020-06-26 11:28:41 -04:00
Artem Bilan
5d7f0d2d1c Fix RedisQueueMessageDrEndpointTests for Java 14 2020-06-24 12:43:12 -04:00
Artem Bilan
6a3bea5a64 Add keepAlive(true) to testing Redis CF 2020-06-24 11:58:09 -04:00
Artem Bilan
94956859a2 Fix RedisAvailableRule for the proper evaluate
* Fix `RedisQueueMessageDrivenEndpointTests` for missed options
and for proper stop when the polling from the test queue is over
2020-06-23 16:27:45 -04:00
Artem Bilan
d24c2c8431 Fix Checkstyle violations & RMI tests
* Increase latch timeouts for `RedisLockRegistryLeaderInitiatorTests`
* Some RMI tests fail with different outcome because an RMI registry might be
available on default `1099` port.
Fix them to not rely on that port
2020-04-07 13:21:46 -04:00
Artem Bilan
87c8e47a88 GH-3192: pub-sub DSL for broker-backed channels (#3193)
* GH-3192: pub-sub DSL for broker-backed channels

Fixes https://github.com/spring-projects/spring-integration/issues/3192

* Introduce a `BroadcastCapableChannel` abstract to indicate those `SubscribableChannel`
implementations which can provide a pub-sub functionality
* Implement a `BroadcastCapableChannel` in broker-baked channels with pub-sub option
* Introduce a `BaseIntegrationFlowDefinition.publishSubscribeChannel()` based
on the `BroadcastCapableChannel` and `BroadcastPublishSubscribeSpec` to let to
configure sub-flow subscribers in fluent manner

* * Add some JavaDocs and document new feature

* * Show the channel bean definition in the doc
* Fix typo
2020-02-25 13:01:37 -05:00
Artem Bilan
370e943428 Remove deprecations from previous versions
* Remove Boon dependency and its usage
* Remove overloaded methods from the `IntegrationFlowDefinition`
- we can simply rely now on the super class
* Remove (or rework) deprecated entities in the docs
* Fix tests for removed deprecated APIs
* Rework affected tests to JUnit 5
2020-01-09 15:54:04 -05:00
Artem Bilan
5ac262f866 GH-3132: Remove usage of super();
Fixes https://github.com/spring-projects/spring-integration/issues/3132

It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation

* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions

* Fix new Sonar smells
2019-12-27 15:13:00 -05:00
Artem Bilan
9dc3519f20 Start version 5.3
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
 1. Align with SF
 2. The version for XSD doesn't matter for the current jar version,
 since only the version is available in classpath is from the current
 jar version
 3. Modify `spring.schemas` to map all the possible versions for XSD
 into the current one in a jar.
 This way target applications can upgrade without changing the version
 for XSD location and we don't need to require `versionless` variant
 any more
 4. The jar in classpath can handle only its own XSD, so independently
 of version or no-version variants we still may fail because the current
 jar may not support end-user code any more - need some fix over there
 anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
2019-12-26 15:41:19 -05:00
Artem Bilan
3dc0fb5980 GH-3041: Deprecate RedisUtils.isUnlinkAvailable()
Fixes https://github.com/spring-projects/spring-integration/issues/3041

Some Redis clients/servers don't allow to perform an `INFO` command,
therefore we are not able to determine if we can perform `UNLINK` or
not.

* Deprecate `RedisUtils.isUnlinkAvailable()` as not reliable source of
through; use trial with fallback algorithm in the target logic around
`UNLINK` command calls.

**Cherry-pick to 5.1.x**
2019-08-30 13:02:59 -04:00
Gary Russell
0edce21cfe Remove remaining redundant superinterfaces 2019-07-31 11:52:22 -04:00
Artem Bilan
80d679a9b0 GH-2748: More bean definitions into exceptions
Fixes https://github.com/spring-projects/spring-integration/issues/2748

* Refactor more `MessageHandlingException`s to include `this` into an
exception message
* Revert using `MessagingException` in some places which really are not
about messaging.
This helps to wrap them into `MessageHandlingException` later in the
`MessageHandler` for the `BeanDefinition` reference
* Remove `volatile` from configuration properties in the affected
classes
* Remove already deprecated `JmsOutboundGateway.setPriority()`
* Add `resource` and `source` for `BeanDefinition` in the
`AbstractChannelAdapterParser` & `AbstractInboundGatewayParser`
* Document the feature
2019-07-23 15:19:56 -04:00
Artem Bilan
c109e1df34 GH-2987: Add HTTPS entries into spring.schemas
Fixes https://github.com/spring-projects/spring-integration/issues/2987

To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file

**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
2019-07-18 14:05:18 -04:00
Artem Bilan
5e1a92db2e Fix race condition around Redis key
https://build.spring.io/browse/INT-FATS5IC-922/
https://build.spring.io/browse/INT-MJATS41-1764/

The `RedisQueueMessageDrivenEndpointTests` may be called from different
CI plans against the same Redis instance.

So, clean up keys before and after every unit test

**Cherry-pick until 4.3.x**
2019-07-01 14:36:27 -04:00
Artem Bilan
f89e8aafa5 Fix Redis components for JDK deserialization
It turns out that `JdkSerializationRedisSerializer` by default is
based on the default Java class loader which may lead into
`ClassCastException` downstream after deserialization

* Make all the `JdkSerializationRedisSerializer` usage (default)
in Redis components based on the BeanFactory `ClassLoader`
* Fix tests to call `setBeanClassLoader()`
* Fix Mark Fisher's name in the `MultipartFileReader` :-)

**Cherry-pick to 5.1.x, 5.0.x & 4.3.x after restoring diamonds**
2019-06-28 16:16:05 -04:00
Artem Bilan
315f0e711f Fix Sonar vulnerabilities for varargs
* Fix smell for static `AmqpInboundGateway.attributesHolder`

* Fix readOnlyHeaders in the `MessageBuilder`
2019-05-30 15:41:53 -04:00
Gary Russell
6d7bc1fc39 Sonar: repeated literals
* Polishing - PR Comments

* GatewayParser: Restore suppress warnings; remove size from `toArray()`.

* Merge conflict resolution
2019-05-03 12:39:02 -04:00