Commit Graph

10574 Commits

Author SHA1 Message Date
Artem Bilan
040fa703ca GH-3455: default MqttMessConv.toMessageBuilder()
Fixes https://github.com/spring-projects/spring-integration/issues/3455

After introduction `MqttMessageConverter.toMessageBuilder()`
(https://github.com/spring-projects/spring-integration/issues/3181)
the existing `MqttMessageConverter` must also implement this new method
where in most cases `toMessage()` must call `toMessageBuilder()` instead though.

* Make `MqttMessageConverter.toMessageBuilder()` as a `default` with a delegation
to the `toMessage()` allowing target implementors to keep their code during
migration without any breaking changes

**Cherry-pick to 5.3.x**
2021-01-11 17:05:07 -05:00
Artem Bilan
732a1b87e1 GH-3454: From MQTT conversion error - to error ch (#3456)
* GH-3454: From MQTT conversion error - to error ch

Fixes https://github.com/spring-projects/spring-integration/issues/3454

The message converter may return null when we try to covert from the
MQTT message.
The thrown exception may also reset the client connect.

* Fix `MqttPahoMessageDrivenChannelAdapter` to catch any conversion errors
(including `null` result) and try to send an `ErrorMessage` with that info
into the provided `errorChannel`.
Otherwise re-throw it as as

**Cherry-pick to `5.4.x` & `5.3.x`**

* * Apply review language-specific changes
2021-01-08 11:30:32 -05:00
Artem Bilan
ebe79aedfb GH-3452: Warn Publisher.onError in NullChannel (#3453)
* GH-3452: Warn Publisher.onError in NullChannel

Fixes https://github.com/spring-projects/spring-integration/issues/3452

The `NullChannel` can subscribe to the `Publisher` payload fully ignoring
the possible data according the `NullChannel` nature.
However an error thrown from the reactive stream processing is also ignored.

* Log WARN message from the `Subscriber.onError()` when `NullChannel`
subscribes to the `Publisher` produced to this channel.
* Mention the logic in the `NullChannel` docs; point to the
`ReactiveRequestHandlerAdvice` for further possible error handling in
the target application

**Cherry-pick to `5.4.x`**

* * Remove unused imports

* * Fix `PseudoTransactionalMessageSourceTests` for `LogAccessor`
from `NullChannel`
2021-01-05 15:09:01 -05:00
Artem Bilan
5ed8c11c58 Subscribe to Publisher in the NullChannel (#3448)
* Subscribe to Publisher in the NullChannel

If a payload of the message in the flow is a `org.reactivestreams.Publisher`,
it must be subscribed somewhere downstream to initiate reactive processing.
The `NullChannel` just ignores the message altogether and therefore `Publisher`
is lost

* Check the payload of the message in the `NullChannel` for `Publisher` type
and subscribe to it
* Verify in the test and mention this logic in the docs

**Cherry-pick to 5.4.x**

* Apply suggestions from code review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-12-22 12:20:45 -05:00
Artem Bilan
1bb58f49fa Add https://plugins.gradle.org/m2 repo into build
**Cherry-pick to `5.3.x & 5.2.x`**
2020-12-21 11:18:08 -05:00
Olivier Hubaut
a79c245cac GH-3445: Fix JdbcLock for IllegalMonitorStateEx
Fixes https://github.com/spring-projects/spring-integration/issues/3445

The `JdbcLockRegistry` improperly unlock the delegate (a `ReentrantLock`)
in the case of `TransientDataAccessException` or a `TransactionTimedOutException` is thrown.
As this occurs in a while loop, the next time the delegate is unlocked,
it not longer as a owner thread and throws an `IllegalMonitorStateException`.

* Move `delegate.unlock()` outside the while loop in the `JdbcLock.unlock()` method
* Add `JdbcLockRegistryDelegateTests` for mocked `LockRepository` to add more coverage
for `JdbcLockRegistry` functionality

**Cherry-pick to 5.4.x, 5.3.x & 5.2.x**

(cherry picked from commit 7e66509682)
2020-12-21 11:07:41 -05:00
Spring Buildmaster
05a8aa608d [artifactory-release] Next development version 2020-12-09 21:00:19 +00:00
Spring Buildmaster
401626d372 [artifactory-release] Release version 5.4.2 2020-12-09 21:00:15 +00:00
Artem Bilan
c4ed33543f Upgrade dependencies; prepare for release 2020-12-09 15:31:42 -05:00
Artem Bilan
4909f030a1 GH-3434: Optimize unlinkAvailable in Redis
Fixes https://github.com/spring-projects/spring-integration/issues/3434

Each `RedisLock` has their own `unlinkAvailable` property
therefore we try to `unlink` on every unlock call and also log a WARN
about unavailability of this command in Redis, plus exception

* Move `unlinkAvailable` property to the `RedisLockRegistry` level
to have a check only once on a first `unlock()` call
* Move the whole exception logging onto the DEBUG level, leaving the WARN
only with the `ex.getMessage()`
* Optimize logging the same way in the `RedisMessageStore`

Cherry-pick to `5.3.x & 5.2.x`
2020-12-01 14:50:51 -05:00
Artem Bilan
97e1692a36 Fix some Sonar smells 2020-11-30 12:00:59 -05:00
Artem Bilan
c12acb9ff8 Fix FluxMessCh for subscription race condition
Related to https://build.spring.io/browse/INT-MASTER-2240

Turns out the `doOnRequest()` is still not enough to be sure
that subscriber is accepted into the `Publisher`.
Probably because `doOnRequest()` maybe really asked for the
subscription by itself for some logic

* Use `Mono` with `repeatWhenEmpty()` until
`this.sink.currentSubscriberCount() > 0`
2020-11-25 15:22:46 -05:00
Artem Bilan
efcc01804c Some miscellaneous fixes
Related to https://build.spring.io/browse/INT-MASTERSPRING40-JOB1-1271

* Fix race condition with subscription in the `FluxMessageChannel`:
rely on the `doOnRequest()` to start producing from upstream publishers
* Remove `SourcePollingChannelAdapterTests` since
`Class.getDeclaredConstructor().newInstance()` doesn't rethrow an exception as is
but wrap it into the `InvocationTargetException`.
This is probably the main reason to deprecate a regular `Class.newInstance()`
* Use `LettuceConnectionFactory.setEagerInitialization(true)` in the `RedisAvailableRule`
to avoid possible dead lock with lazy init
* Adjust Redis tests for new `RedisAvailableRule` behavior
2020-11-25 12:13:33 -05:00
Artem Bilan
63bd3e4938 Some miscellaneous docs related fixed 2020-11-19 14:04:25 -05:00
Artem Bilan
a12306be45 Clean repositories in build script 2020-11-16 10:08:10 -05:00
Spring Buildmaster
d1d4a7edae [artifactory-release] Next development version 2020-11-11 19:47:07 +00:00
Spring Buildmaster
21fdd4e4a4 [artifactory-release] Release version 5.4.1 2020-11-11 19:47:02 +00:00
Artem Bilan
b11bb3aed3 Upgrade dependencies; prepare for release 2020-11-11 14:23:23 -05:00
Artem Bilan
7eb07344cc GH-3428: Make Kotlin dependency optional again
Fixes https://github.com/spring-projects/spring-integration/issues/3428

The Kotlin Gradle plugin starting with Kotlin `1.4` adds a standard
Kotlin lib into `compile` scope fully ignoring an `optional` variant.
See more info here: https://kotlinlang.org/docs/reference/using-gradle.html#dependency-on-the-standard-library

* Add `kotlin.stdlib.default.dependency=false` into `gradle.properties`
to rely only on the explicit dependency definitions in the project config
* Remove explicit Kotlin deps from test scope in the target modules since
it is declared globally for all the modules
* Fix author emails in the `publish-maven.gradle` to their VMware variants
* Remove usage of `org.jetbrains.annotations` in favor of `@Nullable`
from Spring Framework
2020-11-11 11:04:24 -05:00
Artem Bilan
6565b7cc43 Fix mngmt dependency for MetricsCaptor
The `IntegrationManagementConfigurer` is a `BeanPostProcessor`
so it must not have direct dependency injection for other beans.
In our case it is a `MetricsCaptor` injected from the
`IntegrationManagementConfiguration`

* Fix `IntegrationManagementConfiguration` and `IntegrationManagementConfigurer`
to rely on the `ObjectProvider<MetricsCaptor>` instead

Tested against latest Spring Boot

**Cherry-pick to `5.3.x` & `5.2.x`**
2020-11-10 16:00:57 -05:00
Artem Bilan
fce0fef98f GH-3425: Remove mngmt gauges from CtxClosedEvent
Fixes https://github.com/spring-projects/spring-integration/issues/3425

It turns out that `IntegrationManagementConfigurer.destroy()` is still too
late to remove `gauges` and Micrometer tries to gather them on application
context close, when many beans are already destroyed

* Catch `ContextClosedEvent` in the `IntegrationManagementConfigurer`
and removed `gauges` from there
* Remove `destroy()` impl since it is out of use already:
the `IntegrationManagementConfigurer` is not supposed to be in the
target application directly, so it looks safe to remove the `DisposableBean`
altogether

**Cherry-pick to `5.3.x` & `5.2.x`**
2020-11-10 15:16:39 -05:00
Artem Bilan
0331933c16 Rely on MProducerSupport.active for Flux (#3423)
* Rely on `MProducerSupport.active` for `Flux`

* Fix `MessageProducerSupport` to extract an `active` flag and set it before
`isRunning` - the `Flux` subscription relies on the `takeWhile()`
where in case of `autoStartup = false` we will never start consume because
it is set to `true` already after `doStart()`
* Refactor all the `MessageProducerSupport` implementation with similar
`active` state to use already one from the super class

**Cherry-pick to 5.3.x**

* * Remove `MessageProducerSupport.setActive()`
to not let to mutate it from the implementations
* Set `active` to `false` in the `destroy()`
* Clean up and fix typos in the affected `JmsMessageDrivenEndpoint`

* * Pull `active` flag down to the `AbstractEndpoint`
* Set `active = true` in the `start()` before calling `doStart()`
* Do same for `active = false` in the `stop()`
* Clean up `AbstractEndpoint` impls to not call `doStart/doStop` for nothing
* Refactor endpoints to rely on the `active` state from the `AbstractEndpoint`
not their own
2020-11-06 13:51:03 -05:00
Gary Russell
13807eeb38 GH-3418: Fix Poller Undeclared Checked Exceptions
Resolves https://github.com/spring-projects/spring-integration/issues/3418

`Class.newInstance()` can propagate checked exceptions that are not declared.

**cherry-pick/back-port to 5.3.x, 5.2.x, 4.3.x**
2020-11-02 11:37:40 -05:00
Spring Buildmaster
04f89f5421 [artifactory-release] Next development version 2020-10-28 21:15:24 +00:00
Spring Buildmaster
1b38a30410 [artifactory-release] Release version 5.4.0 2020-10-28 21:15:19 +00:00
Artem Bilan
ac6a0982e5 Fix deprecations from Spring's StringUtils
* Use `LogAccessor` in the `XmlValidatingMessageSelector`
2020-10-28 16:33:51 -04:00
Artem Bilan
0a2a82a2f3 Upgrade dependencies; prepare for release 2020-10-28 15:38:01 -04:00
Artem Bilan
2dc8702130 Add Docs for R2DBC Channel adapters 2020-10-28 14:47:02 -04:00
rohan mukesh
51ebf40a5f Add Docs for Redis Stream Channel adapters
* Fix JavaDoc for the `ReactiveRedisStreamMessageProducer.setExtractPayload()`
2020-10-28 13:43:15 -04:00
Artem Bilan
80d3c67847 * Add @NonNullApi to ZeroMQ packages 2020-10-26 16:11:44 -04:00
Artem Bilan
5221ac1740 Add Java DSL for R2DBC module (#3415)
* Add Java DSL for R2DBC module

* Add JavaDocs for R2DBC component setters
* Introduce a `R2dbcMessageSource.SelectCreator` to narrow
a `StatementMapper` API access from SpEL
* Add missed `package-info.java` into R2DBC packages
* Add `hamcrest-core` as test dependency into SI-R2DBC
since Awaitility doesn't work without it

* * Add `@NonNullApi` to R2DBC packages
2020-10-26 16:10:34 -04:00
Artem Bilan
7807e0ec59 GH-3301: Improve JMS receiveTimeout docs
Fixes https://github.com/spring-projects/spring-integration/issues/3301

Explain in docs how to configure a `JmsTemplate.receiveTimeout(-1)`
2020-10-26 16:07:56 -04:00
Artem Bilan
053ea1dd37 GH-3394: Fix Redis (and some other) docs
Fixes https://github.com/spring-projects/spring-integration/issues/3394

* Fix (wrong) copied/pasted text in the Redis Queue Inbound Gateway attribute
descriptions
* Fix some docs typos
* Add `jmx.adoc` into an `index.adoc` and `index-single.adoc`
* Add R2DBC, Redis Stream and ZeroMQ channel adapters into an `endpoint-summary.adoc`
2020-10-23 11:12:50 -04:00
Gary Russell
a79be71ed2 GH-3410: Add UDP SocketCustomizer
Resolves https://github.com/spring-projects/spring-integration/issues/3410

**cherry-pick to 5.3.x**

* Doc polishing.
2020-10-22 13:32:56 -04:00
Artem Bilan
b94c20a0ce Gateway: Propagate Error to the errorChannel
See SO for more info:
https://stackoverflow.com/questions/64456946/handle-exceptions-errors-other-than-messagingexception-ie-other-error-excepti

In the versions before `5.2.x` the `Error` was wrapped to the `MessagingException`
in the `AbstractRequestHandlerAdvice` and this one was handled properly
on the gateway level with its `errorChannel` configured

* Fix `MessagingGatewaySupport` to catch all the `Throwable` and try to send them as `ErrorMessage`
to the `errorChannel`.
Otherwise unwrap returned `MessagingException` and re-throw an `Error` as is to keep
the current behavior for non-handled exceptions

* Do not wrap `Error` into a `MessagingException` and re-throw as is if there is no `errorChannel`

**Cherry-pick to 5.3.x & 5.2.x**
2020-10-22 11:41:34 -04:00
Artem Bilan
fb35a736d6 Get correct next execution date from the trigger
With the `time-source` in Spring Cloud Stream we noticed
that first two items have the same (or close) value
when the `time-supplier` is used with the reactive output channel.
Turns out the `contactMap()` with `prefetch == 1` asks two
upstream elements immediately not waiting for a completion
for the fist one.
This way we were not able to update `triggerContext` for the proper
`lastCompletionTime`

* Change `AbstractPollingEndpoint` to `prefetch = 0` for the
"fair" upstream request
* Verify the behavior with the new
`ReactiveInboundChannelAdapterTests.testTimeSupplierConsistency()`
* Mark `ReactiveInboundChannelAdapterTests` as `@LongRunningTest`
since it is now pretty long waiting for all the 3 dates to verify
* Fix `InboundChannelAdapterAnnotationPostProcessor` to properly
register `MethodInvokingMessageSource` when we have more than one
`Supplier` with the `@InboundChannelAdapter` in the same configuration
class
2020-10-21 14:00:51 -04:00
Gary Russell
3cf66cc539 Fix previous commit 2020-10-19 11:48:18 -04:00
Gary Russell
b6d60bfd69 KPMH - Remove unused parameter 2020-10-19 11:32:33 -04:00
Gary Russell
75d14d32e7 Further Improve Kafka Test Run Time
Configurable `timeoutBuffer`.
2020-10-16 16:28:34 -04:00
Artem Bilan
a136b5b49a Fix What's New for the proper link to Kafka chapter 2020-10-16 14:43:27 -04:00
Gary Russell
9c6000e453 Kafka Outbound Endpoints - Add futuresChannel
Currently, the only way to block is to set `sync=true` which waits for the future.
The problem with this is it only supports one-at-a-time publication.

Add an option to send the send futures to a channel, allowing the application to
send multiple records and then wait on the futures later.

Also fix long-running test.

* Add docs.
2020-10-16 14:37:37 -04:00
Artem Bilan
4027adc392 Fix MeterRegistry eager load
If there is a `MeterRegistry` bean (any) in the application context,
we add a `MicrometerMetricsCaptor` bean which populates meters further
from the components.
In some case we may load a `MicrometerMetricsCaptor` bean too early
so, not all the stuff around `MeterRegistry` maybe ready.
See Spring Boot and its `MetricsAutoConfiguration`

* Fix the `MicrometerMetricsCaptorRegistrar` the way to rely on the
`ObjectProvider<MeterRegistry>` instead
* Add package protected ctor to the `MicrometerMetricsCaptor` to
provide a target `MeterRegistry` on demand

All of that will ensure that we use an already post-processed `MeterRegistry`
including Spring Boot auto-configuration

**Cherry-pick to 5.3.x & 5.2.x**
2020-10-15 15:13:16 -04:00
Spring Buildmaster
0d8ea39c72 [artifactory-release] Next development version 2020-10-14 19:04:02 +00:00
Spring Buildmaster
c67f92b912 [artifactory-release] Release version 5.4.0-RC1 2020-10-14 19:03:57 +00:00
Artem Bilan
13b34e6d11 Upgrade dependencies(and Gradle); prepare release 2020-10-14 13:52:42 -04:00
rohan mukesh
b1a383060d Add Error Handling to RedisStreamMessageProducer
* Use 'onErrorContinue' to continue receiving messages from Stream
* Send an `ErrorMessage` to the provided `errorChannel` (if any)
* And `@Nullable` to some `MessageProducerSupport` API which definitely
may accept `null`
* Extract common `buildMessageFromRecord()` method in the `ReactiveRedisStreamMessageProducer`,
so all the headers from the stream `Record` are carried to the message independently of the
 record state - normal send or error sending
2020-10-14 13:36:01 -04:00
Artem Bilan
5b74db8417 Upgrade dependencies; improve Redis Streams tests 2020-10-13 16:58:54 -04:00
Artem Bilan
3e501905e3 GH-3356: Add the way to not load mail message (#3402)
* GH-3356: Add the way to not load mail message

Fixes https://github.com/spring-projects/spring-integration/issues/3356

In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream

* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-10-12 17:36:22 -04:00
Artem Bilan
cadd4e6b7f Upgrade to the latest Reactor & SF
Related to https://github.com/spring-projects/spring-framework/issues/25884

* Don't use `MonoProcessor` & `FluxProcessor` since they are deprecated now
* Fix RSocket components to use an `AtomicReference` header instead of deprecated
`MonoProcessor`
* Introduce an internal `IntegrationRSocketPayloadReturnValueHandler` to handle
properly a Spring Integration case for inbound requests with its
`IntegrationRSocketEndpoint` implementation
* Don't do response handling in the `RSocketInboundGateway` any more
since it is deferred now to the `IntegrationRSocketPayloadReturnValueHandler`
* Remove internal `ChannelSendOperator` since it is out of use from now on
2020-10-12 11:24:39 -04:00
Artem Bilan
20447df7a2 USe different consumer group for no ack test 2020-10-08 13:52:53 -04:00