Commit Graph

10541 Commits

Author SHA1 Message Date
Gary Russell
a79be71ed2 GH-3410: Add UDP SocketCustomizer
Resolves https://github.com/spring-projects/spring-integration/issues/3410

**cherry-pick to 5.3.x**

* Doc polishing.
2020-10-22 13:32:56 -04:00
Artem Bilan
b94c20a0ce Gateway: Propagate Error to the errorChannel
See SO for more info:
https://stackoverflow.com/questions/64456946/handle-exceptions-errors-other-than-messagingexception-ie-other-error-excepti

In the versions before `5.2.x` the `Error` was wrapped to the `MessagingException`
in the `AbstractRequestHandlerAdvice` and this one was handled properly
on the gateway level with its `errorChannel` configured

* Fix `MessagingGatewaySupport` to catch all the `Throwable` and try to send them as `ErrorMessage`
to the `errorChannel`.
Otherwise unwrap returned `MessagingException` and re-throw an `Error` as is to keep
the current behavior for non-handled exceptions

* Do not wrap `Error` into a `MessagingException` and re-throw as is if there is no `errorChannel`

**Cherry-pick to 5.3.x & 5.2.x**
2020-10-22 11:41:34 -04:00
Artem Bilan
fb35a736d6 Get correct next execution date from the trigger
With the `time-source` in Spring Cloud Stream we noticed
that first two items have the same (or close) value
when the `time-supplier` is used with the reactive output channel.
Turns out the `contactMap()` with `prefetch == 1` asks two
upstream elements immediately not waiting for a completion
for the fist one.
This way we were not able to update `triggerContext` for the proper
`lastCompletionTime`

* Change `AbstractPollingEndpoint` to `prefetch = 0` for the
"fair" upstream request
* Verify the behavior with the new
`ReactiveInboundChannelAdapterTests.testTimeSupplierConsistency()`
* Mark `ReactiveInboundChannelAdapterTests` as `@LongRunningTest`
since it is now pretty long waiting for all the 3 dates to verify
* Fix `InboundChannelAdapterAnnotationPostProcessor` to properly
register `MethodInvokingMessageSource` when we have more than one
`Supplier` with the `@InboundChannelAdapter` in the same configuration
class
2020-10-21 14:00:51 -04:00
Gary Russell
3cf66cc539 Fix previous commit 2020-10-19 11:48:18 -04:00
Gary Russell
b6d60bfd69 KPMH - Remove unused parameter 2020-10-19 11:32:33 -04:00
Gary Russell
75d14d32e7 Further Improve Kafka Test Run Time
Configurable `timeoutBuffer`.
2020-10-16 16:28:34 -04:00
Artem Bilan
a136b5b49a Fix What's New for the proper link to Kafka chapter 2020-10-16 14:43:27 -04:00
Gary Russell
9c6000e453 Kafka Outbound Endpoints - Add futuresChannel
Currently, the only way to block is to set `sync=true` which waits for the future.
The problem with this is it only supports one-at-a-time publication.

Add an option to send the send futures to a channel, allowing the application to
send multiple records and then wait on the futures later.

Also fix long-running test.

* Add docs.
2020-10-16 14:37:37 -04:00
Artem Bilan
4027adc392 Fix MeterRegistry eager load
If there is a `MeterRegistry` bean (any) in the application context,
we add a `MicrometerMetricsCaptor` bean which populates meters further
from the components.
In some case we may load a `MicrometerMetricsCaptor` bean too early
so, not all the stuff around `MeterRegistry` maybe ready.
See Spring Boot and its `MetricsAutoConfiguration`

* Fix the `MicrometerMetricsCaptorRegistrar` the way to rely on the
`ObjectProvider<MeterRegistry>` instead
* Add package protected ctor to the `MicrometerMetricsCaptor` to
provide a target `MeterRegistry` on demand

All of that will ensure that we use an already post-processed `MeterRegistry`
including Spring Boot auto-configuration

**Cherry-pick to 5.3.x & 5.2.x**
2020-10-15 15:13:16 -04:00
Spring Buildmaster
0d8ea39c72 [artifactory-release] Next development version 2020-10-14 19:04:02 +00:00
Spring Buildmaster
c67f92b912 [artifactory-release] Release version 5.4.0-RC1 2020-10-14 19:03:57 +00:00
Artem Bilan
13b34e6d11 Upgrade dependencies(and Gradle); prepare release 2020-10-14 13:52:42 -04:00
rohan mukesh
b1a383060d Add Error Handling to RedisStreamMessageProducer
* Use 'onErrorContinue' to continue receiving messages from Stream
* Send an `ErrorMessage` to the provided `errorChannel` (if any)
* And `@Nullable` to some `MessageProducerSupport` API which definitely
may accept `null`
* Extract common `buildMessageFromRecord()` method in the `ReactiveRedisStreamMessageProducer`,
so all the headers from the stream `Record` are carried to the message independently of the
 record state - normal send or error sending
2020-10-14 13:36:01 -04:00
Artem Bilan
5b74db8417 Upgrade dependencies; improve Redis Streams tests 2020-10-13 16:58:54 -04:00
Artem Bilan
3e501905e3 GH-3356: Add the way to not load mail message (#3402)
* GH-3356: Add the way to not load mail message

Fixes https://github.com/spring-projects/spring-integration/issues/3356

In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream

* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-10-12 17:36:22 -04:00
Artem Bilan
cadd4e6b7f Upgrade to the latest Reactor & SF
Related to https://github.com/spring-projects/spring-framework/issues/25884

* Don't use `MonoProcessor` & `FluxProcessor` since they are deprecated now
* Fix RSocket components to use an `AtomicReference` header instead of deprecated
`MonoProcessor`
* Introduce an internal `IntegrationRSocketPayloadReturnValueHandler` to handle
properly a Spring Integration case for inbound requests with its
`IntegrationRSocketEndpoint` implementation
* Don't do response handling in the `RSocketInboundGateway` any more
since it is deferred now to the `IntegrationRSocketPayloadReturnValueHandler`
* Remove internal `ChannelSendOperator` since it is out of use from now on
2020-10-12 11:24:39 -04:00
Artem Bilan
20447df7a2 USe different consumer group for no ack test 2020-10-08 13:52:53 -04:00
Artem Bilan
1df4f92cfd Stop channel adapter and reset conn between tests 2020-10-08 13:27:13 -04:00
Artem Bilan
04296664e3 One more attempt for Redis Streams test 2020-10-08 13:07:48 -04:00
Artem Bilan
20348ae38f Fix compatibility with the latest Reactor
* Fix missed long-running tests according components under testing changes
2020-10-08 12:41:35 -04:00
Artem Bilan
d020d7abfb Increase consume timeout for Redis Streams test 2020-10-08 11:45:56 -04:00
Artem Bilan
6c2498de34 AFTER_EACH_TEST_METHOD for Redis Streams tests 2020-10-08 11:26:49 -04:00
Artem Bilan
dbbd855fb8 More Redis Streams tests tweaks 2020-10-08 11:04:30 -04:00
Artem Bilan
68379b6fb1 Some Redis Streams tests improvements 2020-10-08 10:35:26 -04:00
rohan mukesh
6d6f53fa87 Fix r2bdc componentType; Redis Streams ack issue
The R2DBC channel adapters have wrong componentType

* Fix R2DBC component type to not mention `reactive-`: they are reactive by R2DBC definition

The `ReactiveRedisStreamMessageProducer` calls ack() explicitly
when populates `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header,
but must provide an implementation of `SimpleAcknowledgment` instead

* Fix `ReactiveRedisStreamMessageProducer` to populate the proper
`IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header and in the proper moment
* Fix `ReactiveRedisStreamMessageProducerTests` to verify that `autoAck=false` works as expected
together with the `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header functionality
2020-10-07 17:06:49 -04:00
Artem Bilan
94e6521510 Fix some new Sonar smells 2020-10-07 12:31:44 -04:00
Artem Bilan
d8dd19106c Fix some unchecked warnings in tests 2020-10-07 11:54:21 -04:00
Artem Bilan
bc63a0dac8 More LogAccessor; some new Sonar smells 2020-10-06 16:31:43 -04:00
Artem Bilan
fe0e54c46d Fix AMQP & MQTT tests for LogAccessor 2020-10-06 14:41:32 -04:00
Artem Bilan
c7ff99a4e8 Use LogAccessor from SF
* Change main classes to use a `LogAccessor` API to simplify code flow
* Fix tests according `LogAccessor` property
* Fix some Sonar smells
2020-10-06 13:56:50 -04:00
Gary Russell
a66e82b0aa GH-3396: Add Docs for Micrometer MeterFilters
Resolves https://github.com/spring-projects/spring-integration/issues/3396

* Add an example.
2020-10-02 14:32:44 -04:00
Gary Russell
cbe37e8942 GH-3395: Fix XML expression default for ARFOGateway
Resolves https://github.com/spring-projects/spring-integration/issues/3395

Outbound remote file gateway parser requires `expression` even though some
commands don't need or use it.

* Propagate the empty string for `expression` attribute
* Fix default (`payload`) expression logic in the `AbstractRemoteFileOutboundGateway`

**Cherry-pick to `5.3.x`**
2020-10-02 14:31:53 -04:00
Artem Bilan
c934adbd1b Various fixes
* Adapt to WebFlux deprecations: https://github.com/spring-projects/spring-framework/issues/25751
* Back to SNAPSHOTs
* Fix new Sonar smells
2020-09-28 15:38:16 -04:00
Alex Sherwin
0cad53abcf "no" to "not" typo fix 2020-09-28 11:42:06 -04:00
Artem Vozhdayenko
d75dd095a3 GH-3194: Add generics to Amqp and Jms channel impls
Fixes https://github.com/spring-projects/spring-integration/issues/3194

For now we keep the hierarchy, but try to generalize channel specs
a bit, so that ones who use `publishSubscribeChannel` can count on
`BroadcastCapableChannel` as interface to utilize.
2020-09-25 14:55:10 -04:00
Artem Vozhdayenko
a8430c12a6 GH-3306: Remove setters from registration object
Fixes https://github.com/spring-projects/spring-integration/issues/3306

To prevent API misuse (one can set ID after doing a registration which
will do nothing actually), it is safer to remove set* methods from
`StandardIntegrationFlowRegistration` object.
Providing some related refactoring like making fields 'final', 
adjusting usages to keep correct behavior of the builder.
2020-09-25 14:48:52 -04:00
Artem Bilan
a62a7d1ddd INT-3045: Add in & out ZeroMq channel adapters (#3388)
* INT-3045: Add in & out ZeroMq channel adapters

JIRA: https://jira.spring.io/browse/INT-3045

* Add `ZeroMqMessageHandler` to produce messages into one-way ZeroMq sockets
* Add `ZeroMqMessageProducer` to consumer messages from one-way ZeroMq sockets
* Add `ConvertingBytesMessageMapper` impl for the `BytesMessageMapper` to
delegate an actual conversion into the provided `MessageConverter`
* Add `ZeroMqHeaders` for message headers constants representing ZeroMq message attributes
* Fix `ZeroMqChannel` for the proper deferred `zeroMqProxy` evaluation
* Add more JavaDocs
* Fix `ZeroMqChannelTests.testPubSubBind()` to be sure that really all the
subscribed channels get the same message from the `PUB` socket

* * Fix typo in the `ConvertingBytesMessageMapper`
* Add `this` for `doOnError()` in the `ZeroMqChannel` & `ZeroMqMessageProducer`
* Change the bind logic in the `ZeroMqMessageProducer` to `port` and let it to
bind to random port.
The actual port is available later via `getBoundPort()`
* Introduce a `ZeroMqMessageProducer.receiveRaw()` to let received `ZMsg` to
be produce as a `payload`
* Add a logic into `ZeroMqMessageHandler` to treat `ZMsg` in the payload of
request message as is without any conversion
* Fix race condition in the `ZeroMqMessageProducer` to destroy `consumerScheduler`
when the main `Flux` is complete

* * Add Java DSL for ZeroMq components
* Extract `ReactiveMessageHandlerSpec` for `ReactiveMessageHandler` impls
* Add debug message into `EmbeddedJsonHeadersMessageMapper` when cannot `decodeNativeFormat()`
* Make `ReactiveMongoDbMessageHandlerSpec` extending `ReactiveMessageHandlerSpec`
* Make `ZeroMqProxy` `autoStartup` by default
* Add `ZeroMqDslTests` to cover all the Java DSL for ZeroMq
* Introduce a `MimeTypeSerializer`  to serialize a `MimeType` into JSON as a plain string;
use it as extra serializer in the `JacksonJsonUtils.messagingAwareMapper()`
* Fix typo for the `AllowListTypeResolverBuilder` inner class

* * Add some docs
* Fix Checkstyle violations

* * More docs

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-09-25 13:46:01 -04:00
Artem Bilan
9d34cfd4dd Fix new Sonar smell 2020-09-25 11:21:03 -04:00
Artem Bilan
c94eaf7b52 Fix IntegrationReactiveUtils
The `Mono.doOnSuccess()` is always called for completed `MonoSink`
even if the value is `null`

* Fix `IntegrationReactiveUtils.messageSourceToFlux()` to check a
message for `null` before calling `AckUtils.autoAck()`
* Add an `logger.error()` message when `doOnError()` is invoked for that `Mono`

**Cherry-pick to 5.3.x**
2020-09-24 09:47:07 -04:00
Artem Bilan
e714310dae GH-3386: Add method signature for gateway proxy (#3387)
* GH-3386: Add method signature for gateway proxy

Fixes https://github.com/spring-projects/spring-integration/issues/3386

All the gateway proxy method invokers are supplied with the same
bean name inherited from the proxy.

* Add method signature for proxy method bean to fine-grain the
management for those bean in the logs, message history and metrics

* * Use `Class.getSimpleName()` instead to make gateway names less verbose
2020-09-18 15:40:32 -04:00
Spring Buildmaster
e223d341f3 [artifactory-release] Next development version 2020-09-16 17:29:48 +00:00
Spring Buildmaster
e2d5517544 [artifactory-release] Release version 5.4.0-M3 2020-09-16 17:29:43 +00:00
Artem Bilan
a36377a8d8 Upgrade to SK-2.6.1 2020-09-16 12:24:24 -04:00
Artem Bilan
be82e572f5 Upgrade dependencies; fix deprecations
* Prepare for release
2020-09-16 11:03:31 -04:00
Artem Bilan
e154067a1f GH-3376: Remove gauges on application ctx close (#3377)
* GH-3376: Remove gauges on application ctx close

Fixes https://github.com/spring-projects/spring-integration/issues/3376

The `MeterRegistry` may request meters on application shutdown.
The gauges for channels, handlers and message sources don't make sense
at the moment since all those beans are going to be destroyed.

* Remove gauges for channel, handler and message source numbers from the
`IntegrationManagementConfigurer.destroy()`

**Cherry-pick to 5.3.x & 5.2.x**

* * Add `MicrometerImportSelector` to conditionally load
a  `MicrometerMetricsCaptorConfiguration` when `MeterRegistry`
is on class path.
* Make `MicrometerMetricsCaptorConfiguration.integrationMicrometerMetricsCaptor()`
 bean dependant on the `ObjectProvider<MeterRegistry>`
* Make `IntegrationManagementConfiguration.managementConfigurer()`
dependant on the `ObjectProvider<MetricsCaptor>`.
This way the `IntegrationManagementConfigurer` is destroyed before
`MeterRegistry` when application context is closed
* Deprecate `MicrometerMetricsCaptor.loadCaptor()` in favor of
`@Import(MicrometerImportSelector.class)`

* * Add `MicrometerMetricsCaptorRegistrar` to register a `MICROMETER_CAPTOR_NAME`
bean when `MeterRegistry` is on class path and no `MICROMETER_CAPTOR_NAME` bean yet.
* Make `IntegrationManagementConfiguration.managementConfigurer()`
dependant on the `ObjectProvider<MetricsCaptor>`.
This way the `IntegrationManagementConfigurer` is destroyed before
`MeterRegistry` when application context is closed
* Deprecate `MicrometerMetricsCaptor.loadCaptor()` in favor of
`@Import(MicrometerMetricsCaptorRegistrar.class)`
* Fix test to make a `MeterRegistry` bean as `static` since
`@EnableIntegrationManagement` depends on this bean definition now
2020-09-16 10:32:39 -04:00
Artem Bilan
7a97eb6e1c GH-3336: Change MongoDb Store sequence to long (#3385)
* GH-3336: Change MongoDb Store sequence to long

Fixes https://github.com/spring-projects/spring-integration/issues/3336

Turns out there are some scenarios where too many messages
are transferred through the message store, so `int` for
sequence is not enough as a type

* Change sequence to `long` to widen a sequence lifespan

* * Change MongoDb store to deal with `Number.longValue()`
instead of casting which doesn't work from `Integer` to `Long`.
This way we can keep an old sequence document with an `int`
type for value
* Documents with new `long` type for their sequence field are OK.
The `NumberToNumberConverter` has an effect converting `int` to `long`
properly.
2020-09-15 12:55:09 -04:00
Artem Bilan
7f330850d1 GH-3372: Expose (S)FTP remoteComparator for DSL
Fixes https://github.com/spring-projects/spring-integration/issues/3372

**Cherry-pick to 5.3.x & 5.2.x**
2020-09-14 16:49:49 -04:00
Artem Bilan
ee7ebf3530 GH-3373: Support IPV6 in AbstractInboundFileSynch
Fixes https://github.com/spring-projects/spring-integration/issues/3373

The `AbstractInboundFileSynchronizer` doesn't consider that
`hostPort` from `Session` could be in an IPv6 syntax

* Parse the `hostPort` from `Session` in a manner that only the last
`:` is treated as a port delimiter

**Cherry-pick to 5.3.x & 5.2.x**
2020-09-14 16:06:51 -04:00
Artem Bilan
19b59bbde8 Upgrade to Reactor 2020.0.0-RC1
* Handle `Emission.FAIL_ZERO_SUBSCRIBER` in the
`FluxMessageChannel` and `IntegrationReactiveUtils`
2020-09-14 11:59:20 -04:00
Artem Bilan
5bac9bf66e Fix some tests race conditions
* Fix unused import in the `IntegrationRequestMappingHandlerMapping`
* Fix deprecations from Reactor
* Fix race condition in the `AbstractCorrelatingMessageHandlerTests`:
 the discard message is sent much earlier than group is removed from the store.
Iterate group count call until it pass or 10 seconds timeout
* Remove list size assert in the `FtpServerOutboundTests`: looks like
it is not updated properly even if we have an expected content in the collection
* Increase timeout to assert remote files removal in the `FtpRemoteFileTemplateTests`
2020-09-12 11:58:57 -04:00