* INT-4377: aggregator groupTimeout as Date
JIRA: https://jira.spring.io/browse/INT-4377
Change the `groupTimeoutExpression` logic to let it to be evaluated to `Date`
instance for some fine-grained scheduling use-case, e.g. to determine a
scheduling moment from the group creation time (`timestamp`) instead of a
current message arrival
* Fix language in docs accoridng PR review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* INT-4444: Introduce `@Reactive` & `reactive()`
JIRA: https://jira.spring.io/browse/INT-4444
Right now the high-level API creates a `ReactiveStreamsConsumer`
only when the input channel is a `Publisher<?>` impl or target handler
is a `ReactiveMessageHandler`
* Add `@Reactive[] reactive()` attribute to messaging annotations
* Add `ConsumerEndpointSpec.reactive()`
Both options point to the same `ConsumerEndpointFactoryBean.setReactiveCustomizer()`
making the target endpoint always as a `ReactiveStreamsConsumer` independently of
the input channel and target handler
* Use the `Function` to customize a source `Flux` from the channel
* Test and document a new feature
* * Fix links in docs
* * Fix `ReactiveStreamsTests`
* * Rework `reactive()` attribute of messaging annotations ot a single `@Reactive` value
with default as `@Reactive(ValueConstants.DEFAULT_NONE)`
* Fix language in docs
* Fix `MessagingAnnotationUtils.resolveAttribute()` to use `requiredType.isInstance()`
instead of comparing classes since annotation instances are `Proxy` at runtime
Since Nashorn JavaScript Engine is deprecated in Java 11 (https://openjdk.java.net/jeps/335)
it is better do nothing with JavaScript in the project any more
* Replace JS script tests to some other languages
* Mentioned deprecation in the Docs
* Rework `scripting.adoc` for code snippet switches
* FileInbound DSL: Add `recursive` for convenience
Related to https://stackoverflow.com/questions/66171881/how-to-read-nested-txt-file-from-spring-integration-file
The `FileInboundChannelAdapterSpec` can be configured with an external `DirectoryScanner`,
but it sometimes becomes burden for end-users to extract a scanner bean and configure it
with all the required file filters
* Expose a `recursive(boolean)` option for better end-user experience
* Rework `FileTests` for JUnit 5
* Mention a new option in the docs
* * Restore accidentally removed code
* Restore special symbols in the `FileTests`
* Fix language in the docs according review
Having a bean as a `java.util.Properties` is not confusing
and may lead to some conflicts in the real application.
Plus it is not so easy to configure: need to know all
the possible integration properties - bad end-user experience
* Make an `IntegrationProperties` as a public POJO
for easy configuration of global properties
* Deprecate a presence of the `java.util.Properties`
* Leave framework-created `integrationGlobalProperties` as
a `Properties` instance for backward compatibility
* Fix tests to expose an `IntegrationProperties` bean instead
of deprecated `Properties`
* Fix docs according the change and current recommendations
* JMS-in-gateway: destination to requestDestination
Related to https://stackoverflow.com/questions/66174779/what-is-the-dsl-version-of-int-jmsinbound-gateway-request-destination
In Java DSL, the `JmsInboundGatewaySpec` expose a `destination()` option which is not correlated
what we have in XML with a `request-destination` and what we have with a `replyQueue` option
* Deprecate an existing `destination()` option in the `JmsInboundGatewaySpec`
in favor of newly introduced `requestDestination()`
* Clean up a bit `jms.adoc` and add some code block-switch sections to show
Java & Kotlin DSLs along side with existing XML configs
* Fix typo.
Co-authored-by: Gary Russell <grussell@vmware.com>
* Add MongoDbMessageSource UPDATE option
* Extract `AbstractMongoDbMessageSource` with common options and methods
for both `MongoDbMessageSource` and `ReactiveMongoDbMessageSource`
* Add an `updateExpression` option into MongoDb source implementations
* Implement respective `update` logic after fetching the data from the collection
* Cover both reactive and blocking updates with tests
* Add `MongoDbMessageSourceSpec` into Java DSL for MongoDb channel adapters
* Expose an `update` XML attribute for the `<int-mongo:inbound-channel-adapter>`
* Upgrade MongoDb driver for latest Spring Data compatibility
* Document a new feature
* Upgrade `mongodb.adoc` for code block switch whenever it is appropriate
* * Add a Kotlin sample for `MongoDb.outboundGateway()` DSL
* Apply suggestions from code review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Resolves https://github.com/spring-projects/spring-integration/issues/3490
- also fix double import of `jmx.adoc`
- also fix `redis.adoc`
* Remove redundant repo in `docs.gradle`
* Use `stylesheet.css` for `asciidoctorj` plugin config
* Fix `stylesheet.css` to import `spring.css` from the same dir
since they both are copied over there into the same dir
* Fix link for JMX chapter from the `metrics.adoc`
* Fix some inconsistency in the `amqp.adoc`
Resolves https://github.com/spring-projects/spring-integration/issues/3488
Resolves two problems:
- When changes are made deep in the directory tree, they were not detected because
the directory is in the metadata store and only passes the filter if a file
immediately under it is changed, changing the directory's timestamp.
This is solved by subclassing `AbstractDirectoryAwareFileListFilter`, allowing its
`alwaysAcceptDirectories` property to be set.
- Only the filename was used as a metadata key; causing problems if a file with the
same name appears multiple times in the tree.
This is solved with a new property on `AbstractDirectoryAwareFileListFilter` used by
the gateways to determine whether to filter the raw file names returned by the session
(previous behavior) or the full path relative to the root directory.
**cherry-pick to 5.4.x, 5.3.x**
* Some code style clean up
* GH-3464: Treat 0 as special for polling endpoint
Fixes https://github.com/spring-projects/spring-integration/issues/3464
The `maxMessagePerPoll <= 0` is considered as an unbound `receive()` call.
End-users would like to have a special treatment for `0` value -
skip the `receive()` call altogether for the current polling cycle.
* Change the logic for a scheduled poller to not call `pollForMessage()`
and just log an INFO when `maxMessagePerPoll == 0`
* Fix reactive poller to deal with `maxMessagePerPoll == 0` properly
* Expose `maxMessagesPerPoll` as a `@ManagedAttribute` to let it to be
modified via Control Bus and JMX
* Test and document a new behavior
* * Fix unused imports in the test class
* Fix language in docs accoridng PR review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3483: Fallback to RestTemplate.converters
Fixes https://github.com/spring-projects/spring-integration/issues/3483
When there is no reasonable way to determine a `Content-Type`
from the request message, do not set an `application/x-java-serialized-object`
as a fallback and let the `RestTemplate` to determine the target type and
conversion through its `HttpMessageConverter` set
* Remove `application/x-java-serialized-object` fallback from the
`AbstractHttpRequestExecutingMessageHandler`
* Adjust its log messages according `LogAccessor`
* Un`@Disable` `WebFluxDslTests` since fix was done in Spring Security
* Add `HttpRequestExecutingMessageHandlerTests.testNoContentTypeAndSmartConverter()`
* Mention change in `What's New`
* Fix language in whats-new.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Expose `error.ignoreFailures` global property
The global default `errorChannel` can have more then only one logging subscriber.
There is no guarantee in which order subscribers are called and one of them fails
the rest are not called.
* Expose `spring.integration.channels.error.ignoreFailures` global property for
default `errorChannel` and make it `true` by default for better end-user experience
when it is expected to have error being handled by all the subscribers
* * Fix language in the `whats-new.adoc`
* * Fix `JmsTests` for error channel to propagate an exception
* More language fixes for whats-new.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3446
* For better resources utilization provide a `Stream<Message<?>>` API
on the `MessageGroupStore`, `MessageGroup` and `MessageGroupQueue`
* Use this API in the `DelayHandler` when it reschedules persisted messages
Fixes https://github.com/spring-projects/spring-integration/issues/3439
The latest Spring Data Redis has introduced an `onErrorResume` function
option for the `StreamReceiver` and this one is now recommended way
to handle errors in the `Flux` from this receiver
* Expose all the `StreamReceiver.StreamReceiverOptionsBuilder` option
onto the `ReactiveRedisStreamMessageProducer`, including `onErrorResume`
* Have a default function as it was before - send into an error channel
supporting (n)ack in the failed message based on the failed record
* Make new setters mutually exclusive with an explicit `StreamReceiver.StreamReceiverOptions`
* Doc polishing
* GH-3437: Add `MessageRecoverer` to AMQP Inbounds
Fixes https://github.com/spring-projects/spring-integration/issues/3437
For better end-user experience with AMQP Inbound Endpoints, expose
a `MessageRecoverer` option
* Test the feature and document this new option
* * Fix Checkstyle & update Copyright
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3450: Expose PubSub requireSubscribers option
Fixes https://github.com/spring-projects/spring-integration/issues/3450
* Add `PublishSubscribeChannel` ctors with the `requireSubscribers` option
with a direct delegation to the same option of underlying `BroadcastingDispatcher`
* Add factory methods to the `Channels` & `MessageChannels` to expose
this new `requireSubscribers` option
* Expose an XML `require-subscribers` attribute for the `<publish-subscribe-channel`
* Document this new `requireSubscribers` option
* Introduce a global `spring.integration.channels.error.requireSubscribers` property
for a default `errorChannel`
* Document this property and explain its default `true` in the `error-handling.adoc`
* Remove docs for `spring.integration.postProcessDynamicBeans` since it purpose was
removed since version `5.1` in favor of "always post-process behavior" as it was
always with all the `BeanPostProcessor`s
* * Fix language in docs according review
* * Add more docs about `requireSubscribers` and cross-links between chapters
* GH-3454: From MQTT conversion error - to error ch
Fixes https://github.com/spring-projects/spring-integration/issues/3454
The message converter may return null when we try to covert from the
MQTT message.
The thrown exception may also reset the client connect.
* Fix `MqttPahoMessageDrivenChannelAdapter` to catch any conversion errors
(including `null` result) and try to send an `ErrorMessage` with that info
into the provided `errorChannel`.
Otherwise re-throw it as as
**Cherry-pick to `5.4.x` & `5.3.x`**
* * Apply review language-specific changes
* GH-3452: Warn Publisher.onError in NullChannel
Fixes https://github.com/spring-projects/spring-integration/issues/3452
The `NullChannel` can subscribe to the `Publisher` payload fully ignoring
the possible data according the `NullChannel` nature.
However an error thrown from the reactive stream processing is also ignored.
* Log WARN message from the `Subscriber.onError()` when `NullChannel`
subscribes to the `Publisher` produced to this channel.
* Mention the logic in the `NullChannel` docs; point to the
`ReactiveRequestHandlerAdvice` for further possible error handling in
the target application
**Cherry-pick to `5.4.x`**
* * Remove unused imports
* * Fix `PseudoTransactionalMessageSourceTests` for `LogAccessor`
from `NullChannel`
To avoid extra `Publisher` subscription work in the target
application "flatten" a returned empty `Flux` for the `void`
expected type into a `Mono.empty()` for automatic subscription
on the output channel
* Subscribe to Publisher in the NullChannel
If a payload of the message in the flow is a `org.reactivestreams.Publisher`,
it must be subscribed somewhere downstream to initiate reactive processing.
The `NullChannel` just ignores the message altogether and therefore `Publisher`
is lost
* Check the payload of the message in the `NullChannel` for `Publisher` type
and subscribe to it
* Verify in the test and mention this logic in the docs
**Cherry-pick to 5.4.x**
* Apply suggestions from code review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Replace self-closing `<hr/>` with plain `<hr>` in the `overview.html`;
add `lang="en"` to avoid warning
* Use Java 15 API link for JavaDocs
* Use `/current` API link for all the required Spring dependencies
* Fix compatibility with upgraded dependencies
* Move `What's New` for `5.4` to `history`
* Add `XmlUnit` dependency since the latest AssertJ has deprecated its `isXmlEqualTo()`
* Remove `javax.annotation-api` and `apiguardian-api` as redundant dependencies
Fixes https://github.com/spring-projects/spring-integration/issues/3394
* Fix (wrong) copied/pasted text in the Redis Queue Inbound Gateway attribute
descriptions
* Fix some docs typos
* Add `jmx.adoc` into an `index.adoc` and `index-single.adoc`
* Add R2DBC, Redis Stream and ZeroMQ channel adapters into an `endpoint-summary.adoc`
Currently, the only way to block is to set `sync=true` which waits for the future.
The problem with this is it only supports one-at-a-time publication.
Add an option to send the send futures to a channel, allowing the application to
send multiple records and then wait on the futures later.
Also fix long-running test.
* Add docs.
* GH-3356: Add the way to not load mail message
Fixes https://github.com/spring-projects/spring-integration/issues/3356
In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream
* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* INT-3045: Add in & out ZeroMq channel adapters
JIRA: https://jira.spring.io/browse/INT-3045
* Add `ZeroMqMessageHandler` to produce messages into one-way ZeroMq sockets
* Add `ZeroMqMessageProducer` to consumer messages from one-way ZeroMq sockets
* Add `ConvertingBytesMessageMapper` impl for the `BytesMessageMapper` to
delegate an actual conversion into the provided `MessageConverter`
* Add `ZeroMqHeaders` for message headers constants representing ZeroMq message attributes
* Fix `ZeroMqChannel` for the proper deferred `zeroMqProxy` evaluation
* Add more JavaDocs
* Fix `ZeroMqChannelTests.testPubSubBind()` to be sure that really all the
subscribed channels get the same message from the `PUB` socket
* * Fix typo in the `ConvertingBytesMessageMapper`
* Add `this` for `doOnError()` in the `ZeroMqChannel` & `ZeroMqMessageProducer`
* Change the bind logic in the `ZeroMqMessageProducer` to `port` and let it to
bind to random port.
The actual port is available later via `getBoundPort()`
* Introduce a `ZeroMqMessageProducer.receiveRaw()` to let received `ZMsg` to
be produce as a `payload`
* Add a logic into `ZeroMqMessageHandler` to treat `ZMsg` in the payload of
request message as is without any conversion
* Fix race condition in the `ZeroMqMessageProducer` to destroy `consumerScheduler`
when the main `Flux` is complete
* * Add Java DSL for ZeroMq components
* Extract `ReactiveMessageHandlerSpec` for `ReactiveMessageHandler` impls
* Add debug message into `EmbeddedJsonHeadersMessageMapper` when cannot `decodeNativeFormat()`
* Make `ReactiveMongoDbMessageHandlerSpec` extending `ReactiveMessageHandlerSpec`
* Make `ZeroMqProxy` `autoStartup` by default
* Add `ZeroMqDslTests` to cover all the Java DSL for ZeroMq
* Introduce a `MimeTypeSerializer` to serialize a `MimeType` into JSON as a plain string;
use it as extra serializer in the `JacksonJsonUtils.messagingAwareMapper()`
* Fix typo for the `AllowListTypeResolverBuilder` inner class
* * Add some docs
* Fix Checkstyle violations
* * More docs
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3386: Add method signature for gateway proxy
Fixes https://github.com/spring-projects/spring-integration/issues/3386
All the gateway proxy method invokers are supplied with the same
bean name inherited from the proxy.
* Add method signature for proxy method bean to fine-grain the
management for those bean in the logs, message history and metrics
* * Use `Class.getSimpleName()` instead to make gateway names less verbose
Fixes https://github.com/spring-projects/spring-integration/issues/3367
Introduce a `requestTimeout` and `triggerTimeout` for `BarrierMessageHandler`
For instance, if an HTTP request sends a message to the barrier,
it should time out after 1min if no trigger message is received.
If the trigger message then arrives late and the HTTP request is no longer waiting,
it shouldn't wait for 1min before discarding the request but do so immediately.
* INT-3045: Add `ZeroMqChannel` support
JIRA: https://jira.spring.io/browse/INT-3045
Provide a `SubscribableChannel` implementation for ZeroMQ
The general idea is to let to have a distributed channel implementation
where every client can connect to a single server backed by the channel.
The logic in the channel is fully transparent for end-user and there is just
enough to send message to it and subscribe for receiving on the other side.
If PUB/SUB model is used, all the subscribes (even over the network) going to
receive the same published message.
In case of PUSH/PULL only one subscriber in the whole cluster is going to get
the published message
* Use Reactor for better threading control
* JeroMQ is not interruptible-friendly: use control sockets to stop proxy loop
* Name Reactor's schedulers to avoid daemon threads
* * Use try-catch-with-resource to close sockets automatically
* Fix Checkstyle violations
* Use `Mono.handle()` to receive data from the socket
* * Optimize local for just a couple of PAIR sockets
* Implement TCP binding
* Add PUB/SUB tests
* * Fix subscriber scheduler name
* Optimize socket create logic
* Add PUSH/PULL over TCP test
* * Fix subscriber scheduler name
* Optimize socket create logic
* Add PUSH/PULL over TCP test
* Implement PUB/SUB over TCP
* * Introduce `ZeroMqProxy` - Spring-friendly component to configure and manage ZeroMq proxy
* Use this `ZeroMqProxy` logic as an external component for `ZeroMqChannel` testing
* * Fix Checkstyle
* Apply docs polishing
* Expose a capture socket on the proxy
* Implement `DisposableBean` in the `ZeroMqProxy` to destroy an internal executor service
* Add JavaDocs to `ZeroMqChannel`
* Add one more `ZeroMqChannel` to TCP test to be sure that proxy distribution works well
* * Add `hamcrest-core` dependency for Awatility
* * Add more JavaDocs to `ZeroMqProxy` and `ZeroMqChannel`
* Expose `ZeroMqChannel.setZeroMqProxy()` option for easier
configuration within the same application context
* Make `ZeroMqChannel` sockets configuration and connection
dependant on provided `ZeroMqProxy` (if any)
* Add `Consumer<ZMQ.Socket>` configuration callbacks to the `ZeroMqChannel`
* Expose `ZeroMqChannel.consumeDelay` option
* * Add docs for ZeroMQ
* Some additions into a `reactive-streams.adoc`
* Fix typo in the `xmpp.adoc`
* * Add `optional` `jackson-databind` since `ZeroMqChannel` uses it by default
* More words into docs
* * Fix language in docs according review
* Fix language in docs according review
Co-authored-by: Gary Russell <grussell@vmware.com>
* Apply suggestions from code review
Co-authored-by: Oliver <oli-ver@users.noreply.github.com>
* * Fix threading using a `publishOn()` for specific scheduler after `cache()`
* * Remove unused import
* * Change proxy port check from static `Mono.just()` to `Mono.fromCallable()`
to really evaluate the current port state on every repeat
* Add finite `100` repeat number to avoid infinite blocking when proxy is not started at all
* Add `doOnError()` for proxy `Mono` to log `ERROR` when repeat is exhausted
* * Fix Checkstyle violation
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Oliver <oli-ver@users.noreply.github.com>
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
implement `ManageableLifecycle`. Register an additional MBean for
polled endpoints to control the lifecycle.
* Polishing
- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
In preparation for https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/issues/303
Now that `CorrelationData` has a `Future<?>`, users might simply add
correlation data in a header and not receive confirm/return messages.
- No longer require channels for returns and confirms
- don't build the confirm message if there are no channels
- reduce the log level for no channels to DEBUG
- complete the user's future when a message is returned (async GW)
* Fix typo.
