Fixes https://github.com/spring-projects/spring-integration/issues/3451
The GreenMail provides a comprehensive support for e-mail scenarios
testing.
* Use GreenMail instead our own `TestMailServer` for better and broader
support
* Convert SmtpTests and Pop3Tests to GreenMail
* Document that Pop3MailReceiver does not support autoCloseFolder=false
* Pop3MailReceiver uses `folder.getMessageCount()` to determine if there
are new messages.
This count only gets updated when opening the folder.
* Use GreenMail for remaining mail tests
* convert MailTests and ImapMailReceiverTests to GreenMail
* remove duplicate mail dependencies
* route com.sun.mail logging to log4j (via jul and slf4j)
* INT-1828: Docs about filter & release strategy
JIRA: https://jira.spring.io/browse/INT-1828
Document a corner case when filter endpoint is involved in
the flow logic before an aggregator with sequence size release strategy
* Fix typos in the doc changes
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3504: Explain more an async error handling
Fixes https://github.com/spring-projects/spring-integration/issues/3504
The reaper for an aggregator (as we as a group timeout) are based on the
`TaskScheduler`.
If non-default `TaskScheduler` is used in the configuration, recommend to
supply it with a `MessagePublishingErrorHandler`
* Rewording changes for proper language
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Currently the `MessageHistoryRegistrar` can parse several sources for message history -
`@EnableMessageHistory` and/or `<message-history>`.
Since its logic relies on the reflection it is not compatible with Spring Native.
Plus it causes confusion when several sources are declared so when time comes to
change something in that configuration, we may miss some place to re-align with
our new requirements.
Better to reject extra configurations and enforce end-users to use only one
`@EnableMessageHistory` or `<message-history>`.
This is actually a preferences for many other `@Enable...` in Spring portfolio.
Plus we got a benefit with a Spring Native compatibility
* Fix `MessageHistoryRegistrar` to parse only one `@EnableMessageHistory`.
Register `MessageHistoryConfigurer` function way for Spring Native compatibility
* Clean up `PublisherRegistrar` for better readability
* Fix message history tests which exposed several configurations
* Add JavaDoc into `EnableMessageHistory`
* Refactor `MessageHistoryConfigurer` to let to override patterns configuration
at runtime
* Clean up `message-history.adoc`
* GH-3533: Register WebSocket endpoints at runtime
Fixes https://github.com/spring-projects/spring-integration/issues/3533
* Rework `WebSocketIntegrationConfigurationInitializer` to register beans functional way
to avoid reflection for Spring Native support
* Move `IntegrationServletWebSocketHandlerRegistry` into a separate file for better readability
* Implement `DestructionAwareBeanPostProcessor` for `IntegrationServletWebSocketHandlerRegistry`
to track runtime bean registrations and removals
* Introduce an `IntegrationDynamicWebSocketHandlerMapping` to manage runtime mapping
registrations and removals
* Add `servlet-api` dependency into `websocket` to be able to compile an
`IntegrationDynamicWebSocketHandlerMapping`
* Fix typo in the exception message of the `StandardIntegrationFlowRegistration`
* Start dynamically added beans together with associated `IntegrationFlow` in the
`StandardIntegrationFlowContext`
* Document new feature
* * Fix language in docs
* Don't start those `SmartLifecycle`s together with a dynamic flow
which are not `isAutoStartup()`
* * Fix `TomcatWebSocketTestServer` to configure servlet for `loadOnStartup = 1`
* Fix `WebSocketDslTests` to make `clientWebSocketContainer.setAutoStartup(true)`
* GH-3529: Add HTTP & WebFlux extractResponseBody
Fixes https://github.com/spring-projects/spring-integration/issues/3529
* Expose a convenient `extractResponseBody` option on the HTTP client components
to let end-user to decide if the body of `ResponseEntity` must be extracted (default)
or the whole `ResponseEntity` should be produced as a reply message payload
* Remove a deprecated since `5.3` `encode-uri` option
* Document the new feature
* Rework `webflux.adoc` chapter for the code snippet switcher
* * Fix language in Docs
* Mention a default value in JavaDocs
* GH-3521: Delayer: schedule release task with TX
Fixes https://github.com/spring-projects/spring-integration/issues/3521
There is a race condition when transactional `MessageStore` is used
for `DelayHandler`, so the message is not visible for reads until after TX is
committed, but a scheduled release task may be already ready after delay
* Register a `TransactionSynchronization` with scheduling a releasing task
when TX is committed
**Cherry-pick to `5.4.x` & `5.3.x`**
* Fix language in delayer.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Introduce a `groupConditionSupplier` for MGS
* Add a `MessageGroup.condition` option
* Add a `MessageGroupStore.conditionSupplier` option
* Use it from the `SimpleMessageStore.addMessagesToGroup()` API
to populate `condition` (if any) into a `MessageGroup`
* Introduce a `GroupConditionProvider` contract to be implemented
on those `ReleaseStrategy` contracts which could be aware of group condition
* Populated a `GroupConditionProvider.getGroupConditionSupplier()`
into a `MessageGroupStore` from the `AbstractCorrelatingMessageHandler`
for end-user convenience
* Rework a `FileMarkerReleaseStrategy` to implement a `GroupConditionProvider`
to provide a function which produces a condition from `file_lineCount` header
of the `END` marker message
* Make the `FileMarkerReleaseStrategy` logic already based on the condition from a group
* Delegate `GroupConditionProvider` from the `FileAggregator`
* Add test for empty file aggregation
* * Implement `condition` in the `AbstractKeyValueMessageStore` and `MongoDbMessageStore`
* Test `condition` for `mongo-aggregator-config.xml` and `FileAggregatorTests` against GemFire
* * Implement `condition` in the `ConfigurableMongoDbMessageStore`
* * Implement `condition` in the `JdbcMessageStore`
* `FileAggregatorTests` against `JdbcMessageStore`
* Refactor `JdbcMessageStore` for better handling of message group metadata
* Remove unused `MARKED` column in the DDL in favor of newly introduced `CONDITION`
* * Add docs for message group condition
* * Move `conditionSupplier` option from MGS to AbstractCorrelatingMH
* Make it as a `BiFunction` to propagate existing condition alongside with the
message to consult
* Expose `groupConditionSupplier` in Java & XML DSLs
* * Fix language in docs
* INT-4116: Introduce FileAggregator
JIRA: https://jira.spring.io/browse/INT-4116
* Implement a `FileSplitter.FileMarker`-based aggregation strategies
and utilize them in a general `FileAggregator` component
* Make `HeaderAttributeCorrelationStrategy.attributeName` as `final`; add `Assert.notEmpty()`
* Fix `AggregatorFactoryBean` and `AggregatorSpec` to parse the provided processor
for possible `CorrelationStrategy` and/or `ReleaseStrategy`
* Introduce short-cut methods into Java & Kotlin DSL for an `aggregate()` configuration
* Introduce a `FileHeaders.LINE_COUNT` for header to be populated in the `FileSplitter`.
We need this info in the `FileAggregator` to avoid possible overhead with JSON deserialization
of the `FileSplitter.FileMarker` messages
* Test and document the feature
* Improve `FileSplitter` doc for code block switch (tabs)
* * Rework FileAggregator do not use Java Streams
* INT-4377: aggregator groupTimeout as Date
JIRA: https://jira.spring.io/browse/INT-4377
Change the `groupTimeoutExpression` logic to let it to be evaluated to `Date`
instance for some fine-grained scheduling use-case, e.g. to determine a
scheduling moment from the group creation time (`timestamp`) instead of a
current message arrival
* Fix language in docs accoridng PR review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* INT-4444: Introduce `@Reactive` & `reactive()`
JIRA: https://jira.spring.io/browse/INT-4444
Right now the high-level API creates a `ReactiveStreamsConsumer`
only when the input channel is a `Publisher<?>` impl or target handler
is a `ReactiveMessageHandler`
* Add `@Reactive[] reactive()` attribute to messaging annotations
* Add `ConsumerEndpointSpec.reactive()`
Both options point to the same `ConsumerEndpointFactoryBean.setReactiveCustomizer()`
making the target endpoint always as a `ReactiveStreamsConsumer` independently of
the input channel and target handler
* Use the `Function` to customize a source `Flux` from the channel
* Test and document a new feature
* * Fix links in docs
* * Fix `ReactiveStreamsTests`
* * Rework `reactive()` attribute of messaging annotations ot a single `@Reactive` value
with default as `@Reactive(ValueConstants.DEFAULT_NONE)`
* Fix language in docs
* Fix `MessagingAnnotationUtils.resolveAttribute()` to use `requiredType.isInstance()`
instead of comparing classes since annotation instances are `Proxy` at runtime
Since Nashorn JavaScript Engine is deprecated in Java 11 (https://openjdk.java.net/jeps/335)
it is better do nothing with JavaScript in the project any more
* Replace JS script tests to some other languages
* Mentioned deprecation in the Docs
* Rework `scripting.adoc` for code snippet switches
* FileInbound DSL: Add `recursive` for convenience
Related to https://stackoverflow.com/questions/66171881/how-to-read-nested-txt-file-from-spring-integration-file
The `FileInboundChannelAdapterSpec` can be configured with an external `DirectoryScanner`,
but it sometimes becomes burden for end-users to extract a scanner bean and configure it
with all the required file filters
* Expose a `recursive(boolean)` option for better end-user experience
* Rework `FileTests` for JUnit 5
* Mention a new option in the docs
* * Restore accidentally removed code
* Restore special symbols in the `FileTests`
* Fix language in the docs according review
Having a bean as a `java.util.Properties` is not confusing
and may lead to some conflicts in the real application.
Plus it is not so easy to configure: need to know all
the possible integration properties - bad end-user experience
* Make an `IntegrationProperties` as a public POJO
for easy configuration of global properties
* Deprecate a presence of the `java.util.Properties`
* Leave framework-created `integrationGlobalProperties` as
a `Properties` instance for backward compatibility
* Fix tests to expose an `IntegrationProperties` bean instead
of deprecated `Properties`
* Fix docs according the change and current recommendations
* JMS-in-gateway: destination to requestDestination
Related to https://stackoverflow.com/questions/66174779/what-is-the-dsl-version-of-int-jmsinbound-gateway-request-destination
In Java DSL, the `JmsInboundGatewaySpec` expose a `destination()` option which is not correlated
what we have in XML with a `request-destination` and what we have with a `replyQueue` option
* Deprecate an existing `destination()` option in the `JmsInboundGatewaySpec`
in favor of newly introduced `requestDestination()`
* Clean up a bit `jms.adoc` and add some code block-switch sections to show
Java & Kotlin DSLs along side with existing XML configs
* Fix typo.
Co-authored-by: Gary Russell <grussell@vmware.com>
* Add MongoDbMessageSource UPDATE option
* Extract `AbstractMongoDbMessageSource` with common options and methods
for both `MongoDbMessageSource` and `ReactiveMongoDbMessageSource`
* Add an `updateExpression` option into MongoDb source implementations
* Implement respective `update` logic after fetching the data from the collection
* Cover both reactive and blocking updates with tests
* Add `MongoDbMessageSourceSpec` into Java DSL for MongoDb channel adapters
* Expose an `update` XML attribute for the `<int-mongo:inbound-channel-adapter>`
* Upgrade MongoDb driver for latest Spring Data compatibility
* Document a new feature
* Upgrade `mongodb.adoc` for code block switch whenever it is appropriate
* * Add a Kotlin sample for `MongoDb.outboundGateway()` DSL
* Apply suggestions from code review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Resolves https://github.com/spring-projects/spring-integration/issues/3490
- also fix double import of `jmx.adoc`
- also fix `redis.adoc`
* Remove redundant repo in `docs.gradle`
* Use `stylesheet.css` for `asciidoctorj` plugin config
* Fix `stylesheet.css` to import `spring.css` from the same dir
since they both are copied over there into the same dir
* Fix link for JMX chapter from the `metrics.adoc`
* Fix some inconsistency in the `amqp.adoc`
Resolves https://github.com/spring-projects/spring-integration/issues/3488
Resolves two problems:
- When changes are made deep in the directory tree, they were not detected because
the directory is in the metadata store and only passes the filter if a file
immediately under it is changed, changing the directory's timestamp.
This is solved by subclassing `AbstractDirectoryAwareFileListFilter`, allowing its
`alwaysAcceptDirectories` property to be set.
- Only the filename was used as a metadata key; causing problems if a file with the
same name appears multiple times in the tree.
This is solved with a new property on `AbstractDirectoryAwareFileListFilter` used by
the gateways to determine whether to filter the raw file names returned by the session
(previous behavior) or the full path relative to the root directory.
**cherry-pick to 5.4.x, 5.3.x**
* Some code style clean up
* GH-3464: Treat 0 as special for polling endpoint
Fixes https://github.com/spring-projects/spring-integration/issues/3464
The `maxMessagePerPoll <= 0` is considered as an unbound `receive()` call.
End-users would like to have a special treatment for `0` value -
skip the `receive()` call altogether for the current polling cycle.
* Change the logic for a scheduled poller to not call `pollForMessage()`
and just log an INFO when `maxMessagePerPoll == 0`
* Fix reactive poller to deal with `maxMessagePerPoll == 0` properly
* Expose `maxMessagesPerPoll` as a `@ManagedAttribute` to let it to be
modified via Control Bus and JMX
* Test and document a new behavior
* * Fix unused imports in the test class
* Fix language in docs accoridng PR review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3483: Fallback to RestTemplate.converters
Fixes https://github.com/spring-projects/spring-integration/issues/3483
When there is no reasonable way to determine a `Content-Type`
from the request message, do not set an `application/x-java-serialized-object`
as a fallback and let the `RestTemplate` to determine the target type and
conversion through its `HttpMessageConverter` set
* Remove `application/x-java-serialized-object` fallback from the
`AbstractHttpRequestExecutingMessageHandler`
* Adjust its log messages according `LogAccessor`
* Un`@Disable` `WebFluxDslTests` since fix was done in Spring Security
* Add `HttpRequestExecutingMessageHandlerTests.testNoContentTypeAndSmartConverter()`
* Mention change in `What's New`
* Fix language in whats-new.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Expose `error.ignoreFailures` global property
The global default `errorChannel` can have more then only one logging subscriber.
There is no guarantee in which order subscribers are called and one of them fails
the rest are not called.
* Expose `spring.integration.channels.error.ignoreFailures` global property for
default `errorChannel` and make it `true` by default for better end-user experience
when it is expected to have error being handled by all the subscribers
* * Fix language in the `whats-new.adoc`
* * Fix `JmsTests` for error channel to propagate an exception
* More language fixes for whats-new.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3446
* For better resources utilization provide a `Stream<Message<?>>` API
on the `MessageGroupStore`, `MessageGroup` and `MessageGroupQueue`
* Use this API in the `DelayHandler` when it reschedules persisted messages
Fixes https://github.com/spring-projects/spring-integration/issues/3439
The latest Spring Data Redis has introduced an `onErrorResume` function
option for the `StreamReceiver` and this one is now recommended way
to handle errors in the `Flux` from this receiver
* Expose all the `StreamReceiver.StreamReceiverOptionsBuilder` option
onto the `ReactiveRedisStreamMessageProducer`, including `onErrorResume`
* Have a default function as it was before - send into an error channel
supporting (n)ack in the failed message based on the failed record
* Make new setters mutually exclusive with an explicit `StreamReceiver.StreamReceiverOptions`
* Doc polishing
* GH-3437: Add `MessageRecoverer` to AMQP Inbounds
Fixes https://github.com/spring-projects/spring-integration/issues/3437
For better end-user experience with AMQP Inbound Endpoints, expose
a `MessageRecoverer` option
* Test the feature and document this new option
* * Fix Checkstyle & update Copyright
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3450: Expose PubSub requireSubscribers option
Fixes https://github.com/spring-projects/spring-integration/issues/3450
* Add `PublishSubscribeChannel` ctors with the `requireSubscribers` option
with a direct delegation to the same option of underlying `BroadcastingDispatcher`
* Add factory methods to the `Channels` & `MessageChannels` to expose
this new `requireSubscribers` option
* Expose an XML `require-subscribers` attribute for the `<publish-subscribe-channel`
* Document this new `requireSubscribers` option
* Introduce a global `spring.integration.channels.error.requireSubscribers` property
for a default `errorChannel`
* Document this property and explain its default `true` in the `error-handling.adoc`
* Remove docs for `spring.integration.postProcessDynamicBeans` since it purpose was
removed since version `5.1` in favor of "always post-process behavior" as it was
always with all the `BeanPostProcessor`s
* * Fix language in docs according review
* * Add more docs about `requireSubscribers` and cross-links between chapters
* GH-3454: From MQTT conversion error - to error ch
Fixes https://github.com/spring-projects/spring-integration/issues/3454
The message converter may return null when we try to covert from the
MQTT message.
The thrown exception may also reset the client connect.
* Fix `MqttPahoMessageDrivenChannelAdapter` to catch any conversion errors
(including `null` result) and try to send an `ErrorMessage` with that info
into the provided `errorChannel`.
Otherwise re-throw it as as
**Cherry-pick to `5.4.x` & `5.3.x`**
* * Apply review language-specific changes
* GH-3452: Warn Publisher.onError in NullChannel
Fixes https://github.com/spring-projects/spring-integration/issues/3452
The `NullChannel` can subscribe to the `Publisher` payload fully ignoring
the possible data according the `NullChannel` nature.
However an error thrown from the reactive stream processing is also ignored.
* Log WARN message from the `Subscriber.onError()` when `NullChannel`
subscribes to the `Publisher` produced to this channel.
* Mention the logic in the `NullChannel` docs; point to the
`ReactiveRequestHandlerAdvice` for further possible error handling in
the target application
**Cherry-pick to `5.4.x`**
* * Remove unused imports
* * Fix `PseudoTransactionalMessageSourceTests` for `LogAccessor`
from `NullChannel`
To avoid extra `Publisher` subscription work in the target
application "flatten" a returned empty `Flux` for the `void`
expected type into a `Mono.empty()` for automatic subscription
on the output channel
* Subscribe to Publisher in the NullChannel
If a payload of the message in the flow is a `org.reactivestreams.Publisher`,
it must be subscribed somewhere downstream to initiate reactive processing.
The `NullChannel` just ignores the message altogether and therefore `Publisher`
is lost
* Check the payload of the message in the `NullChannel` for `Publisher` type
and subscribe to it
* Verify in the test and mention this logic in the docs
**Cherry-pick to 5.4.x**
* Apply suggestions from code review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Replace self-closing `<hr/>` with plain `<hr>` in the `overview.html`;
add `lang="en"` to avoid warning
* Use Java 15 API link for JavaDocs
* Use `/current` API link for all the required Spring dependencies
* Fix compatibility with upgraded dependencies
* Move `What's New` for `5.4` to `history`
* Add `XmlUnit` dependency since the latest AssertJ has deprecated its `isXmlEqualTo()`
* Remove `javax.annotation-api` and `apiguardian-api` as redundant dependencies
Fixes https://github.com/spring-projects/spring-integration/issues/3394
* Fix (wrong) copied/pasted text in the Redis Queue Inbound Gateway attribute
descriptions
* Fix some docs typos
* Add `jmx.adoc` into an `index.adoc` and `index-single.adoc`
* Add R2DBC, Redis Stream and ZeroMQ channel adapters into an `endpoint-summary.adoc`
Currently, the only way to block is to set `sync=true` which waits for the future.
The problem with this is it only supports one-at-a-time publication.
Add an option to send the send futures to a channel, allowing the application to
send multiple records and then wait on the futures later.
Also fix long-running test.
* Add docs.
* GH-3356: Add the way to not load mail message
Fixes https://github.com/spring-projects/spring-integration/issues/3356
In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream
* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
