Fixes: #9988
Issue link: https://github.com/spring-projects/spring-integration/issues/9988
This change allows dynamic determination of `FileExistsMode` using SpEL expressions,
making the component more flexible when handling file existence conflicts.
* Add `fileExistsModeExpression` field and setter methods
* Use `resolveFileExistsMode()` in put and get operations
* Add changes to the docs
* Ignore `temporaryFileName` when `FileExistsMode.APPEND`
Improve runtime behavior by ignoring temporary filename settings when file exists mode is `APPEND`.
Now, in `FileExistsMode.APPEND` mode, content is always appended directly to the original file regardless of `useTemporaryFileName` setting.
In `RemoteFileTemplate`:
- Remove exception validation when `APPEND` mode is used with temporary filenames
- Modify logic to skip applying `temporaryFileSuffix` in `APPEND` mode
In `AbstractRemoteFileOutboundGateway`:
- Remove logic that disabled temporary filenames when setting `APPEND` mode
* Apply review feedback on `FileExistsMode` expression
- Optimize `EvaluationContext` usage by creating it once in `doInit()`
- Enhance expression evaluation to support String representation of `FileExistsMode`
- Optimize temporary filename handling logic in `RemoteFileTemplate`
- Add warning message for incompatible `APPEND` mode with temporary filenames
- Rename method to `setFileExistsModeExpressionString` for consistency
- Update Java DSL support in `RemoteFileOutboundGatewaySpec`
- Update reference documentation and release notes
* Apply additional review feedback on `FileExistsMode` expression
- Add `Function` variant to `RemoteFileOutboundGatewaySpec`
- Update documentations to use one-sentence-per-line style
- Improve code flow in `resolveFileExistsMode()` method
* Fix additional review feedback on `FileExistsMode` expression
- Use `Object` instead of `String` in `fileExistsModeFunction`
- Fix return method call in `fileExistsModeFunction` (`remoteDirectoryExpression` -> `fileExistsModeExpression`)
- Fix `standardEvaluationContext` initialization in `doInit()`
- Ensure proper `EvaluationContext` usage in other methods
Signed-off-by: Jooyoung Pyoung <pyoungjy@gmail.com>
Fixes: https://github.com/spring-projects/spring-integration/issues/10018
The `GatewayEndpointSpec` configuration of the `gateway()` operator support already an `async(true)` option.
However, it is silently ignored internally since no real async contract provided for the gateway proxy.
* Introduce the `AsyncRequestReplyExchanger` interface to use instead of `RequestReplyExchanger`,
when `gateway()` operator is opted-in for the `async(true)`
* Use this new `AsyncRequestReplyExchanger` in the `GatewayMessageHandler` when `async(true)`
* Also, expose a `GatewayEndpointSpec.asyncExecutor(Executor)` option to support async behavior similar to the `@MessagingGateway`
Fixes: https://github.com/spring-projects/spring-integration/issues/9441
* Upgrade to Hazelcast `5.5`
* Deprecate `LeaderInitiator`, `HazelcastLockRegistry` since they cannot be used in Open Source due to Hazelcast CP Subsystem migration under commercial support
* Deprecate `HazelcastMembershipListener` and `HazelcastLocalInstanceRegistrar` in favor of Hazelcast `Cluster` API.
* Mention changes in the docs
Fixes: https://github.com/spring-projects/spring-integration/issues/8592
The `XADD` command for the record can be customized with a `RedisStreamCommands.XAddOptions`.
* Expose `ReactiveRedisStreamMessageHandler.setAddOptionsFunction(Function<Message<?>, RedisStreamCommands.XAddOptions> addOptionsFunction)`
to allow to configure those option against the specific request message.
Fixes: https://github.com/spring-projects/spring-integration/issues/6380
The `MapSqlParameterSource` is much faster, then reflection or SpEL,
so, it would be great to have such an interaction when we evaluate values for SQL queries
* Enhance `BeanPropertySqlParameterSourceFactory` to use `MapSqlParameterSource` if `input` is a `Map`
* Expose `JdbcMessageHandler.usePayloadAsParameterSource` for convenience with `SqlParameterSourceFactory`,
especially when the payload is a map
Fixes: https://github.com/spring-projects/spring-integration/issues/9441
The most important Hazelcast CP Subsystem has been migrated to the Enterprise Edition.
Therefore, we cannon provide Open Source support for feature based on those in the commercial Hazelcast.
Fixes: https://github.com/spring-projects/spring-integration/issues/9869
Currently, FileListFilters on the last modified attribute of a file are limited to the use case of discarding files that are too young.
It would be great to have a filter implementation which would accept files which are not old enough.
* Implement `AbstractRecentFileListFilter` for all the supported file protocols
Fixes: https://github.com/spring-projects/spring-integration/issues/9854
The well-known `Dispatcher has no subscribers` is not very informative
when a message is produced from early application context initialization phase
* Add internal `ApplicationRunningController` bean to handle early `start()` event
* Check for this bean status from the `AbstractMessageChannel.send()`
* Throw specific `MessageDispatchingException` to indicate that the message was produced from a wrong place
* Adjust `ApplicationEventListeningMessageProducer` logic for `ContextStoppedEvent` & `ContextClosedEvent`
to indicate that `AbstractMessageChannel` bean might not dispatch a message because the application context is not running
Fixes: https://github.com/spring-projects/spring-integration/issues/9801
Spring Integration Apache Kafka inbound channel adapters have always produced
messages without `ID` & `TIMESTAMP` headers.
See `MessagingMessageConverter` used over there by default.
However, for consistency with the rest of Spring Integration channel adapters,
it would be better to have Kafka-specific behave same way.
* Configure default `MessagingMessageConverter` in the `KafkaMessageSource` & `KafkaMessageDrivenChannelAdapter`
for generating `ID` & `TIMESTAMP` headers.
Fixes: https://github.com/spring-projects/spring-integration/issues/9754
Right now a correlation handler can discard messages in the expired group one by one.
In some scenarios it would be useful to have single message in discard for the whole group.
* Expose `discardIndividuallyOnExpiry` for the `AbstractCorrelatingMessageHandler`,
and `AggregatorFactoryBean`, and respective `CorrelationHandlerSpec` for DSL.
This new option takes action only if a `discardChannel` is provided,
and `sendPartialResultOnExpiry` is not set to `true`.
When `discardIndividuallyOnExpiry` is false, the messages in the expired group are packed
into a list for payload of a discarding single message.
* Test and document the new feature
Fixes: https://github.com/spring-projects/spring-integration/issues/9743
Spring Integration provides observation for the `MessageChannel`, `MessageHandler`
and `MessageProducerSupport`.
The `SourcePollingChannelAdapter` is missing, and it is that only special endpoint which
deals with `MessageSource` implementations via scheduled tasks in the poller.
Essentially, this endpoint is a start of the flow, but it still is a consumer of data from the source system.
* Add an `Observation` logic to the `SourcePollingChannelAdapter`.
* Divide it into two phases: start (and open scope) when message is received; stop (and close scope) when the whole polling task for a message is done.
We need this separation because of transaction scope for the polling task.
At the same time we don't want to emit an observation for a void polling task.
* Change `MessageReceiverContext` to accept a `handlerType`.
The `MessageHandler` contributes a `handler`.
The new support in the `SourcePollingChannelAdapter` - `message-source`.
And change `MessageProducerSupport` to contribute a `message-producer`
* Verify the single trace is supported for the whole flow (including transaction synchronization) starting from a `SourcePollingChannelAdapter` in a new `SourcePollingChannelAdapterObservationTests`
* Document this new feature
Fixes: https://github.com/spring-projects/spring-integration/issues/5123
When `RedisMessageStore`, for example, adds and removes messages, it operates on two keys separately, which may cause problems in multi-threading due to non-atomic operations.
Although using Redis to delay messages is not a good idea, the abnormal loss of messages in the logs alerted me when the number of requests was not large.
By comparing the logs, the problem that the message group representing the metadata is not consistent with the actual message.
A simple solution is to add lock like in the `SimpleMessageStore`, which is also the approach taken in this pull request.
* Add `LockRegistry` to `AbstractMessageGroupStore`
* Normalize access levels and method name about the lock of `MessageGroupStore`
* Add document about the lock of `AbstractMessageGroupStore`
Fixes: https://github.com/spring-projects/spring-integration/issues/9706
Previously, if a `MessageGroupProcessor` returns a collection of payloads,
the `AbstractCorrelatingMessageHandler` has failed with the `IllegalArgumentException`
stating that only collection of messages is possible.
From now on such a restriction is eliminated and returned collection of payloads
is emitted as a single reply message from the aggregator.
* Add `AbstractCorrelatingMessageHandler.isResultCollectionOfMessages()`
to return `true` only if result is a collection of messages, treating them as a "partial sequence".
* Deprecate `AbstractCorrelatingMessageHandler.verifyResultCollectionConsistsOfMessages()`
since it is out of use now.
Fixes: https://github.com/spring-projects/spring-integration/issues/9683
Since we don't support SpEL-based Control Bus functionality anymore,
there is no need to keep separate `controlBusOnRegistry()`
* In all DSLs Deprecate `controlBusOnRegistry()` and restore `controlBus()` which is now fully based
on the `ControlBusCommandRegistry`
* Deprecate now out of use `<control-bus use-registry="">` attribute
* Fix `ControlBusParser` to not deal with `use-registry` attribute anymore
* Remove deprecated before `ExpressionControlBusFactoryBean` and `ExpressionCommandMessageProcessor`
* Remove `use-registry` from test configs
* Fix `ControlBusChainTests` to rely on a new Control Bus functionality
* Upgrade dependencies to latest reasonable
* Add `6.4.x` branch to Dependabot config
* Move `whats-new.adoc` content to `changes-6.3-6.4.adoc`
* Optimize `build.gradle` moving `add-opens` for `java.base/java.util` to common config block
Fixes: #9695
Issue link: https://github.com/spring-projects/spring-integration/issues/9695
The `txSize` in the `SimpleMessageListenerContainer` has been replaced with more reasonable `batchSize`.
Spring Integration XML support for AMQP module has missed to fix this change: we didn't have a respective test.
* Deprecate `tx-size` (will be removed in `6.5`) XML attribute for the `<amqp:inbound-channel-adapter>`
and introduce `batch-size`
* Cover with the tests
* Fix docs from `tx-size` to `batch-size`
**Auto-cherry-pick to `6.3.x`**
Related to: https://github.com/spring-projects/spring-integration/issues/9683.
Since we are going to restore `controlBus()` for convenience,
and only one way of Control Bus pattern, based on the `ControlBusCommandRegistry`, there is no need to have another `controlBusOnRegistry()` method
* Also fix link for `error-handling` chapter from the `special-channels.adoc`
Fixes: #9613
Issue link: https://github.com/spring-projects/spring-integration/issues/9613
Sometimes even if `\Recent` or user flag is not supported by mail server, the `\Flagged` is also undesirable
* Expose `AbstractMailReceiver.setFlaggedAsFallback()` to disable setting `\Flagged` on the message as fallback
* As well as expose `MailInboundChannelAdapterSpec.flaggedAsFallback()`
* Document the new option
Fixes: #9617
Issue link: https://github.com/spring-projects/spring-integration/issues/9617
The `ListenableFuture` is marked `forRemoval` in Spring Framework.
So, fix the code base to use `@SuppressWarnings("removal")`.
Also, add a warning into logs that `ListenableFuture` support will be removed in `7.0`.
* Fix JavaDocs where `ListenableFuture` is mentioned in favor of `CompletableFuture`
Fixes: #9416
Issue link: https://github.com/spring-projects/spring-integration/issues/9416
The `MessageBuilderFactory` bean could be used a central place to provide custom `Message`
implementation into the application.
For example, the `GenericMessage.toString()` can be overridden to remove or mask sensitive
information from the payload or headers.
* Extract a `BaseMessageBuilder` from the `MessageBuilder` class to simplify
a custom `MessageBuilderFactory` implementation
* Test and document new feature and its purpose
* Upgrade Gradle to `8.10.2`
* Upgrade Antora extensions
* Upgrade to the latest `spring-io/antora-ui-spring`
* Upgrade MySQL connector-j to `9.1.0`
* Some code modernization for the `TcpNioConnectionTests`
Although `RenewableLockRegistry` provides a renew interface, it is inconvenient for users.
Developers hope to have a lock that can be automatically renewed.
On the one hand, it can avoid subsequent failures caused by locks that will not expire when abnormal exits,
and on the other hand, it can avoid unlock failures caused by lock expired.
* Add `RenewableLockRegistry.setRenewalTaskScheduler()` and when it is set, schedule a `renew()` script periodically
when lock is acquired from Redis with `1/3` of `expireAfter`
* Test and document the feature
Fixes: #9524
Issue link: https://github.com/spring-projects/spring-integration/issues/9524
It is useful in some use-cases to be able to inject a custom `TaskScheduler`
(e.g. with a `TaskDecorator`) into a source polling channel adapter.
* Add `SourcePollingChannelAdapterFactoryBean.setTaskScheduler()`
and call it from the `SourcePollingChannelAdapterSpec.taskScheduler()`
* Fix JavaDocs typos in the `ConsumerEndpointSpec`
* Test custom `TaskScheduler` usage and mention new option in the `whats-new.adoc`
Fixes: #3616
Issue link: https://github.com/spring-projects/spring-integration/issues/3616
The `KafkaProducerMessageHandler` uses an unnecessary logic to determine a default topic/partition.
It is better to push such a logic down to the `ReplyingKafkaTemplate` as more general solution
* Remove `KafkaProducerMessageHandler.getReplyTopic()` logic altogether
* Clean up tests for removed logic
* Add `exclude group: 'ch.qos.logback'` to be able to control logging for SI-Kafka module
* Remove out-dated sentence from `kafka.adoc`
Fixes: #9492
Issue link: https://github.com/spring-projects/spring-integration/issues/9492
Switches `MockIntegrationContextCustomizerFactory` & `SpringIntegrationTestExecutionListener`
to use `TestContextAnnotationUtils` in order to properly support
`@NestedTestConfiguration` semantics.
Adds an integration test with a base class and several
`@Nested` inner test classes to verify the various
permutations of `@SpringIntegrationTest` inherit/override
behavior when used w/ `@NestedTestConfiguration`.
**Auto-cherry-pick to `6.3.x` & `6.2.x`**
Fixes: #9455
Issue link: https://github.com/spring-projects/spring-integration/issues/9455
* Add an `IntegrationKeepAlive` infrastructure bean to initiate a long-lived non-daemon thread
to keep application alive when it cannot be kept like that for various reason, but has to.
* Expose `spring.integration.keepAlive` global property to disable an `IntegrationKeepAlive` auto-startup
* Test and document the feature
Fixes: #9507
Issue link: https://github.com/spring-projects/spring-integration/issues/9507
* Deprecate `PythonScriptExecutor` in favor of `PolyglotScriptExecutor` with a `python` as language
* Add handling for `PolyglotWrapper` return type of the script evaluation
* Rework `DeriveLanguageFromExtensionTests.testParseLanguage()` to the `@ParameterizedTest`
Fixes: #9452
Issue link: https://github.com/spring-projects/spring-integration/issues/9452
* Move the logic about `fileNameGenerator` configuration out of `AbstractZipTransformer`
since this property is not used by `UnZipTransformer` implementation
* Add more docs about `ZipTransformer` logic
Fixes: #9436
Issue link: https://github.com/spring-projects/spring-integration/issues/9436
* Expose `IndexAccessor` configuration options on the `AbstractEvaluationContextFactoryBean`
and `SpelPropertyAccessorRegistrar`
* Expose `<index-accessors>` sub-element for the `<spel-property-accessors>`
* Adjust tests
* Document the feature, including recently added `JsonIndexAccessor`
Fixes: #9428
Issue link: https://github.com/spring-projects/spring-integration/issues/9428
When `ClientManager` is used for MQTT channel adapters, a `MqttMessageDeliveredEvent`
is not emitted since callback for the `ClientManager` is not aware about `deliveryComplete`
* Use a `MqttActionListener` abstraction for the `publish` operation instead of a `deliveryComplete`
from a common callback
* Make some other refactoring into the `MqttPahoMessageHandler` and `Mqttv5PahoMessageHandler`
extracting a common logic into their `AbstractMqttMessageHandler` superclass
* Introduce an `MqttMessageNotDeliveredEvent` to be emitted from the `MqttActionListener.onFailure()` callback
* Adapt mocks in the `MqttAdapterTests` for a new code flow
* Add delivery events verification into the `ClientManagerBackToBackTests`
* Fix race condition in the `ClientManagerBackToBackTests`
Looks like the message can be consumed even before we just emit that `MqttMessageSentEvent`
The extra REST API to get commands for specific bean does not make sense.
Exactly same data can be extracted from the JSON returned by the `getCommands` REST API
