The Checkstyle violation for unused imports in the `FileTailingMessageProducerTests`
Leftover from the previous fix for this class
**Cherry-pick to `6.1.x`**
The `OSDelegatingFileTailingMessageProducer` publishes event from scheduled task
* Modify `FileTailingMessageProducerTests.testGuts()` to use `CountDownLatch`
to wait for those asynchronous events
**Cherry-pick to `6.1.x`**
Fixes: gh-8800
The `SmbFile.getName()` returns plain file name, not with a
sub-directory as it is expected by the `AbstractRemoteFileOutboundGateway`
for (S)FTP protocols
* Introduce `AbstractRemoteFileOutboundGateway.getFullFileName()`
with dir and file name concatenation by default (as it was before)
and override it in the `SmbOutboundGateway` to return the full path according this protocol.
* Fix `SmbOutboundGateway.enhanceNameWithSubDirectory()` to return an `SmbFile` instance
since it has all the info about remote file, include requested directory
* Fix `SmbOutboundGateway.getFilename()` to remove the trailing `/` in the dir name
since this is not what expected by the `AbstractRemoteFileOutboundGateway` logic
**Cherry-pick to `6.1.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/8797
After migration to Apache MINA we have missed to fix `DefaultSftpSessionFactory.timeout`
to be `0` by default as it states in its Javadocs and reference manual
It is `null` by default which really means an infinite wait.
* Fix `DefaultSftpSessionFactory.timeout` to be a reasonable 30 seconds by default
* Fix `setTimeout()` Javadocs and respective `session-factory.adoc`
* Propagate this `timeout` down to the `SftpClient` for its commands interactions
Fixes https://github.com/spring-projects/spring-integration/issues/8792
1. Use a `SftpStreamingMessageSource` with a `maxFetchSize = 5` and a `ChainFileListFilter` filter composed with
`SftpSystemMarkerFilePresentFileListFilter` which `supportsSingleFileFiltering == false`
2. Put 2 files in the folder and invoke `SftpStreamingMessageSource.receive()` method twice.
3. Put 5 files in the folder and invoke `SftpStreamingMessageSource.receive()`` method five times.
4. The last two files won't be received.
When you set max fetch size to a number bigger than one (for example 5) and at a certain point
it is necessary to `this.toBeReceived.clear()` inside `AbstractRemoteFileStreamingMessageSource.doReceive()`,
those removed elements from toBeReceived are not rolled back.
* Fix `AbstractRemoteFileStreamingMessageSource.listFiles()` to calculate `maxFetchSize` as `getMaxFetchSize() - this.fetched.get()`
**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/8786
If `FtpSession.readRaw()` fails, the next `FtpSession.finalizeRaw()` call
would lead to `FTPClient.completePendingCommand()` failure since
there is no command to finish.
* Fix `FtpSession.finalizeRaw()` to exit earlier positively in case of
`FTPReply.isNegativePermanent()` for the current reply code set by the failure
from a previous `FtpSession.readRaw()`
**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
* Move `api project(':spring-integration-core')` to the common `dependencies` section
* Remove redundant `exclude group: 'org.springframework'`
* Add some required deps for `CassandraContainer`
The listener on `IMap` is called in async manner.
* Add `await().untilAsserted()` for the first `verify(listener)`
* Migrate this `HazelcastMetadataStoreTests` to JUnit 5
Fixes https://github.com/spring-projects/spring-integration/issues/8785
The `ClientWebSocketContainer.start()` delegates to the `IntegrationWebSocketConnectionManager`
which performs an async connection to the server.
* Wait for `connectionLatch` in the `ClientWebSocketContainer.start()`
and check for `this.openConnectionException != null` to re-throw.
Mark `ClientWebSocketContainer` as stopped in that case
**Cherry-pick to `6.1.x` & `6.0.x`**
Related to https://github.com/spring-projects/spring-amqp/issues/2522
No need to use an `AddressResolver` with the latest RabbitMQ Streams Client library
The current configuration is reflecting whatever Spring Boot auto-configuration experience
would expect from us
Fixes https://github.com/spring-projects/spring-integration/issues/8773
The https://github.com/spring-projects/spring-integration/issues/8732 introduced a filtering for messages in group.
So, plain `removeMessage()` doesn't work any more if message is connected to some group yet.
Therefore, `DelayHandler` is failing.
* Introduce `getMessageFromGroup()` and `removeMessageFromGroupById()` into `MessageGroupStore` API
and implement it respectively in all the stores
* Remove `@LongRunningTest` from delayer integration tests and adjust its config to delay not for a long
**Cherry-pick to `6.1.x`**
* KafkaMessageDrivenChannelAdapter's ATTRIBUTES_HOLDER should be isolated.
In order to achieve custom retry in batch mode, we may to use a RetryTemplate in listener itself. But if the RetryTemplate is shared with another KafkaMessageDrivenChannelAdapter, batch mode's ATTRIBUTES_HOLDER might be over-written by another KafkaMessageDrivenChannelAdapter's IntegrationRecordMessageListener.
The situation is like shown below.
- There is only one RetryTemple bean in the application.
- There are two KafkaMessageDrivenChannelAdapters(A,B) in the application.
- A KafkaMessageDrivenChannelAdapter is batch mode and utilizing the retryTemplate in the listener.
- B KafkaMessageDrivenChannelAdapter is record mode and using the retryTemplate itself.
- (B KafkaMessageDrivenChannelAdapter's recordListener is registered in the retryTemplate.)
- When A retry is attempted in the listener, it will trigger B KafkaMessageDrivenChannelAdapter's recordListener.
- B KafkaMessageDrivenChannelAdapter's recordListener will overwrite A KafkaMessageDrivenChannelAdapter's ATTRIBUTES_HOLDER.
* should not mutate an externally provided RetryTemplate
* should not expose KafkaInboundEndpoint's methods outside the class and fix a checkstyle error.
* removing unused code and polishing
* restore retry around batch and bring back KafkaInboundEndpoint to endpoints.
* remove retry logic in batch mode and move ATTRIBUTES_HOLDER into KafkaInboundEndpoint
* remove generic type parameters of KafkaInboundEndpoint
* fix style error
---------
Co-authored-by: kurt <kurt@weversecompany.com>
* GH-8778: Fix KafkaMessageSource deadlock
Fixes https://github.com/spring-projects/spring-integration/issues/8778
The `KafkaMessageSource.doReceive()` have a lock around its whole body.
That includes the `pollRecord()` which can be blocked on the `KafkaConsumer.poll()`.
This way the rest of lifecycle management callbacks can be blocked until `KafkaConsumer.poll()` returns.
* Rework lifecycle management flags to `AtomicBoolean` since there is not too much work
in their respective callbacks
* Decrease a locking block in the `doReceive()` just to consumer setup part.
Leave `pollRecord()` outside of the lock
* Add `this.consumer.wakeup()` into `stopConsumer()` to break a `poll()` cycle
and return immediately for the next `close()` call
**Cherry-pick to `6.1.x` & `6.0.x`**
* * Use `compareAndSet` in `start` & `stop`
* GH-8770: Add `PostgresSubsChannel.errorHandler`
Fixes https://github.com/spring-projects/spring-integration/issues/8770
The problem with the `PostgresSubscribableChannel.notifyUpdate()` is that the try-catch block is outside the loop,
so the loop will die on an exception, leaving further messages unprocessed.
* Add ``PostgresSubscribableChannel.errorHandler` option to be invoked
after a `RetryTemplate` and for every failed message.
* The `askForMessage()` new logic is to catch an exception on a message and call `errorHandler`
returning a `FALLBACK_STUB` to continue an outer loop in the `notifyUpdate()`
**Cherry-pick to `6.1.x` & `6.0.x`**
* * Rename private `PostgresSubscribableChannel.askForMessage()` method to more specific `pollAndDispatchMessage()`
The `PostgresSubscribableChannel` uses a task executor
for dispatching messages.
Even if we stop `PostgresChannelMessageTableSubscriber` and
unsubscribe from the channel, the task might be ongoing.
* Use explicit `ThreadPoolTaskExecutor` in the test
to shout it down and wait for tasks to be completed before
verifying DB status
* Optimize `PostgresSubscribableChannel` to mark TX for rollback
when we got a message from DB, but no handlers subscribed
The `SecurityContextPropagationChannelInterceptor` has been migrated to `spring-security-messaging`.
Since it was only the class in the `spring-integration-security`, it is now fully considered
as deprecated
* Remove all the tests from `spring-integration-security`
* Modify `HttpDslTests` to demonstrate the `spring-security-messaging` in action
which has been replaced with whatever there was in `spring-integration-security`
* Remove redundant `exclude group: 'org.springframework'` for security
dependencies in `build.gradle` since all of them rely on the same SF deps
as SI
Fixes https://github.com/spring-projects/spring-integration/issues/8760
* Make `PostgresChannelMessageStoreQueryProvider` to use single `DELETE ... RETURNING` for polling statements
* Add `isUsingSingleStatementForPoll` and use it from `JdbcChannelMessageStore`
* Execute Postgres init scripts to `PostgresContainerTest`
* Code clean up
* Document the new feature
Fixes https://github.com/spring-projects/spring-integration/issues/8757
* Add ctor `ObjectStringMessageConverter(Charset)`
* create tests for `ObjectStringMessageConverter`
* fix codestyle issues
* revise JavaDoc
* `@NonNullApi` for `org.springframework.integration.support.converter`
* revise `SimpleMessageConverter`
* `inboundMessageMapper` and `outboundMessageMapper` are initialized to their default implementations
Bboth are set in non-default ctors with final setters
* remove `@Nullable` from `private BeanFactory beanFactory`
* Handle empty `String... headerNames`
* Some code clean up
