* Fix NPEs in DSL Specs
The `BaseWsInboundGatewaySpec` and `TailAdapterSpec` don't override super methods
and when we call them we fail with NPE since `target` was not populated.
* Fix `BaseWsInboundGatewaySpec` and its inheritors to populate the `target`
from their ctors.
* Remove redundant methods from `BaseWsInboundGatewaySpec` hierarchy
* Propagate `AbstractWebServiceInboundGateway` properties directly to the target
from the `BaseWsInboundGatewaySpec` inheritors
* Override `MessageProducerSpec` methods in the `TailAdapterSpec`
to populate respective option into the `FileTailInboundChannelAdapterFactoryBean`
* Expose more missed options for producer endpoint in the `FileTailInboundChannelAdapterFactoryBean`
**Cherry-pick to `6.0.x` & `5.5.x`**
* * `acceptIfNotNull()` for `errorMessageStrategy` in the `FileTailInboundChannelAdapterFactoryBean`
* * Fix `Unmarshaller` population logic in the `MarshallingWsInboundGatewaySpec`
* Improve Java DSL for Rabbit Streams
* Expose simple properties for `streamName` and `superStream`
on the `RabbitStreamInboundChannelAdapterSpec`
* Add `superStream(String superStream, String name, int consumers)` option
* Add `outboundStreamAdapter(Environment environment, String streamName)` factory for
simple use-cases
* Add `RabbitStreamTests` integration test to cover Rabbit Streams support
and demonstrate respective Java DSL
* Mention the change in the docs
* * Fix typos in code and docs
* GH-3557: Add maxDepth, dirPredicate to FileReadMS
Fixes https://github.com/spring-projects/spring-integration/issues/3557
* Expose a `watchMaxDepth` on the `FileReadingMessageSource` for its `Files.walkFileTree()` API usage
* Add `watchDirPredicate` option ot the `FileReadingMessageSource` to skip sub-tree for `Files.walkFileTree()`
scanning according to some condition against directory `Path`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Currently, the `MessageTriggerAction.trigger` is configured
on the `BaseIntegrationFlowDefinition` to be called via reflection
in the `MessagingMethodInvokerHelper`
* Represent a `MessageTriggerAction.trigger` as a `Consumer<Message<?>>` method reference
and use a `LambdaMessageProcessor` for direct call
* Add a `Consumer` support for `LambdaMessageProcessor`
* GH-8586: Deprecate IntegrationComponentSpec.get()
Fixes https://github.com/spring-projects/spring-integration/issues/8586
The `IntegrationComponentSpec` is not a plain wrapper around single component.
Sometimes it comes with several components where all of them must be registered
as beans.
If `IntegrationComponentSpec.get()` is called from end-user code, we may lose
other related components, for example filters in the `FileInboundChannelAdapterSpec`.
* Deprecate `IntegrationComponentSpec.get()` with no-op for end-user,
rather encourage to leave it as is and let the framework take care about its lifecycle
and related components registration
* Fix `IntegrationComponentSpec` logic to deal as a simple `FactoryBean` instead of
extra overhead via `AbstractFactoryBean`
* Use `IntegrationComponentSpec.getObject()` in the framework code where `get()` was called
* Fix tests to expose `IntegrationComponentSpec` as beans instead of previously called `get()`
* Some other clean up and typos fixes in the affected classes
* Document the change
* * Revert `ObjectStringMapBuilder` in the `KafkaInboundGatewaySpec.getComponentsToRegister()`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
* * Remove trailing whitespace in the `ScriptMessageSourceSpec`
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Propagate Reactor context over headers
When we do something like `Flux.from(Publisher)`
and don't compose it with the one involved in the `Subscriber` context,
we lose this context.
* Provide a mechanism to propagate a Reactor context over message header
produce within that context.
* Restore this context in the `FluxMessageChannel` for a new publisher
we use in this channel
**Cherry-pick to `6.0.x`**
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3869
* Move `ContextHolderRequestHandlerAdvice` to the `core` module for more general purposes
* Add `ContextHolderRequestHandlerAdviceTests`
* Rework `DelegatingSessionFactoryTests` to rely on the `ContextHolderRequestHandlerAdvice`.
This allows us to remove unnecessary XML configuration for this test class
* Document the feature
Fixes https://github.com/spring-projects/spring-integration/issues/8582
* Introduce a `PostgresSubscribableChannel.setTransactionManager()`
to wrap a message polling and dispatching operation into a transaction
* In addition add a `RetryTemplate` support around transaction attempts
**Cherry-pick to `6.0.x`**
Some tests deliberately don't expect a reply, but they still block
on a gateway's `sendAndReceive()`
* Improve `Jsr223ScriptExecutingMessageProcessorTests`
to verify that script variables work
* GH-8577: Revise `ImapIdleChannelAdapter` logic
Fixes https://github.com/spring-projects/spring-integration/issues/8577
When we process mail messages in async manner, it is possible that we end up
in a race condition situation where the next idle cycle closes the folder.
It is possible to reopen the folder, but feels better to block the current idle
cycle until we are done with the message and therefore keep folder opened.
* Deprecate `ImapIdleChannelAdapter.sendingTaskExecutor` in favor of an `ExecutorChannel`
as an output for this channel adapter or similar async hand-off downstream.
* Make use of `shouldReconnectAutomatically` as it is advertised for this channel adapter
* Optimize the proxy creation for message sending task
* * Remove `ImapIdleChannelAdapter.sendingTaskExecutor`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/8581
Do not overwrite configuration of externally provided `SshClient` in the `DefaultSftpSessionFactory`
* Replace JUnit `assertDoesNotThrow` by AssertJ `assertThatNoException` in test
**Cherry-pick to `6.0.x`**
* Add `spring-web` test dep for `spring-integration-graphql` module
* Fix `MultipartAsRawByteArrayTests` according to the changes in the
`ByteArrayHttpMessageConverter` where it uses `ReadNBytes()` even for
`0` in `Content-Length` header
* Fix Kafka tests to use `RetryListener` interface with `default`
methods instead of already deprecated `RetryListenerSupport`
Currently, many timeouts in the project are like `-1` or other negative value
with a meaning to wait indefinitely.
According to distributed systems design and bad demo developing experience
it is not OK to block forever.
* Rework most of the timeouts in the framework to be `30` seconds.
Only one remained as `1` seconds is a `PollingConsumer` where it is
better to not block even for those 30 seconds when no messages in the queue,
but let the polling task be rescheduled.
* Remove the `MessagingGatewaySupport.replyTimeout` propagation down to the
`PollingConsumer` correlator where it was a `-1` before and blocked
the polling thread on the `Queue.poll()`.
This fixed the problem with a single thread in a pool for auto-configured `TaskScheduler`.
Now with 1 seconds wait time we are able to switch to other scheduled tasks
even with only 1 thread in the pool
* Warn about dropped message in filter
Buy default the `MessageFilter` just drops a discarded message silently.
If a request-reply gateway is used upstream, then it becomes unclear
why the flow sometimes doesn't work.
* Add a waring log ot emit for default behavior.
This still doesn't fix the request-reply problem, but at least it can
give a clue what is going on
* * Mention `nullChannel` variant to ignore even warn
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
It is not obvious from the `@MessagingGateway` Javadocs how to configure
an individual service method
* Add links and explanation about per-method `@Gateway`
annotation into the `@MessagingGateway`
* Explicitly enable Jacoco XML report
Must be `xml.required = true`
* Starting with version 3 the Sonarqube Gradle plugin
does not do any automatic dependencies on other tasks:
https://docs.sonarqube.org/latest/analyzing-source-code/scanners/sonarscanner-for-gradle/
* It is also does not apply the task to sub-projects.
So, to make the root `sonarqube` task we need to have config like this:
```
rootProject.tasks['sonarqube'].dependsOn jacocoTestReport
```
Note: the `sonarqube` task is deprecated in favor of `sonar`,
but that is exactly what our current Bamboo Sonar plugin does.
So, when that is upgraded, we need to revise our config again
* We don't need `xml.outputLocation` since its default is what Sonar expects
Fix race condition, when methods `RedisLockRegistry#expireUnusedOlderThan` and `RedisLockRegistry#obtain` are executed successively.
It's possible to delete the lock from `RedisLockRegistry#expireUnusedOlderThan` method, when lock is created but is not acquired (`RedisLock#getLockedAt = 0`)
It can lead to the situation, when `RedisLockRegistry#obtain` returns multiple locks with the same redis-key, which shouldn't happen at all.
* Skip locks from expiration when their `lockedAt == 0` - new, not acquired yet.
**Cherry-pick to `6.0.x` & `5.5.x`**
* Some `LockRegistryLeaderInitiator` improvements
It is better to not go to the target lock provider at all
if the current thread is already interrupted.
* Check for the `Thread.currentThread().isInterrupted()` in the `while` loop
and `restartSelectorBecauseOfError()` immediately without checking for a lock
* Fix some other simple typos in the `LockRegistryLeaderInitiator`
**Cherry-pick to `6.0.x`**
* * Introduce a `LeaderSelector.yielding` flag
to revoke leader smoothly.
Turns out just canceling the `Future` may lead to a broken lock
where we cannot unlock it because the target lock repository may not work
with interrupted threads.
This way a new leader must wait until the lock is expired in the store
* Remove the use of instance suppliers on bean definitions
that are processed during the AOT phase.
* The remaining areas that use instance suppliers do so
at runtime and do not use reflection, but instead
are passed the configured bean to register.
See https://github.com/spring-cloud/spring-cloud-stream/issues/2655
**Cherry-pick to `6.0.x`**
* GH-8559: Document how to enable SOCKS for SFTP
Fixes https://github.com/spring-projects/spring-integration/issues/8559
An out-of-the-box `SshClient` does not provide a smooth HTTP/SOCKS proxy configuration.
* Mention in the `sftp.adoc` that `JGitSshClient`, configured with SOCKS,
can be injected into a `DefaultSftpSessionFactory`
* Fix Javadocs for `DefaultSftpSessionFactory`, respectively
* Fix language in doc
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8562: Fix streaming source for remote calls
Fixes https://github.com/spring-projects/spring-integration/issues/8562
The `AbstractRemoteFileStreamingMessageSource.doReceive()` takes files first from a `toBeReceived` queue.
When `AbstractRemoteFileStreamingMessageSource.remoteFileToMessage()` fails to fetch the file content
because of interim connection issue, we reset this file from a filter and rethrow an exception.
The next `receive()` call will just go ahead to the next entry in the `toBeReceived` queue, but the
file we have just failed for will be retried only on the next list call to the remove directory.
This essentially breaks a possible in-order target application logic.
* Introduce `AbstractRemoteFileStreamingMessageSource.strictOrder` option to clear the `toBeReceived` queue
when we fail in the `remoteFileToMessage()`, so the next `receive()`
call would re-fetch files from remote dir, because the filter has been reset for those files.
* Fix `AbstractFileInfo.toString()` to not perform remote calls when we just log this file.
For example, we reset the file for connection failure and log the message about it,
but it fails again because we request `size` of the file which may require a remote connection.
**Cherry-pick to `6.0.x` & `5.5.x`**
* * Revert `AbstractFileInfo` changes
* Override `toString()` in `SmbFileInfo` instead -
exactly the place where connection is used to obtain
file attributes like `size` or `lastModified`
