* Fix Groovy DSL for new splitter style
* Introduce a new `GroovyIntegrationFlowDefinition.splitWith()`
to avoid clashing with existing method
* Deprecate all other `split()` variants, but left one without arguments
for a default splitting options
* Fix doc for new splitter style
* * Fix tab indentations in `groovy-dsl.adoc`
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
* Remove unused ThreadLocal from RedisStoreMSource
The `RedisStoreMessageSource` relies on an `IntegrationResourceHolder`
for a while already.
Its internal `ThreadLocal` is just out of use.
* Remove that `resourceHolder` property and related methods to interact with it.
* Mention the `IntegrationResourceHolder` and `store` attribute in the `redis.adoc`
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8626: Provide cleaner `transform()` DSL
Fixes https://github.com/spring-projects/spring-integration/issues/8626
* Add missed `transform(String beanName, @Nullable String methodName)` API
* Introduce a `TransformerSpec` to expose a strict API to configure
transformer variants.
* Introduce `transformWith(Consumer<TransformerSpec>)` as a single point
of all possible transformer and its endpoint options
* Deprecate those `IntegrationFlowDefinition.transform()` variants
which are harder to configure as several lambda arguments
This change will make Kotlin & Groovy DSL more readable and straightforward
* * Use new `transformWith()` in tests where a deprecated API still used
* Add JavaDocs to `TransformerSpec`
* Fix generic types for `BaseIntegrationFlowDefinition.transformWith()` -
they make sense exactly on the `TransformerSpec.transformer()` only
* Apply `transformWith()` for Groovy DSL
* Introduce a new `ClassUtils.isLambda(Object candidate)`
and add a check for Groovy `Closure`
* Fix `GroovyIntegrationFlowDefinition.createConfigurerIfAny()` to propagate a `Consumer` argument
down to the `Closure`
* * Rename `TransformerSpec -> TransformerEndpointSpec` for better context
meaning of the class
* Introduce `KotlinTransformerEndpointSpec` as an extension of the `TransformerEndpointSpec`
to have an `inline fun <reified P> transformer(crossinline function: (P) -> Any)`
for Kotlin style
* Add `KotlinIntegrationFlowDefinition.transformWith(KotlinTransformerEndpointSpec)`
* Deprecate Kotlin methods which are covered by the mentioned `transformWith()`
* Fix tests to use new API
* Mentioned the change in the doc
Fixes https://github.com/spring-projects/spring-integration/issues/8655
In the text about Outbound Channel Adapters for JMS it reads:
> The following configuration produces an adapter that receives Spring Integration messages from the `exampleChannel` [...]
However, the is no this channel definition in Kotlin and Java DSL snippets
* Add an `exampleChannel` to Kotlin and Java DSL samples
* Add Groovy DSL sample for the same flow definition
* Fix some typos and update to the actual types in the `graphql.adoc`
* Don't use `@Container` in the `DebeziumMySqlTestContainer`,
but rather start it manually in the `@BeforeAll` and let one container
to survive between tests.
The Ryuk container then takes care about other containers on JVM exist
Related to https://github.com/spring-projects/spring-integration/issues/8642
For consistency with other Spring requirements and realignment with virtual threads,
it is better to require a `TaskExecutor` injection instead of `ThreadFactory`
* Fix `DebeziumMessageProducer` to rely on a `TaskExecutor` API instead of `ThreadFactory`
and `ExecutorService`
* * Remove unused import from the `DebeziumMessageProducerSpec`
* Mention in the Javadocs of the `DelayHandler`, `DelayerEndpointSpec`,
`BaseIntegrationFlowDefinition.delay()`, `GroovyIntegrationFlowDefinition.delay()`,
`KotlinIntegrationFlowDefinition.delay()` that `messageGroupId` is required option
* Explain in the docs why `messageGroupId` is required and why it cannot rely on a bean name
* Improve Delayer DSL
Move `groupId` option from a `delay()` method arg to the `DelayerEndpointSpec`
to make it cleaner from code reading perspective
* Expose new DSL method based on just a `DelayerEndpointSpec` for Kotlin &v Groovy
* Deprecate multi-arg `delay()` methods in favor of `Consumer<DelayerEndpointSpec>`-based
* * Fix language and code style
Resolves https://github.com/spring-projects/spring-integration/issues/8638
Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.
**cherry-pick to all supported branches**
* Fix race condition in the `JdbcPollingChannelAdapterParserTests`
making SELECT and UPDATE as a part of the same transaction
* Remove `inProcess = JAVA_EXEC` from ASCIIDoc Gradle tasks in attempt
to make them working in parallel
Fixes https://github.com/spring-projects/spring-integration/issues/8632
* Debezium DSL initial support
* additional dsl debezium factory
* debezium dsl improvements and tests
* impove debezium docs and streamline dsl testing
* docs clarifications
* fix doc cross-reference
* updgrade debezium to 2.2.1.Final. Clean docs
* fix multiflow config tests
* improve batch tests
* Code and doc formatting
* Make `name` Debezium property as random according to its docs:
```
Unique name for the connector.
Attempting to register again with the same name fails.
This property is required by all Kafka Connect connectors.
```
* Code style clean up
Fixes https://github.com/spring-projects/spring-integration/issues/3779
initial debezium doc
address some reviews
resolve some classpath conflicts
hacking failed test
fixing tests and dependecies
address review comments
improve test coverage
fix test checkstyle
remove kafak references. hit support for batch
improve java doc
Initial batch support
Convert the list of Change events into list of Messages.
Use the same rules for buidling messages as the non-batch mode.
Refine batch implementation and tests
harden the testcontainers start/stop lifecycle
simplify batch mode
adjust test log config
clean gradle config
Add `HeaderMapper` filter configuration. Fix JavaDocs
Use `CustomizableThreadFactory` for Exec Service. IT header tests
more debeizum documentation
Remove external Executor support in favor of configurable ThreadFactory
minor `Threadfactory` naming fix
fix support package structure
* Clean up code style and language typos
* GH-8625: Add Duration support for `<poller>`
Fixes https://github.com/spring-projects/spring-integration/issues/8625
The duration can be represented in a ISO 8601 format, e.g. `PT10S`, `P1D` etc.
The `<poller>` and `@Poller` don't support such a format.
* Introduce a `PeriodicTriggerFactoryBean` to accept string values for
trigger options and parse them manually before creating the target `PeriodicTrigger`
* Use this `PeriodicTriggerFactoryBean` in the `PollerParser` and `AbstractMethodAnnotationPostProcessor`
where we parse options for the `PeriodicTrigger`
* Modify tests to ensure that feature works
* Document the duration option
* Add more cross-links into polling docs
* Fix typos in the affected doc files
* Add `-parameters` for compiler options since SF 6.1 does not support `-debug` anymore
for method parameter names discovery
* Fix typos
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Add Christian Tzolov to author name in the doc
* Move `whats-new.adoc` content to new `changes-6.0-6.1.adoc`
* Upgrade to Gradle `8.1.1`
* Upgrade to Spring Framework `6.1`
* Fix `StompSessionManagerTests` for deprecated `ConcurrentTaskScheduler` ctor
* Fix XML configs for Kafka to use types for ctor args instead of their names.
Apparently SF doesn't do a discovery by names anymore.
* Introduce `PartitionedChannel`
* Implement a `PartitionedChannel` as an extension of the `AbstractExecutorChannel`
* Supply this channel with a `PartitionedDispatcher` which is an extension of the `AbstractDispatcher`
* The target partition is essentially a `UnicastingDispatcher` with a single thead executor
* * Fix language in Javadocs
* Add docs for `PartitionedChannel` into `channel.adoc`
* Pre-populate partitions in the `PartitionedDispatcher`
to ensure a thread number reflection of the partition it is used for (for default `ThreadFactory`)
* Add Javadocs to `public` methods
* Add Java DSL `PartitionedChannelSpec` and respective factory methods into `Channels`
* Use `IntegrationMessageHeaderAccessor.CORRELATION_ID` header as a default partition key
* * Fix language in Javadocs
* Fix Checkstyle violations
* Mark `PartitionedDispatcher.prePopulatedPartitions()` with `synchronized`
* * Populate partitions from `PartitionedDispatcher` ctor
and when a custom `ThreadFactory` is set
* * Clear `executors` in `populatedPartitions()`
* * Bring back `synchronized populatedPartitions()` and use it in the `dispatch()`
When we configure an output channel for the handler as a `FluxMessageChannel`,
in most cases we assume an async processing for the reply.
Especially this is critical when reply is a reactive type, so in the async
mode it is "flattened" by the mentioned `FluxMessageChannel`.
Therefore, it is a bit awkward to require to set async explicitly,
when we have already configured output channel as a `FluxMessageChannel`
* Remove redundant config for async from `R2dbcDslTests`
* Mention the change in the docs
* Demonstrate `@Publisher` with an `@EventListener`
The `@EventListener` from Spring Framework is a good tool for POJO configuration
to subscribe to events from an `ApplicationContext`.
The `@Publisher` is an AOP tool to publish a `Message` from POJO method call.
* Add a test and docs to demonstrate how the `@Publisher` can be used together with an `@EventListener`
* Add `@Publisher` with a `@RabbitListener` sample in docs
* * Fix FQCN for `@Queue` in the `AmqpTests`
Fixes https://github.com/spring-projects/spring-integration/issues/3866
Some RDBMS vendors (or their JDBC drivers) may just log the problem
without throwing an exception.
* Expose setters for UPDATE and INSERTS queries in the `DefaultLockRepository`
to let end-user to modify them respectively, e.g. be able to add a PostgreSQL
`ON CONFLICT DO NOTHING` hint.
* Refactor `DefaultLockRepository` to `Instant` instead of `LocalDateTime`
with zone offset.
* Refactor `ttl` property to `Duration` type
* Fix `dead-lock` typo to `deadlock`
* GH-3763: Add `handleReactive()` for Java DSL
Fixes https://github.com/spring-projects/spring-integration/issues/3763
Add a convenient terminal operator to `BaseIntegrationFlowDefinition`
based on a `ReactiveMessageHandler`.
Also add an overload like `handleReactive(ReactiveMessageHandlerSpec)`
to let end-user to choose a protocol-specific channel adapter
* * Fix `Namespace Factory` wording in the `BaseIntegrationFlowDefinition` Javadocs
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Improve Java DSL for Rabbit Streams
* Expose simple properties for `streamName` and `superStream`
on the `RabbitStreamInboundChannelAdapterSpec`
* Add `superStream(String superStream, String name, int consumers)` option
* Add `outboundStreamAdapter(Environment environment, String streamName)` factory for
simple use-cases
* Add `RabbitStreamTests` integration test to cover Rabbit Streams support
and demonstrate respective Java DSL
* Mention the change in the docs
* * Fix typos in code and docs
* GH-3557: Add maxDepth, dirPredicate to FileReadMS
Fixes https://github.com/spring-projects/spring-integration/issues/3557
* Expose a `watchMaxDepth` on the `FileReadingMessageSource` for its `Files.walkFileTree()` API usage
* Add `watchDirPredicate` option ot the `FileReadingMessageSource` to skip sub-tree for `Files.walkFileTree()`
scanning according to some condition against directory `Path`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8586: Deprecate IntegrationComponentSpec.get()
Fixes https://github.com/spring-projects/spring-integration/issues/8586
The `IntegrationComponentSpec` is not a plain wrapper around single component.
Sometimes it comes with several components where all of them must be registered
as beans.
If `IntegrationComponentSpec.get()` is called from end-user code, we may lose
other related components, for example filters in the `FileInboundChannelAdapterSpec`.
* Deprecate `IntegrationComponentSpec.get()` with no-op for end-user,
rather encourage to leave it as is and let the framework take care about its lifecycle
and related components registration
* Fix `IntegrationComponentSpec` logic to deal as a simple `FactoryBean` instead of
extra overhead via `AbstractFactoryBean`
* Use `IntegrationComponentSpec.getObject()` in the framework code where `get()` was called
* Fix tests to expose `IntegrationComponentSpec` as beans instead of previously called `get()`
* Some other clean up and typos fixes in the affected classes
* Document the change
* * Revert `ObjectStringMapBuilder` in the `KafkaInboundGatewaySpec.getComponentsToRegister()`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
* * Remove trailing whitespace in the `ScriptMessageSourceSpec`
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Propagate Reactor context over headers
When we do something like `Flux.from(Publisher)`
and don't compose it with the one involved in the `Subscriber` context,
we lose this context.
* Provide a mechanism to propagate a Reactor context over message header
produce within that context.
* Restore this context in the `FluxMessageChannel` for a new publisher
we use in this channel
**Cherry-pick to `6.0.x`**
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3869
* Move `ContextHolderRequestHandlerAdvice` to the `core` module for more general purposes
* Add `ContextHolderRequestHandlerAdviceTests`
* Rework `DelegatingSessionFactoryTests` to rely on the `ContextHolderRequestHandlerAdvice`.
This allows us to remove unnecessary XML configuration for this test class
* Document the feature
Fixes https://github.com/spring-projects/spring-integration/issues/8582
* Introduce a `PostgresSubscribableChannel.setTransactionManager()`
to wrap a message polling and dispatching operation into a transaction
* In addition add a `RetryTemplate` support around transaction attempts
**Cherry-pick to `6.0.x`**
* GH-8577: Revise `ImapIdleChannelAdapter` logic
Fixes https://github.com/spring-projects/spring-integration/issues/8577
When we process mail messages in async manner, it is possible that we end up
in a race condition situation where the next idle cycle closes the folder.
It is possible to reopen the folder, but feels better to block the current idle
cycle until we are done with the message and therefore keep folder opened.
* Deprecate `ImapIdleChannelAdapter.sendingTaskExecutor` in favor of an `ExecutorChannel`
as an output for this channel adapter or similar async hand-off downstream.
* Make use of `shouldReconnectAutomatically` as it is advertised for this channel adapter
* Optimize the proxy creation for message sending task
* * Remove `ImapIdleChannelAdapter.sendingTaskExecutor`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Currently, many timeouts in the project are like `-1` or other negative value
with a meaning to wait indefinitely.
According to distributed systems design and bad demo developing experience
it is not OK to block forever.
* Rework most of the timeouts in the framework to be `30` seconds.
Only one remained as `1` seconds is a `PollingConsumer` where it is
better to not block even for those 30 seconds when no messages in the queue,
but let the polling task be rescheduled.
* Remove the `MessagingGatewaySupport.replyTimeout` propagation down to the
`PollingConsumer` correlator where it was a `-1` before and blocked
the polling thread on the `Queue.poll()`.
This fixed the problem with a single thread in a pool for auto-configured `TaskScheduler`.
Now with 1 seconds wait time we are able to switch to other scheduled tasks
even with only 1 thread in the pool
* Warn about dropped message in filter
Buy default the `MessageFilter` just drops a discarded message silently.
If a request-reply gateway is used upstream, then it becomes unclear
why the flow sometimes doesn't work.
* Add a waring log ot emit for default behavior.
This still doesn't fix the request-reply problem, but at least it can
give a clue what is going on
* * Mention `nullChannel` variant to ignore even warn
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8559: Document how to enable SOCKS for SFTP
Fixes https://github.com/spring-projects/spring-integration/issues/8559
An out-of-the-box `SshClient` does not provide a smooth HTTP/SOCKS proxy configuration.
* Mention in the `sftp.adoc` that `JGitSshClient`, configured with SOCKS,
can be injected into a `DefaultSftpSessionFactory`
* Fix Javadocs for `DefaultSftpSessionFactory`, respectively
* Fix language in doc
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3772
* Add basic To/From Protocol Buffer's com.google.protobuf.Message transformers.
* Allow the proto_type header to specify the type.
* Add tests.
* add Protobuf docs
* Leverage the Spring ProtobufMessageConverter
* move protobuf-java-util to test dependecies as optional
* protobuf docs improvements
* improve the expected type handling
* expected type expression
* fix indentation
* fix indentation for generated test classes
* suppress style check for proto generated classes
* address the transformer doc format
* fix whats new merge conflict
* fix doc sample code
* Some code clean up; fixing typos
* GH-3957: Add JmsInboundGateway.replyToExpression
Fixes https://github.com/spring-projects/spring-integration/issues/3957
Sometimes we cannot use a standard `JmsReplyTo` property for sending replies from the server.
A `DestinationResolver` API does not have access to the request message.
* Introduce a `ChannelPublishingJmsMessageListener.replyToExpression` property to evaluate
a reply destination against request JMS `Message`
* Use this expression only of no `JmsReplyTo` property
* Expose this property on Java DSL level
* To simplify end-user experience with lambda configuration for this property, introduce a `CheckedFunction`
which essentially re-throws exception "sneaky" way
* Fix Javadoc for `CheckedFunction`
* * Fix language in docs
* Fix Javadocs lines length
* Regular `catch` and re-throw in the `CheckedFunction`
* GH-3813: ClientWebSocketContainer URI setting
Fixes https://github.com/spring-projects/spring-integration/issues/3813
Introduce a `ClientWebSocketContainer(WebSocketClient client, URI uri)` ctor
to let end-user to decide what and how should be encoded the URI for WebSocket connection
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Upgrade dependencies whenever it is reasonable
* Fix deprecations and some new API paths
* Move docs to a new version with respective branching for the `changes-5.5-6.0.adoc`
* GH-4001: Doc for cooperation with some HZ objects
Fixes https://github.com/spring-projects/spring-integration/issues/4001
The `IQueue`, `ITopic` and `IExecutorService` can be used with Spring Integration
channel as is without any extra component implementations.
* Document the cooperation feature with Hazelcast objects via samples
* * Add a sample about an Inbound Channel Adapter on the `IQueue`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3998: Fix gateway docs for `@Payload`
Fixes https://github.com/spring-projects/spring-integration/issues/3998
The `GatewayProxyFactoryBean` now deals only with a `MethodArgsHolder` as a root evaluation context object.
There is no `#args` and `#method` SpEL variables anymore.
However, the `gateway.adoc` still refer to those.
* Fix `gateway.adoc` for a proper expressions and actual wording
* * Cross link to `gateway-expressions` paragraph for better context
* Revise JS support for regular JVM
Turns out that we just need to have `org.graalvm.sdk:graal-sdk` and `org.graalvm.js:js`
dependencies for regular JVM to enable JavaScript support for Spring Integration Scripting module
* Add respective `providedImplementation` dependencies
* Remove `@EnabledIfSystemProperty` from unit tests
* Mention those dependencies in the doc
* Rework sentence in the doc for better English
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3986
The `ReactiveStreamsConsumer` in addition to plain `Subscriber` and `MessageHandler`
also supports a `ReactiveMessageHandler`, but `MockIntegrationContext` doesn't
handle a scenario when `ReactiveMessageHandler` is provided for consumer
* Rework the logic in the `MockIntegrationContext` to substitute a `ReactiveMessageHandler`
in the `ReactiveStreamsConsumer` which has a precedence in its logic over plain `Subscriber`.
* Wrap a plain `MessageHandler` mock into a `ReactiveMessageHandler` before substitution
in the `ReactiveStreamsConsumer`
* Reset `ReactiveStreamsConsumer.reactiveMessageHandler` with source `ReactiveMessageHandler`
or `null` respectively to an original `ReactiveStreamsConsumer` configuration
* Mention `ReactiveMessageHandler` use-case in the `testing.adoc`
**Cherry-pick to `5.5.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3974
* Treat a leading `/` as an indicator of absolute path request in the `SftpSession.doList()`
* Call `sftpClient.canonicalPath()` for path without a leading `/` to resolve it as relative path in the user home
* Add a note in docs for `LS` command
* GH-3959: MqttConFailedEvent for normal disconnect
Fixes https://github.com/spring-projects/spring-integration/issues/3959
The `MqttCallback.disconnected(MqttDisconnectResponse)` in Paho v5 client is also
called when server initiates a disconnection.
In this case that `MqttDisconnectResponse` does not have a `cause` value
* Modify `MqttConnectionFailedEvent` to make a `cause` property optional
* Fix `Mqttv5PahoMessageDrivenChannelAdapter` & `Mqttv5PahoMessageHandler`
to not check for `cause`, but emit an `MqttConnectionFailedEvent` for any `disconnected()` calls
Unfortunately current Paho v3 client does not call `connectionLost()` for normal disconnections
and we cannot react for this callback with an `MqttConnectionFailedEvent`
**Cherry-pick to `5.5.x`**
* Fix language in doc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
