Commit Graph

11418 Commits

Author SHA1 Message Date
Artem Bilan
72fc79dbd5 Fix Groovy DSL for new splitter style (#8669)
* Fix Groovy DSL for new splitter style

* Introduce a new `GroovyIntegrationFlowDefinition.splitWith()`
to avoid clashing with existing method
* Deprecate all other `split()` variants, but left one without arguments
for a default splitting options
* Fix doc for new splitter style

* * Fix tab indentations in `groovy-dsl.adoc`
2023-07-10 11:08:44 -04:00
Artem Bilan
34f901f7db Improve split() DSL (#8666)
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
2023-07-03 10:35:21 -04:00
pziobron
0798c8df95 GH-8659: Fix WatchService to react for renames
Fixes https://github.com/spring-projects/spring-integration/issues/8659

* GH-8659: Updating the documentation as requested

* GH-8659: Fixes requested after coder review

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-29 14:23:33 -04:00
abilan
019e96c925 Disable ImapMailTests.testIdleWithMessageMapping
It is not clear why IMAP GreenMail server closes its socket accidentally
2023-06-28 14:35:55 -04:00
Artem Bilan
43473836ef Remove unused ThreadLocal from RedisStoreMSource (#8663)
* Remove unused ThreadLocal from RedisStoreMSource

The `RedisStoreMessageSource` relies on an `IntegrationResourceHolder`
for a while already.
Its internal `ThreadLocal` is just out of use.

* Remove that `resourceHolder` property and related methods to interact with it.
* Mention the `IntegrationResourceHolder` and `store` attribute in the `redis.adoc`

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-06-28 13:40:48 -04:00
abilan
229e3cc081 Increase wait timeout in FluxMessageChannelTests 2023-06-28 13:21:41 -04:00
abilan
1690be45fc Decrease read timeout for GreenMail tests
This may give a chance for IDLE to restart more often
to read the actual INBOX state during the test
2023-06-28 12:53:25 -04:00
Artem Bilan
4583afe0ad GH-8611: Extract MessagingAnnotationBeanPostProcessor (#8661)
Fixes https://github.com/spring-projects/spring-integration/issues/8611

According to the latest Spring Framework requirements related to AOT the
`BeanDefinitionRegistryPostProcessor` must not do anything but only bean registrations

* Extract a `MessagingAnnotationBeanPostProcessor` component with a `BeanPostProcessor` logic
from the `MessagingAnnotationPostProcessor` and leave the last one as just
a `BeanDefinitionRegistryPostProcessor` impl
* Fix tests according to a new logic
2023-06-28 08:59:37 -04:00
Artem Bilan
13980af1e1 GH-8611: Extract IntegrationConfigurationReport (#8660)
Fixes https://github.com/spring-projects/spring-integration/issues/8611

According to the latest Spring Framework requirements related to AOT the
`BeanDefinitionRegistryPostProcessor` must not do anything but only bean registrations

* Extract an `IntegrationConfigurationReport` component
and move `IntegrationProperties` printing over here out from the
`DefaultConfiguringBeanFactoryPostProcessor`
* Remove a `SmartInitializingSingleton` impl from the `DefaultConfiguringBeanFactoryPostProcessor`
2023-06-27 12:32:33 -04:00
Artem Bilan
65c7e5dc5f Fix Kotlin DSL delegation (#8658)
The `ConsumerEndpointSpec` extensions for Kotlin
don't delegate to the provided `endpointFactoryBean`

* Introduce `KotlinConsumerEndpointSpec` extension for `ConsumerEndpointSpec`
with the proper delegation to the provided spec
* Use `KotlinConsumerEndpointSpec` in the Kotlin-specific `Spec` classes

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-26 15:11:00 -04:00
Artem Bilan
af95829a3f GH-8654: Fix bean deps for messaging annotations (#8657)
Fixes https://github.com/spring-projects/spring-integration/issues/8654

Spring Framework has an ability to start dependant beans automatically
when we start the current one.

The `AbstractMethodAnnotationPostProcessor` is missing a bean dependency
registration causing errors in target applications when Messaging Annotations configuration is used.

* Add `registerDependentBean()` into an `AbstractMethodAnnotationPostProcessor` when we generate
and register a `ConsumerEndpointFactoryBean`
* Change one of the `ClientManagerBackToBackTests` configuration to rely on a `@ServiceActivator`
for `Mqttv5PahoMessageHandler` bean to ensure that change in the `AbstractMethodAnnotationPostProcessor`
has a proper effect

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-06-26 14:59:55 -04:00
Artem Bilan
070c1c6e60 GH-8626: Provide cleaner transform() DSL (#8653)
* GH-8626: Provide cleaner `transform()` DSL

Fixes https://github.com/spring-projects/spring-integration/issues/8626

* Add missed `transform(String beanName, @Nullable String methodName)` API
* Introduce a `TransformerSpec` to expose a strict API to configure
transformer variants.
* Introduce `transformWith(Consumer<TransformerSpec>)` as a single point
of all possible transformer and its endpoint options
* Deprecate those `IntegrationFlowDefinition.transform()` variants
which are harder to configure as several lambda arguments

This change will make Kotlin & Groovy DSL more readable and straightforward

* * Use new `transformWith()` in tests where a deprecated API still used
* Add JavaDocs to `TransformerSpec`
* Fix generic types for `BaseIntegrationFlowDefinition.transformWith()` -
they make sense exactly on the `TransformerSpec.transformer()` only
* Apply `transformWith()` for Groovy DSL
* Introduce a new `ClassUtils.isLambda(Object candidate)`
and add a check for Groovy `Closure`
* Fix `GroovyIntegrationFlowDefinition.createConfigurerIfAny()` to propagate a `Consumer` argument
down to the `Closure`

* * Rename `TransformerSpec -> TransformerEndpointSpec` for better context
meaning of the class
* Introduce `KotlinTransformerEndpointSpec` as an extension of the `TransformerEndpointSpec`
to have an `inline fun <reified P> transformer(crossinline function: (P) -> Any)`
for Kotlin style
* Add `KotlinIntegrationFlowDefinition.transformWith(KotlinTransformerEndpointSpec)`
* Deprecate Kotlin methods which are covered by the mentioned `transformWith()`
* Fix tests to use new API
* Mentioned the change in the doc
2023-06-26 12:01:13 -04:00
abilan
ceb3daec8e GH-8655: Add channel to JMS samples in doc
Fixes https://github.com/spring-projects/spring-integration/issues/8655

In the text about Outbound Channel Adapters for JMS it reads:

> The following configuration produces an adapter that receives Spring Integration messages from the `exampleChannel` [...]

However, the is no this channel definition in Kotlin and Java DSL snippets

* Add an `exampleChannel` to Kotlin and Java DSL samples
* Add Groovy DSL sample for the same flow definition
2023-06-26 10:51:36 -04:00
abilan
881893dbca Remove debezium-bom import
The `debezium-bom` manages too many third-party deps which causes
an enforcement for our transitive deps where it is not always true
to use old versions

* Exclude `group: 'io.netty'` for `artemis-stomp-protocol`
to avoid possible Netty version override expected by `reactor-netty-http`
2023-06-23 11:05:10 -04:00
abilan
a7d9def99b More exclude from debezium to avoid conflicts 2023-06-22 17:51:16 -04:00
abilan
f50e16e02a Exclude com.datastax.cassandra group
Rely on Cassandra driver as transitive dependency from `spring-data-cassandra`
2023-06-22 17:32:28 -04:00
abilan
d37912f6a0 Remove https://repo.spring.io/release for Gradle 2023-06-22 17:18:46 -04:00
abilan
a9285c3c2c Use assertj dep for test scope explicitly
Looks like something is overriding `assertj-core` version from `3.24.2 -> 3.11.1`
which doesn't have an API we use.

* Adding `testImplementation "org.assertj:assertj-core:$assertjVersion"` explicitly
instead of transitive from the `spring-integration-test-support` fixes the problem
2023-06-22 17:11:56 -04:00
abilan
197a58af5d Use debezium-bom 2023-06-22 16:07:17 -04:00
abilan
34904b3cd6 Upgrade to commons-io-2.13.0; downgrade sshd-sftp
* The latest `sshd-sftp-2.10.0` has a bug not removing trailing `.`
in the "unrooted" path when we ask to create a remote directory.
It works on Windows well, but fails on UNIX file systems.
* The `commons-io-2.13.0` has `Tailer` ctors deprecated
and exposes a builder API.
It starts a tailer process though unconditionally in
its own thread.
Use `setStartThread(false)` to have the tailer process
managed by our own `TaskExecutor`
* Add `ApacheCommonsFileTailingMessageProducer.setPollingDelayDuration(Duration)`
* Deprecate a `TailerListener` impl on the `ApacheCommonsFileTailingMessageProducer`
in favor of an internal instance
2023-06-22 15:41:38 -04:00
abilan
03799293ce Add @LogLevels for SftpRemoteFileTemplateTests
Not clear why `SftpRemoteFileTemplateTests.testINT3412AppendStatRmdir()`
 is failing on CI Linux, so add a tracing diagnostics to see how Apache SSHD behaves over there
2023-06-22 11:55:38 -04:00
abilan
6ded64c0cd Remove org.asciidoctor.jvm.gems plugin
Looks like we don't use any GEMs in out docs: generated PDF and HTML look OK
2023-06-22 10:55:18 -04:00
abilan
9882ce8542 Remove repo.spring.io/plugins-release from Gradle
The https://repo.spring.io/plugins-release requires an authentication now.
Replace it with a `mavenCentral()`
2023-06-22 10:29:45 -04:00
abilan
8a7293c680 An attempt to fix Sftp tests to new sshd-sftp 2023-06-22 10:19:46 -04:00
abilan
8abd72d7c0 Upgrade dependencies; fix problems
* Mockito doesn't allow to `spy()` mocks any more or "overspy"
* Deprecate `KotlinScriptExecutor` if favor of fully supported JSR223
service loaded via `DefaultScriptExecutor`
* Use only one `kotlin-scripting-jsr223` dependency
2023-06-21 17:39:55 -04:00
abilan
a147040072 Some synchronized comments clean up 2023-06-21 13:54:23 -04:00
Christian Tzolov
c38ed96ee9 GH-8643: Replace synchronized with Lock
Fixes https://github.com/spring-projects/spring-integration/issues/8643

* First pass - trivial synchronized blocks
  - Convert the "trivial" `synchronized` block into `ReentrantLock`.

* fix checkstyle

* use blocking lock

* Secon pass - handle multi-lock cases

* javadoc + year

* addres first batch of review suggestions

* fix checkstyle issues

* fix the mqtt parent/child lock monitor sharing

* fix the mqtt parent/child lock monitor sharing, v2

* patch the stomp test
2023-06-21 13:25:45 -04:00
Artem Bilan
9e9bfd0c5c Remove ThreadLocal from RequestHandlerRetryAdvice (#8650)
Related to https://github.com/spring-projects/spring-integration/issues/8644

With virtual threads it is recommended to stay away from `ThreadLocal`
variables to avoid memory exhausting with too many virtual threads

* Fix `RequestHandlerRetryAdvice` to transfer the message context via internal
`IntegrationRetryCallback` implementation.
* Cast to this `IntegrationRetryCallback` in a newly introduced internal `IntegrationRetryListener`
to extract `messageToTry` and set it into a `RetryContext`
`ErrorMessageUtils.FAILED_MESSAGE_CONTEXT_KEY` attribute
* Deprecate a usage of an external `RetryListener` implementation of the `RequestHandlerRetryAdvice`
2023-06-21 11:41:01 -04:00
Vladislav Fefelov
8a29c1e60b Optimise maybeIndex() in JsonPropertyAccessor
The `NumberFormatException` flow control is costly
operation

* Use `Character.isDigit()` check iterating through property String
instead of `NumberFormatException` flow control

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-16 13:43:28 -04:00
abilan
8315b11494 Some tweaks for ImapMailReceiverTests 2023-06-15 13:10:51 -04:00
abilan
656281e74a Use MySQLContainer for Debezium Testcontainer 2023-06-15 11:58:33 -04:00
abilan
99122d7621 Diagnostic: Add LogLevels to ImapMailReceiverTest
* Fix `com.sun.mail` to `jakarta.mail` in Mail module `log4j2-test.xml`
2023-06-15 10:54:37 -04:00
abilan
66e935f92e Fix graphql.adoc and optimize Debezium tests
* Fix some typos and update to the actual types in the `graphql.adoc`
* Don't use `@Container` in the `DebeziumMySqlTestContainer`,
but rather start it manually in the `@BeforeAll` and let one container
to survive between tests.
The Ryuk container then takes care about other containers on JVM exist
2023-06-14 14:13:18 -04:00
Artem Bilan
4db9bad50d GH-8642: Revise executors in the project (#8647)
* GH-8642: Revise executors in the project

Fixes https://github.com/spring-projects/spring-integration/issues/8642

* Rework some `Executors.newSingleThreadExecutor()` to `ExecutorServiceAdapter(new SimpleAsyncTaskExecutor())`
* Expose `TaskExecutor` setters; deprecate `ExecutorService`-based
* Some other code clean up in the effected classes: `LogAccessor`, no `synchronized` in critical blocks
* Give a meaningful prefix for default threads in the context of components, e.g. `SubscribableRedisChannel` - `getBeanName() + "-"`

* * Fix `PostgresChannelMessageTableSubscriberTests` for
`PostgresSubscribableChannel` initialization to let it create
its internal `Executor`

* Use an `AsyncTaskExecutor` injection instead of `ExecutorServiceAdapter` wrapping

* Fix `LockRegistryLeaderInitiatorTests` for `taskExecutor` injection

* Bring back `LockRegistryLeaderInitiator.setExecutorService()`
as an accident after property auto-renaming
2023-06-14 13:33:46 -04:00
Christian Tzolov
aac2adbaa3 Upgrade debezium to 2.3.0.RC1 2023-06-14 09:23:08 -04:00
Artem Bilan
a862692546 Executor instead of ThreadFactory for DebeziumMP
Related to https://github.com/spring-projects/spring-integration/issues/8642

For consistency with other Spring requirements and realignment with virtual threads,
it is better to require a `TaskExecutor` injection instead of `ThreadFactory`

* Fix `DebeziumMessageProducer` to rely on a `TaskExecutor` API instead of `ThreadFactory`
and `ExecutorService`

* * Remove unused import from the `DebeziumMessageProducerSpec`
2023-06-13 16:06:37 -04:00
abilan
b5767483e1 Improve Delayer docs around messageGroupId
* Mention in the Javadocs of the `DelayHandler`, `DelayerEndpointSpec`,
`BaseIntegrationFlowDefinition.delay()`, `GroovyIntegrationFlowDefinition.delay()`,
`KotlinIntegrationFlowDefinition.delay()` that `messageGroupId` is required option
* Explain in the docs why `messageGroupId` is required and why it cannot rely on a bean name
2023-06-13 10:59:06 -04:00
Artem Bilan
81af20aabf Improve Delayer DSL (#8645)
* Improve Delayer DSL

Move `groupId` option from a `delay()` method arg to the `DelayerEndpointSpec`
to make it cleaner from code reading perspective
* Expose new DSL method based on just a `DelayerEndpointSpec` for Kotlin &v Groovy
* Deprecate multi-arg `delay()` methods in favor of `Consumer<DelayerEndpointSpec>`-based

* * Fix language and code style
2023-06-12 16:01:40 -04:00
Gary Russell
c4ee5512f6 GH-8638: Kafka: Send All Fails to Failure Channel
Resolves https://github.com/spring-projects/spring-integration/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
2023-06-08 13:44:29 -04:00
abilan
95f1eb8c28 GH-8616: Upgrade to Angus Mail to 2.0.2
Fixes https://github.com/spring-projects/spring-integration/issues/8616

* And Greenmail to `2.1.0`
2023-06-08 13:42:16 -04:00
abilan
9752229cd1 Use UUID in Debezium tests for unique ids 2023-06-08 12:30:13 -04:00
Artem Bilan
33f0b8ec4a Optimize synchronized in PartitionedDispatcher (#8640)
Even if the `PartitionedDispatcher.populatedPartitions()`
is fast, in-memory, non-blocking operation, its active call from the `dispatch()`
on every message sent to the channel may pin the virtual thread.

* Optimize the `populatedPartitions()` for double `if`
where we will step into a `synchronized` block only for first several concurrent messages

**Cherry-pick to `6.1.x`**
2023-06-08 11:23:09 -04:00
Artem Bilan
8b8a7a4c16 Fix DSL for inner bean names generation (#8639)
The `IntegrationFlowBeanPostProcessor.processIntegrationComponentSpec()`
uses a wrong `generateBeanName()` for component to register making the
provided `id` as a prefix

* Use `generateBeanName(Object instance, String prefix, @Nullable String fallbackId, boolean useFlowIdAsPrefix)`
instead to properly "fallback" to the provided name

**Cherry-pick to `6.1.x`**
2023-06-08 09:50:32 -04:00
abilan
79408d7b2c Fix observation race condition in WebFlux test
There is a race condition when we already have a reply,
but the span in the last channel is not closed yet.
2023-06-07 19:10:47 -04:00
abilan
7655d97cee Fix code typos in filter.adoc & router.adoc
* Fix race condition in the `JdbcPollingChannelAdapterParserTests`
making SELECT and UPDATE as a part of the same transaction
* Remove `inProcess = JAVA_EXEC` from ASCIIDoc Gradle tasks in attempt
to make them working in parallel
2023-06-07 15:19:03 -04:00
abilan
0cf81dbbbd Fix WebFluxObservationPropagationTests for HTTP
The `WebTestClient` was previously binding directly to the web layer (no HTTP layer involved).
Because the new instrumentation is done at the HTTP level
(it's required to fully capture error handling and more),
 the test client must bind at the HTTP level with a `Connector`.
2023-06-07 14:12:38 -04:00
abilan
e9f7780e84 Fix race condition in DebeziumMessProducerTests
The `then(debeziumEngineMock).should().run()` cannot be just checked
after `debeziumMessageProducer.start()`: the `DebeziumEngine` is really
started on a separate thread.

* Check for a `run()` interaction with the mock already after calling
`debeziumMessageProducer.stop()`.
The `stop()` waits for an internal `latch` which is fulfilled when
`DebeziumEngine` exists from its `run()` cycle
* Rename `DebeziumMessageProducer.latch` to `lifecycleLatch` to give it
more sense.
2023-06-07 13:55:11 -04:00
abilan
c24d10c8ae Fix deprecations in test from SF
Related to https://github.com/spring-projects/spring-framework/issues/30013

The `WebHttpHandlerBuilder` customization with an `ObservationRegistry`
doesn't add a `SERVER` trace as it was with deprecated `ServerHttpObservationFilter`
2023-06-07 13:12:51 -04:00
abilan
cc113e1eed Fix IntegrationFlowAdapter sample in docs
Java DSL has been improved since the last time this sample
has been added into docs
2023-06-06 10:12:51 -04:00
Christian Tzolov
c9023d114b GH-8632: Add DSL for Debezium module
Fixes https://github.com/spring-projects/spring-integration/issues/8632

* Debezium DSL initial support
* additional dsl debezium factory
* debezium dsl improvements and tests
* impove debezium docs and streamline dsl testing
* docs clarifications
* fix doc cross-reference
* updgrade debezium to 2.2.1.Final. Clean docs
* fix multiflow config tests
* improve batch tests
* Code and doc formatting
* Make `name` Debezium property as random according to its docs:
```
Unique name for the connector.
Attempting to register again with the same name fails.
This property is required by all Kafka Connect connectors.
```
* Code style clean up
2023-06-02 11:07:22 -04:00