Commit Graph

11424 Commits

Author SHA1 Message Date
Gary Russell
fc3c8d2c8d GH-8668: AMQP Docs for Exclusive and S-A Consumers
Resolves https://github.com/spring-projects/spring-integration/issues/8668

**cherry-pick to all supported branches**
2023-07-12 16:50:03 -04:00
Artem Bilan
eb944335ac GH-8664: Do not use broken observeWithContext() (#8665)
Fixes https://github.com/spring-projects/spring-integration/issues/8664

When an `Observation` is turned to `NoopObservation`,
the `observeWithContext()` fails with `ClassCastException`
since `NoopObservation` serves just plain `Context` not the one we supplied

* Fix `MessagingGatewaySupport.sendAndReceiveWithObservation()` same way
as it is in version `6.0.x`
* Modify `IntegrationObservabilityZipkinTests` to reject some `Observation`
via `observationPredicate()` configuration

**Cherry-pick to `6.1.x`**
2023-07-12 14:51:24 -04:00
abilan
076b34dd6f Upgrade some dependencies to latest Milestones 2023-07-12 12:19:38 -04:00
Artem Bilan
e4c1851c07 Fix ImapIdleChA for scheduling race condition (#8670)
* Fix ImapIdleChA for scheduling race condition

The IMAP IDLE is long-lived process and can be blocked
waiting for any reply from the server.
This way it is not suited to be used in a `TaskScheduler`
especially when it has only one thread in its pool in Spring Boot by default.
Another concurrent scheduled task is exactly an `ImapMailReceiver.IdleCanceller`.
With a single thread in a `TaskScheduler` pool it cannot be reached
therefore we never cancel and IDLE task and cannot react to the connection loss properly

* Rework the `ImapIdleChannelAdapter` logic to use a regular `Executor` and `while()` loop
with a `Thread.sleep()` when we lose connection
* Clean up the `ImapMailReceiverTests` from `TaskScheduler` not used anymore.
* Expose new `taskExecutor` option in the `ImapIdleChannelAdapterSpec` for Java DSL
* Enable `ImapMailReceiverTests.testIdleWithMessageMapping()` with an attempt to see
if this fix covers an unclear problem exposed before

* * The `testIdleWithMessageMapping()` still fails on GH actions
2023-07-12 11:00:40 -04:00
Artem Bilan
9597b7addb GH-8563: Fix SFTP RENAME with unconditional DEL (#8671)
Fixes https://github.com/spring-projects/spring-integration/issues/8563

* Rework the `SftpSession.rename()` logic to be similar to what is there in the `FtpSession`, for example:
try to remove a remote target file before renaming.
* Fix `SftpSession.remove()` to check for `SftpConstants.SSH_FX_NO_SUCH_FILE` response status
to return `false`
2023-07-12 10:57:03 -04:00
abilan
c5f6c126e8 Upgrade to H2 2.2.220 2023-07-10 12:05:00 -04:00
Artem Bilan
72fc79dbd5 Fix Groovy DSL for new splitter style (#8669)
* Fix Groovy DSL for new splitter style

* Introduce a new `GroovyIntegrationFlowDefinition.splitWith()`
to avoid clashing with existing method
* Deprecate all other `split()` variants, but left one without arguments
for a default splitting options
* Fix doc for new splitter style

* * Fix tab indentations in `groovy-dsl.adoc`
2023-07-10 11:08:44 -04:00
Artem Bilan
34f901f7db Improve split() DSL (#8666)
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
2023-07-03 10:35:21 -04:00
pziobron
0798c8df95 GH-8659: Fix WatchService to react for renames
Fixes https://github.com/spring-projects/spring-integration/issues/8659

* GH-8659: Updating the documentation as requested

* GH-8659: Fixes requested after coder review

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-29 14:23:33 -04:00
abilan
019e96c925 Disable ImapMailTests.testIdleWithMessageMapping
It is not clear why IMAP GreenMail server closes its socket accidentally
2023-06-28 14:35:55 -04:00
Artem Bilan
43473836ef Remove unused ThreadLocal from RedisStoreMSource (#8663)
* Remove unused ThreadLocal from RedisStoreMSource

The `RedisStoreMessageSource` relies on an `IntegrationResourceHolder`
for a while already.
Its internal `ThreadLocal` is just out of use.

* Remove that `resourceHolder` property and related methods to interact with it.
* Mention the `IntegrationResourceHolder` and `store` attribute in the `redis.adoc`

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-06-28 13:40:48 -04:00
abilan
229e3cc081 Increase wait timeout in FluxMessageChannelTests 2023-06-28 13:21:41 -04:00
abilan
1690be45fc Decrease read timeout for GreenMail tests
This may give a chance for IDLE to restart more often
to read the actual INBOX state during the test
2023-06-28 12:53:25 -04:00
Artem Bilan
4583afe0ad GH-8611: Extract MessagingAnnotationBeanPostProcessor (#8661)
Fixes https://github.com/spring-projects/spring-integration/issues/8611

According to the latest Spring Framework requirements related to AOT the
`BeanDefinitionRegistryPostProcessor` must not do anything but only bean registrations

* Extract a `MessagingAnnotationBeanPostProcessor` component with a `BeanPostProcessor` logic
from the `MessagingAnnotationPostProcessor` and leave the last one as just
a `BeanDefinitionRegistryPostProcessor` impl
* Fix tests according to a new logic
2023-06-28 08:59:37 -04:00
Artem Bilan
13980af1e1 GH-8611: Extract IntegrationConfigurationReport (#8660)
Fixes https://github.com/spring-projects/spring-integration/issues/8611

According to the latest Spring Framework requirements related to AOT the
`BeanDefinitionRegistryPostProcessor` must not do anything but only bean registrations

* Extract an `IntegrationConfigurationReport` component
and move `IntegrationProperties` printing over here out from the
`DefaultConfiguringBeanFactoryPostProcessor`
* Remove a `SmartInitializingSingleton` impl from the `DefaultConfiguringBeanFactoryPostProcessor`
2023-06-27 12:32:33 -04:00
Artem Bilan
65c7e5dc5f Fix Kotlin DSL delegation (#8658)
The `ConsumerEndpointSpec` extensions for Kotlin
don't delegate to the provided `endpointFactoryBean`

* Introduce `KotlinConsumerEndpointSpec` extension for `ConsumerEndpointSpec`
with the proper delegation to the provided spec
* Use `KotlinConsumerEndpointSpec` in the Kotlin-specific `Spec` classes

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-26 15:11:00 -04:00
Artem Bilan
af95829a3f GH-8654: Fix bean deps for messaging annotations (#8657)
Fixes https://github.com/spring-projects/spring-integration/issues/8654

Spring Framework has an ability to start dependant beans automatically
when we start the current one.

The `AbstractMethodAnnotationPostProcessor` is missing a bean dependency
registration causing errors in target applications when Messaging Annotations configuration is used.

* Add `registerDependentBean()` into an `AbstractMethodAnnotationPostProcessor` when we generate
and register a `ConsumerEndpointFactoryBean`
* Change one of the `ClientManagerBackToBackTests` configuration to rely on a `@ServiceActivator`
for `Mqttv5PahoMessageHandler` bean to ensure that change in the `AbstractMethodAnnotationPostProcessor`
has a proper effect

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-06-26 14:59:55 -04:00
Artem Bilan
070c1c6e60 GH-8626: Provide cleaner transform() DSL (#8653)
* GH-8626: Provide cleaner `transform()` DSL

Fixes https://github.com/spring-projects/spring-integration/issues/8626

* Add missed `transform(String beanName, @Nullable String methodName)` API
* Introduce a `TransformerSpec` to expose a strict API to configure
transformer variants.
* Introduce `transformWith(Consumer<TransformerSpec>)` as a single point
of all possible transformer and its endpoint options
* Deprecate those `IntegrationFlowDefinition.transform()` variants
which are harder to configure as several lambda arguments

This change will make Kotlin & Groovy DSL more readable and straightforward

* * Use new `transformWith()` in tests where a deprecated API still used
* Add JavaDocs to `TransformerSpec`
* Fix generic types for `BaseIntegrationFlowDefinition.transformWith()` -
they make sense exactly on the `TransformerSpec.transformer()` only
* Apply `transformWith()` for Groovy DSL
* Introduce a new `ClassUtils.isLambda(Object candidate)`
and add a check for Groovy `Closure`
* Fix `GroovyIntegrationFlowDefinition.createConfigurerIfAny()` to propagate a `Consumer` argument
down to the `Closure`

* * Rename `TransformerSpec -> TransformerEndpointSpec` for better context
meaning of the class
* Introduce `KotlinTransformerEndpointSpec` as an extension of the `TransformerEndpointSpec`
to have an `inline fun <reified P> transformer(crossinline function: (P) -> Any)`
for Kotlin style
* Add `KotlinIntegrationFlowDefinition.transformWith(KotlinTransformerEndpointSpec)`
* Deprecate Kotlin methods which are covered by the mentioned `transformWith()`
* Fix tests to use new API
* Mentioned the change in the doc
2023-06-26 12:01:13 -04:00
abilan
ceb3daec8e GH-8655: Add channel to JMS samples in doc
Fixes https://github.com/spring-projects/spring-integration/issues/8655

In the text about Outbound Channel Adapters for JMS it reads:

> The following configuration produces an adapter that receives Spring Integration messages from the `exampleChannel` [...]

However, the is no this channel definition in Kotlin and Java DSL snippets

* Add an `exampleChannel` to Kotlin and Java DSL samples
* Add Groovy DSL sample for the same flow definition
2023-06-26 10:51:36 -04:00
abilan
881893dbca Remove debezium-bom import
The `debezium-bom` manages too many third-party deps which causes
an enforcement for our transitive deps where it is not always true
to use old versions

* Exclude `group: 'io.netty'` for `artemis-stomp-protocol`
to avoid possible Netty version override expected by `reactor-netty-http`
2023-06-23 11:05:10 -04:00
abilan
a7d9def99b More exclude from debezium to avoid conflicts 2023-06-22 17:51:16 -04:00
abilan
f50e16e02a Exclude com.datastax.cassandra group
Rely on Cassandra driver as transitive dependency from `spring-data-cassandra`
2023-06-22 17:32:28 -04:00
abilan
d37912f6a0 Remove https://repo.spring.io/release for Gradle 2023-06-22 17:18:46 -04:00
abilan
a9285c3c2c Use assertj dep for test scope explicitly
Looks like something is overriding `assertj-core` version from `3.24.2 -> 3.11.1`
which doesn't have an API we use.

* Adding `testImplementation "org.assertj:assertj-core:$assertjVersion"` explicitly
instead of transitive from the `spring-integration-test-support` fixes the problem
2023-06-22 17:11:56 -04:00
abilan
197a58af5d Use debezium-bom 2023-06-22 16:07:17 -04:00
abilan
34904b3cd6 Upgrade to commons-io-2.13.0; downgrade sshd-sftp
* The latest `sshd-sftp-2.10.0` has a bug not removing trailing `.`
in the "unrooted" path when we ask to create a remote directory.
It works on Windows well, but fails on UNIX file systems.
* The `commons-io-2.13.0` has `Tailer` ctors deprecated
and exposes a builder API.
It starts a tailer process though unconditionally in
its own thread.
Use `setStartThread(false)` to have the tailer process
managed by our own `TaskExecutor`
* Add `ApacheCommonsFileTailingMessageProducer.setPollingDelayDuration(Duration)`
* Deprecate a `TailerListener` impl on the `ApacheCommonsFileTailingMessageProducer`
in favor of an internal instance
2023-06-22 15:41:38 -04:00
abilan
03799293ce Add @LogLevels for SftpRemoteFileTemplateTests
Not clear why `SftpRemoteFileTemplateTests.testINT3412AppendStatRmdir()`
 is failing on CI Linux, so add a tracing diagnostics to see how Apache SSHD behaves over there
2023-06-22 11:55:38 -04:00
abilan
6ded64c0cd Remove org.asciidoctor.jvm.gems plugin
Looks like we don't use any GEMs in out docs: generated PDF and HTML look OK
2023-06-22 10:55:18 -04:00
abilan
9882ce8542 Remove repo.spring.io/plugins-release from Gradle
The https://repo.spring.io/plugins-release requires an authentication now.
Replace it with a `mavenCentral()`
2023-06-22 10:29:45 -04:00
abilan
8a7293c680 An attempt to fix Sftp tests to new sshd-sftp 2023-06-22 10:19:46 -04:00
abilan
8abd72d7c0 Upgrade dependencies; fix problems
* Mockito doesn't allow to `spy()` mocks any more or "overspy"
* Deprecate `KotlinScriptExecutor` if favor of fully supported JSR223
service loaded via `DefaultScriptExecutor`
* Use only one `kotlin-scripting-jsr223` dependency
2023-06-21 17:39:55 -04:00
abilan
a147040072 Some synchronized comments clean up 2023-06-21 13:54:23 -04:00
Christian Tzolov
c38ed96ee9 GH-8643: Replace synchronized with Lock
Fixes https://github.com/spring-projects/spring-integration/issues/8643

* First pass - trivial synchronized blocks
  - Convert the "trivial" `synchronized` block into `ReentrantLock`.

* fix checkstyle

* use blocking lock

* Secon pass - handle multi-lock cases

* javadoc + year

* addres first batch of review suggestions

* fix checkstyle issues

* fix the mqtt parent/child lock monitor sharing

* fix the mqtt parent/child lock monitor sharing, v2

* patch the stomp test
2023-06-21 13:25:45 -04:00
Artem Bilan
9e9bfd0c5c Remove ThreadLocal from RequestHandlerRetryAdvice (#8650)
Related to https://github.com/spring-projects/spring-integration/issues/8644

With virtual threads it is recommended to stay away from `ThreadLocal`
variables to avoid memory exhausting with too many virtual threads

* Fix `RequestHandlerRetryAdvice` to transfer the message context via internal
`IntegrationRetryCallback` implementation.
* Cast to this `IntegrationRetryCallback` in a newly introduced internal `IntegrationRetryListener`
to extract `messageToTry` and set it into a `RetryContext`
`ErrorMessageUtils.FAILED_MESSAGE_CONTEXT_KEY` attribute
* Deprecate a usage of an external `RetryListener` implementation of the `RequestHandlerRetryAdvice`
2023-06-21 11:41:01 -04:00
Vladislav Fefelov
8a29c1e60b Optimise maybeIndex() in JsonPropertyAccessor
The `NumberFormatException` flow control is costly
operation

* Use `Character.isDigit()` check iterating through property String
instead of `NumberFormatException` flow control

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-16 13:43:28 -04:00
abilan
8315b11494 Some tweaks for ImapMailReceiverTests 2023-06-15 13:10:51 -04:00
abilan
656281e74a Use MySQLContainer for Debezium Testcontainer 2023-06-15 11:58:33 -04:00
abilan
99122d7621 Diagnostic: Add LogLevels to ImapMailReceiverTest
* Fix `com.sun.mail` to `jakarta.mail` in Mail module `log4j2-test.xml`
2023-06-15 10:54:37 -04:00
abilan
66e935f92e Fix graphql.adoc and optimize Debezium tests
* Fix some typos and update to the actual types in the `graphql.adoc`
* Don't use `@Container` in the `DebeziumMySqlTestContainer`,
but rather start it manually in the `@BeforeAll` and let one container
to survive between tests.
The Ryuk container then takes care about other containers on JVM exist
2023-06-14 14:13:18 -04:00
Artem Bilan
4db9bad50d GH-8642: Revise executors in the project (#8647)
* GH-8642: Revise executors in the project

Fixes https://github.com/spring-projects/spring-integration/issues/8642

* Rework some `Executors.newSingleThreadExecutor()` to `ExecutorServiceAdapter(new SimpleAsyncTaskExecutor())`
* Expose `TaskExecutor` setters; deprecate `ExecutorService`-based
* Some other code clean up in the effected classes: `LogAccessor`, no `synchronized` in critical blocks
* Give a meaningful prefix for default threads in the context of components, e.g. `SubscribableRedisChannel` - `getBeanName() + "-"`

* * Fix `PostgresChannelMessageTableSubscriberTests` for
`PostgresSubscribableChannel` initialization to let it create
its internal `Executor`

* Use an `AsyncTaskExecutor` injection instead of `ExecutorServiceAdapter` wrapping

* Fix `LockRegistryLeaderInitiatorTests` for `taskExecutor` injection

* Bring back `LockRegistryLeaderInitiator.setExecutorService()`
as an accident after property auto-renaming
2023-06-14 13:33:46 -04:00
Christian Tzolov
aac2adbaa3 Upgrade debezium to 2.3.0.RC1 2023-06-14 09:23:08 -04:00
Artem Bilan
a862692546 Executor instead of ThreadFactory for DebeziumMP
Related to https://github.com/spring-projects/spring-integration/issues/8642

For consistency with other Spring requirements and realignment with virtual threads,
it is better to require a `TaskExecutor` injection instead of `ThreadFactory`

* Fix `DebeziumMessageProducer` to rely on a `TaskExecutor` API instead of `ThreadFactory`
and `ExecutorService`

* * Remove unused import from the `DebeziumMessageProducerSpec`
2023-06-13 16:06:37 -04:00
abilan
b5767483e1 Improve Delayer docs around messageGroupId
* Mention in the Javadocs of the `DelayHandler`, `DelayerEndpointSpec`,
`BaseIntegrationFlowDefinition.delay()`, `GroovyIntegrationFlowDefinition.delay()`,
`KotlinIntegrationFlowDefinition.delay()` that `messageGroupId` is required option
* Explain in the docs why `messageGroupId` is required and why it cannot rely on a bean name
2023-06-13 10:59:06 -04:00
Artem Bilan
81af20aabf Improve Delayer DSL (#8645)
* Improve Delayer DSL

Move `groupId` option from a `delay()` method arg to the `DelayerEndpointSpec`
to make it cleaner from code reading perspective
* Expose new DSL method based on just a `DelayerEndpointSpec` for Kotlin &v Groovy
* Deprecate multi-arg `delay()` methods in favor of `Consumer<DelayerEndpointSpec>`-based

* * Fix language and code style
2023-06-12 16:01:40 -04:00
Gary Russell
c4ee5512f6 GH-8638: Kafka: Send All Fails to Failure Channel
Resolves https://github.com/spring-projects/spring-integration/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
2023-06-08 13:44:29 -04:00
abilan
95f1eb8c28 GH-8616: Upgrade to Angus Mail to 2.0.2
Fixes https://github.com/spring-projects/spring-integration/issues/8616

* And Greenmail to `2.1.0`
2023-06-08 13:42:16 -04:00
abilan
9752229cd1 Use UUID in Debezium tests for unique ids 2023-06-08 12:30:13 -04:00
Artem Bilan
33f0b8ec4a Optimize synchronized in PartitionedDispatcher (#8640)
Even if the `PartitionedDispatcher.populatedPartitions()`
is fast, in-memory, non-blocking operation, its active call from the `dispatch()`
on every message sent to the channel may pin the virtual thread.

* Optimize the `populatedPartitions()` for double `if`
where we will step into a `synchronized` block only for first several concurrent messages

**Cherry-pick to `6.1.x`**
2023-06-08 11:23:09 -04:00
Artem Bilan
8b8a7a4c16 Fix DSL for inner bean names generation (#8639)
The `IntegrationFlowBeanPostProcessor.processIntegrationComponentSpec()`
uses a wrong `generateBeanName()` for component to register making the
provided `id` as a prefix

* Use `generateBeanName(Object instance, String prefix, @Nullable String fallbackId, boolean useFlowIdAsPrefix)`
instead to properly "fallback" to the provided name

**Cherry-pick to `6.1.x`**
2023-06-08 09:50:32 -04:00
abilan
79408d7b2c Fix observation race condition in WebFlux test
There is a race condition when we already have a reply,
but the span in the last channel is not closed yet.
2023-06-07 19:10:47 -04:00