Commit Graph

11387 Commits

Author SHA1 Message Date
Artem Bilan
ee7ad14b38 GH-8720: Check MQTT topics if not empty strings
Fixes https://github.com/spring-projects/spring-integration/issues/8720

Validate MQTT topics for empty strings in the channel adapters configuration

Use plural names for varargs params

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-09-13 16:59:05 +02:00
Adama Sorho
ba82efd1ca GH-8692 Add createIndexes to MongoDbMessageStore
Fixes https://github.com/spring-projects/spring-integration/issues/8692

* Added `createIndexes` in `AbstractConfigurableMongoDbMessageStore`
* Added Javadoc for `setCreateIndex()` method
* Removed `afterPropertiesSet()` in `MongoDbChannelMessageStore` and update `whats-new.adoc` and `mongodb.adoc` files

**Cherry-pick to `6.1.x` & `6.0.x`**

# Conflicts:
#	src/reference/antora/modules/ROOT/pages/whats-new.adoc
#	src/reference/asciidoc/mongodb.adoc
2023-08-30 11:57:18 -04:00
Artem Bilan
5a936075f4 GH-8708: Fix concurrency around SFTP client (#8709)
Fixes https://github.com/spring-projects/spring-integration/issues/8708

According to the `org.apache.sshd.common.channel.ChannelAsyncOutputStream.writeBuffer()` JavaDocs cannot be used concurrently.

* Introduce internal `DefaultSftpSessionFactory.ConcurrentSftpClient` extension
of the `DefaultSftpClient` to set a `Lock` around `super.send(cmd, buffer);`
* Remove lock from the `SftpSession` since it now is managed by the mentioned `ConcurrentSftpClient`

**Cherry-pick to `6.1.x` & `6.0.x`**
# Conflicts:
#	spring-integration-sftp/src/main/java/org/springframework/integration/sftp/session/SftpSession.java
2023-08-22 11:20:28 -07:00
Eddie Cho
86656de1ed GH-8699: Fix Issue of removeLockKey()
In the current code, an `IllegalStateException` might be thrown from the try block while invoking the `removeLockKeyInnerUnlink()` method, especially when caused by key expiration (resulting in `unlinkResult == false`).

This triggers the check block, which incorrectly sets the `unlinkAvailable` flag to `false`, even if the Redis server supports the unlink operation.

As a consequence, the subsequent `removeLockKeyInnerDelete()` method is invoked when it should not be.

* `IllegalStateException` should not be thrown from try block
* Add a comment and fix Checkstyle violations

**Cherry-pick to `6.1.x` & `6.0.x`**

(cherry picked from commit ba6d35d123)
2023-08-15 13:48:32 -04:00
EddieChoCho
156caa6b37 GH-8699: Atomic Redis script for unlock()
Expected Behavior

Using a single Lua script to verify ownership of the lock and remove it.

Current Behavior

`unlock()` method of `RedisLock` uses two separate Redis operations:

    * `isAcquiredInThisProcess()`` method executes a `GET` operation to verify if the lock is owned by the process.
    * `removeLockKey()`` method executes `UNLINK/DEL` operation to remove the lock.

* The `removeLockKeyInnerUnlink()`, and `removeLockKeyInnerDelete()`
methods will execute a script both verify ownership of the lock and remove it.

**Cherry-pick to `6.1.x` & `6.0.x`**

# Conflicts:
#	spring-integration-redis/src/main/java/org/springframework/integration/redis/util/RedisLockRegistry.java
2023-08-14 10:52:14 -04:00
Artem Bilan
cbcf725860 GH-8697: No autowire for Int ConversionService
Fixes https://github.com/spring-projects/spring-integration/issues/8697

The `IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME` is not designed
for end-user code autowire, rather internal framework requirement

* Mark `BeanDefinition` for `IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME`
as `setAutowireCandidate(false)`

(cherry picked from commit f9dc75c739)
2023-08-11 10:26:58 -04:00
abilan
92910388dd Disable IMAP testIdleWithMessageMapping() 2023-07-24 10:37:17 -04:00
Artem Bilan
11aca689c3 GH-8685: Re-fetch group after setting condition (#8686)
* GH-8685: Re-fetch group after setting condition

Fixes https://github.com/spring-projects/spring-integration/issues/8685

The `AbstractCorrelatingMessageHandler` updates the group metadata in DB
not only for provided `condition`, but also a `lastModified` field.
A subsequent scheduling for group timeout takes the `lastModified`
to compare with the value in the store after re-fetching group in task.
This does not reflect reality since adding `condition` modifies the data in DB,
but in-memory state remains the same.

* Re-fetch a group from the store in the `AbstractCorrelatingMessageHandler.setGroupConditionIfAny()`.
* Verify expected behavior via new `ConfigurableMongoDbMessageGroupStoreTests.groupIsForceReleaseAfterTimeoutWhenGroupConditionIsSet()`

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**

* * Fix Checkstyle violation in the test
2023-07-24 09:11:27 -04:00
Spring Builds
035b838ec0 [artifactory-release] Next development version 2023-07-18 17:25:35 +00:00
Spring Builds
c1bc460a06 [artifactory-release] Release version 6.1.2 2023-07-18 17:25:32 +00:00
abilan
14fcc69d0c Downgrade to Kotlin 1.8.10
Looks like the JSR223 in `1.8.20` is broken and requires major refactoring
like we have now in `6.2.0`: 8abd72d7c0
2023-07-18 12:39:03 -04:00
abilan
1b18e49132 Upgrade dependencies; prepare fpr release 2023-07-18 11:22:17 -04:00
Julian Koch
82bb183023 GH-8678: Add BufferOverflowStrategy for WebScoket
Within Spring Integration's WebSocket support, a `ConcurrentWebSocketSessionDecorator`,
which buffers outbound messages if sending is slow,
is used to decorate all websocket sessions in `IntegrationWebSocketContainer`,
the standard entrypoint for using websockets with Integration.

* Expose a `ConcurrentWebSocketSessionDecorator.OverflowStrategy` option on the `IntegrationWebSocketContainer`

**Cherry-pick to `5.5.x`, `6.0.x` & `6.1.x`**

(cherry picked from commit be53593af9)
2023-07-18 11:07:11 -04:00
Artem Bilan
90dfed55ad GH-8674: Fix ServerKeyVerifier impl for key type (#8675)
Fixes https://github.com/spring-projects/spring-integration/issues/8674

The `ResourceKnownHostsServerKeyVerifier` does not take into account that several different
keys can be present in the known hosts resource for the same host/port

* Fix `ResourceKnownHostsServerKeyVerifier` to find a list of knows host for the requested session.
Then iterate of this result to match the key type first and then compare keys and their `revoked` marker

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-07-17 14:27:27 -04:00
Gary Russell
83b5d1cd39 GH-8668: AMQP Docs for Exclusive and S-A Consumers
Resolves https://github.com/spring-projects/spring-integration/issues/8668

**cherry-pick to all supported branches**

(cherry picked from commit fc3c8d2c8d)
2023-07-12 16:50:57 -04:00
Artem Bilan
41f41ed2ca GH-8664: Do not use broken observeWithContext() (#8665)
Fixes https://github.com/spring-projects/spring-integration/issues/8664

When an `Observation` is turned to `NoopObservation`,
the `observeWithContext()` fails with `ClassCastException`
since `NoopObservation` serves just plain `Context` not the one we supplied

* Fix `MessagingGatewaySupport.sendAndReceiveWithObservation()` same way
as it is in version `6.0.x`
* Modify `IntegrationObservabilityZipkinTests` to reject some `Observation`
via `observationPredicate()` configuration

**Cherry-pick to `6.1.x`**
2023-07-12 14:52:09 -04:00
pziobron
208d7bc09a GH-8659: Fix WatchService to react for renames
Fixes https://github.com/spring-projects/spring-integration/issues/8659

* GH-8659: Updating the documentation as requested

* GH-8659: Fixes requested after coder review

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**

(cherry picked from commit 0798c8df95)
2023-06-29 14:24:18 -04:00
Artem Bilan
8f70757ed3 Fix Kotlin DSL delegation (#8658)
The `ConsumerEndpointSpec` extensions for Kotlin
don't delegate to the provided `endpointFactoryBean`

* Introduce `KotlinConsumerEndpointSpec` extension for `ConsumerEndpointSpec`
with the proper delegation to the provided spec
* Use `KotlinConsumerEndpointSpec` in the Kotlin-specific `Spec` classes

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-26 15:11:25 -04:00
Artem Bilan
8c0ffe3dd9 GH-8654: Fix bean deps for messaging annotations (#8657)
Fixes https://github.com/spring-projects/spring-integration/issues/8654

Spring Framework has an ability to start dependant beans automatically
when we start the current one.

The `AbstractMethodAnnotationPostProcessor` is missing a bean dependency
registration causing errors in target applications when Messaging Annotations configuration is used.

* Add `registerDependentBean()` into an `AbstractMethodAnnotationPostProcessor` when we generate
and register a `ConsumerEndpointFactoryBean`
* Change one of the `ClientManagerBackToBackTests` configuration to rely on a `@ServiceActivator`
for `Mqttv5PahoMessageHandler` bean to ensure that change in the `AbstractMethodAnnotationPostProcessor`
has a proper effect

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-06-26 15:02:28 -04:00
Spring Builds
dbe3eda9e8 [artifactory-release] Next development version 2023-06-20 18:10:39 +00:00
Spring Builds
c3815f9d69 [artifactory-release] Release version 6.1.1 2023-06-20 18:10:37 +00:00
abilan
9bfbb68247 Revert to Kotlin 1.8.10
The `ScriptsTests.testKotlinScript()` fail with:
```
Caused by: javax.script.ScriptException: error: cannot access 'java.io.Serializable' which is a supertype of 'kotlin.Int'. Check your module classpath for missing or conflicting dependencies
2 + bindings["payload"] as Int
  ^

	at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.asJsr223EvalResult(KotlinJsr223JvmScriptEngineBase.kt:104)
	at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.compileAndEval(KotlinJsr223JvmScriptEngineBase.kt:63)
	at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.eval(KotlinJsr223JvmScriptEngineBase.kt:31)
	at java.scripting/javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:231)
	at org.springframework.integration.scripting.jsr223.AbstractScriptExecutor.executeScript(AbstractScriptExecutor.java:84)
```
2023-06-20 13:12:19 -04:00
abilan
e6eb235db2 Upgrade dependencies; prepare for release 2023-06-20 12:46:09 -04:00
Vladislav Fefelov
25790c5574 Optimise maybeIndex() in JsonPropertyAccessor
The `NumberFormatException` flow control is costly
operation

* Use `Character.isDigit()` check iterating through property String
instead of `NumberFormatException` flow control

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-16 13:44:31 -04:00
Gary Russell
01e7091e18 GH-8638: Kafka: Send All Fails to Failure Channel
Resolves https://github.com/spring-projects/spring-integration/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
2023-06-08 13:45:09 -04:00
Artem Bilan
0350a64df1 Optimize synchronized in PartitionedDispatcher (#8640)
Even if the `PartitionedDispatcher.populatedPartitions()`
is fast, in-memory, non-blocking operation, its active call from the `dispatch()`
on every message sent to the channel may pin the virtual thread.

* Optimize the `populatedPartitions()` for double `if`
where we will step into a `synchronized` block only for first several concurrent messages

**Cherry-pick to `6.1.x`**
2023-06-08 11:23:38 -04:00
Artem Bilan
1a8450b1d8 Fix DSL for inner bean names generation (#8639)
The `IntegrationFlowBeanPostProcessor.processIntegrationComponentSpec()`
uses a wrong `generateBeanName()` for component to register making the
provided `id` as a prefix

* Use `generateBeanName(Object instance, String prefix, @Nullable String fallbackId, boolean useFlowIdAsPrefix)`
instead to properly "fallback" to the provided name

**Cherry-pick to `6.1.x`**
2023-06-08 09:51:05 -04:00
Spring Builds
6433445ee7 [artifactory-release] Next development version 2023-05-16 20:34:40 +00:00
Spring Builds
96c7f3034d [artifactory-release] Release version 6.1.0 2023-05-16 20:34:37 +00:00
abilan
af4c7d2ad4 Bring back sshd-sftp-2.9.2
Looks like there is some bug in the latest version,
so the directory cannot be created on Linux

We will revise the problem in the next version
2023-05-16 15:38:04 -04:00
abilan
a937d5ff6c Add diagnostics to SftpRemoteFileTemplateTests 2023-05-16 15:10:29 -04:00
abilan
e70d0ef5e2 Upgrade dependencies; prepare for release
* `--add-opens` for `java.lang` and `java.util`
in Groovy and Scripting modules to let `java.util.Date` scoped prototype bean
to work back with new CGLIB classpath limitations
2023-05-16 13:17:22 -04:00
abilan
3618965491 GH-8623: DefLockRepository: back to LocalDateTime
Fixes https://github.com/spring-projects/spring-integration/issues/8623

Turns out not all JDBC drivers (or RDBMS vendors) support `java.time.Instant`
mapping to their `TIMESTAMP` type.
For example the PostgreSQL fails like:
```
org.postgresql.util.PSQLException: Can't infer the SQL type to use for an instance of java.time.Instant.
```

* Use `LocalDateTime.now(ZoneOffset.UTC)` instead `Instant.now()`.
Essentially bringing back the behavior from the previous version
2023-05-16 11:00:14 -04:00
Artem Bilan
191f693377 Fix memory leak in the FluxMessageChannel (#8622)
The `FluxMessageChannel` can subscribe to any volatile `Publisher`.
For example, we can call Reactor Kafka `Sender.send()` for
input data and pass its result to the `FluxMessageChannel`
for on demand subscription.
These publishers are subscribed in the `FluxMessageChannel`
and their `Disposable` is stored in the internal `Disposable.Composite`
which currently only cleared on `destroy()`

* Extract `Disposable` from those internal `subscribe()` calls
into an `AtomicReference`.
* Use this `AtomicReference` in the `doOnTerminate()`
to remove from the `Disposable.Composite` and `dispose()`
when such a volatile `Publisher` is completed

**Cherry-pick to `6.0.x` & `5.5.x`**
2023-05-15 16:28:29 -04:00
Artem Bilan
3c0927e4ac GH-8585: Add Javadocs to Pollers & PollerFactory (#8621)
* GH-8585: Add Javadocs to Pollers & PollerFactory

Fixes https://github.com/spring-projects/spring-integration/issues/8585

* * Fix Javadoc tags order

* * One more Javadoc tags order
2023-05-15 12:50:46 -04:00
Artem Bilan
396f5fb87b GH-8014: Improve doc for Service Activator (#8619)
* GH-8014: Improve doc for Service Activator

Fixes https://github.com/spring-projects/spring-integration/issues/8014

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-05-15 12:30:14 -04:00
Artem Bilan
0916945096 GH-6827: More XSD docs for SI-ip.xsd (#8620)
Fixes https://github.com/spring-projects/spring-integration/issues/6827
2023-05-15 09:28:31 -04:00
Artem Bilan
5f1e12ea35 Introduce PartitionedChannel (#8617)
* Introduce `PartitionedChannel`

* Implement a `PartitionedChannel` as an extension of the `AbstractExecutorChannel`
* Supply this channel with a `PartitionedDispatcher` which is an extension of the `AbstractDispatcher`
* The target partition is essentially a `UnicastingDispatcher` with a single thead executor

* * Fix language in Javadocs
* Add docs for `PartitionedChannel` into `channel.adoc`
* Pre-populate partitions in the `PartitionedDispatcher`
to ensure a thread number reflection of the partition it is used for (for default `ThreadFactory`)
* Add Javadocs to `public` methods
* Add Java DSL `PartitionedChannelSpec` and respective factory methods into `Channels`
* Use `IntegrationMessageHeaderAccessor.CORRELATION_ID` header as a default partition key

* * Fix language in Javadocs
* Fix Checkstyle violations
* Mark `PartitionedDispatcher.prePopulatedPartitions()` with `synchronized`

* * Populate partitions from `PartitionedDispatcher` ctor
and when a custom `ThreadFactory` is set

* * Clear `executors` in `populatedPartitions()`

* * Bring back `synchronized populatedPartitions()` and use it in the `dispatch()`
2023-05-10 15:49:33 -04:00
abilan
32b6d823c0 Fix race condition in testFluxChannelCleanUp
Turns out `done` in the `SinkManyEmitterProcessor`
is set to `true` when we already processed all the data.
Therefore, it is better to `await().until()` for `done`
condition in the end of test
2023-05-08 11:19:54 -04:00
Artem Bilan
ffe50d2d30 GH-8613: Add JsonPropertyAccessor type for native (#8614)
Fixes https://github.com/spring-projects/spring-integration/issues/8613

If `JsonPropertyAccessor` is registered for SpEL, it would be great
to have it working in native images as well.

Since SpEL is fully based on reflection, expose
`JsonPropertyAccessor$ComparableJsonNode` and `JsonPropertyAccessor$ArrayNodeAsList`
reflection hints for their method invocations from SpEL

**Cherry-pick to `6.0.x`**
2023-05-08 10:31:23 -04:00
Artem Bilan
b999ac109c MH: Async by default for reactive output channel (#8612)
When we configure an output channel for the handler as a `FluxMessageChannel`,
in most cases we assume an async processing for the reply.
Especially this is critical when reply is a reactive type, so in the async
mode it is "flattened" by the mentioned `FluxMessageChannel`.
Therefore, it is a bit awkward to require to set async explicitly,
when we have already configured output channel as a `FluxMessageChannel`

* Remove redundant config for async from `R2dbcDslTests`
* Mention the change in the docs
2023-05-08 10:30:41 -04:00
Kazuki Shimizu
9e7b20a059 GH-8609: Fix TcpConnectorInterceptor chain propagation
Fixes https://github.com/spring-projects/spring-integration/issues/8609

* Changed to passed the self instance to `addNewConnection()` instead of argument's connection
in a `TcpConnectionInterceptorSupport` to compose a chain of interceptors from top to down.
This way the target `Sender` get the last interceptor in a chain as its connection.

**Cherry-pick to `6.0.x` & `5.5.x`**
2023-05-05 09:32:55 -04:00
Artem Bilan
59e676a4e9 Demonstrate @Publisher with an @EventListener (#8603)
* Demonstrate `@Publisher` with an `@EventListener`

The `@EventListener` from Spring Framework is a good tool for POJO configuration
to subscribe to events from an `ApplicationContext`.
The `@Publisher` is an AOP tool to publish a `Message` from POJO method call.

* Add a test and docs to demonstrate how the `@Publisher` can be used together with an `@EventListener`

* Add `@Publisher` with a `@RabbitListener` sample in docs

* * Fix FQCN for `@Queue` in the `AmqpTests`
2023-05-02 11:58:55 -04:00
Artem Bilan
90bc65ea49 GH-3866: DefLockRepository: expose query setters (#8606)
Fixes https://github.com/spring-projects/spring-integration/issues/3866

Some RDBMS vendors (or their JDBC drivers) may just log the problem
without throwing an exception.

* Expose setters for UPDATE and INSERTS queries in the `DefaultLockRepository`
to let end-user to modify them respectively, e.g. be able to add a PostgreSQL
`ON CONFLICT DO NOTHING` hint.
* Refactor `DefaultLockRepository` to `Instant` instead of `LocalDateTime`
with zone offset.
* Refactor `ttl` property to `Duration` type
* Fix `dead-lock` typo to `deadlock`
2023-05-01 17:27:20 -04:00
Artem Bilan
33d13a4649 GH-8583: Add Java & DSL samples into docs (#8608)
* GH-8583: Add Java & DSL samples into docs

Fixes https://github.com/spring-projects/spring-integration/issues/8583

When users jump into docs first thing these days they expect to see a Java DSL sample.

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Improve `router.adoc` for DSL samples
* Add links to DSL chapters

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-05-01 16:49:16 -04:00
Artem Bilan
212bd46d65 GH-3763: Add handleReactive() for Java DSL (#8605)
* GH-3763: Add `handleReactive()` for Java DSL

Fixes https://github.com/spring-projects/spring-integration/issues/3763

Add a convenient terminal operator to `BaseIntegrationFlowDefinition`
based on a `ReactiveMessageHandler`.
Also add an overload like `handleReactive(ReactiveMessageHandlerSpec)`
to let end-user to choose a protocol-specific channel adapter

* * Fix `Namespace Factory` wording in the `BaseIntegrationFlowDefinition` Javadocs

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-05-01 15:24:00 -04:00
abilan
8291fb952d Fix README for logo and required Java version 2023-04-28 11:54:39 -04:00
Artem Bilan
49096b3937 * Use burrunan/gradle-cache-action got GH actions
According to the `gradle/gradle-build-action` docs it caches only a `main` branch.
This is not appropriate for us since we really never build `main` on GH actions.

With this change we will experiment if `burrunan/gradle-cache-action` does what we would like to get from the Gradle cache feature

* Some config adjustments for `gradle-cache-action`
2023-04-26 12:48:35 -04:00
abilan
75769ba883 Back to BEFORE_EACH for ImapMailReceiverTests
Looks like there are some leftovers after interactions with
channel adapters from config
2023-04-22 09:54:09 -04:00
abilan
33aaaba149 Some ImapMailReceiverTests improvements
* Increase reconnectTimeout to avoid race conditions
* Do not stub `protected` methods
* Rearrange properties settings before spying on object
2023-04-22 09:29:54 -04:00