Commit Graph

11470 Commits

Author SHA1 Message Date
Artem Bilan
ce5fabb22a Upgrade to Mockito 5.5.0 2023-09-06 15:05:46 -04:00
Artem Bilan
b8378fc7dc Refactor KafkaMDChA for code duplication (#8724)
Simplify a `KafkaMessageDrivenChannelAdapter.enhanceHeadersAndSaveAttributes()`
method for future maintenance avoiding code duplication
2023-09-06 11:03:20 -04:00
Christian Tzolov
7b16183bc5 Bump Debezium to 2.3.3.Final 2023-09-05 13:45:11 -04:00
Artem Bilan
eef070a3d9 Some code clean up for ChMessStoreQueryProvider
* Add a version for deprecation on `AbstractChannelMessageStoreQueryProvider`
* Remove dead links from `ChannelMessageStoreQueryProvider` impl JavaDocs
* Use text blocks for long query strings
* Move `SELECT_COMMON` constant to the `ChannelMessageStoreQueryProvider` interface
2023-09-01 11:15:09 -04:00
Adama Sorho
0a95091331 Deprecate AbstractChannelMessageStoreQueryProvider
* Move methods from `AbstractChannelMessageStoreQueryProvider` to `default` in the `ChannelMessageStoreQueryProvider` interface.
* Remove usage of this deprecated method.
* Fix `message-store.adoc` to mention `ChannelMessageStoreQueryProvider` instead of deprecated abstract class.
2023-09-01 10:32:20 -04:00
Adama Sorho
571ed20905 GH-8711: Add SKIP LOCKED for QueryProviders
Fixes https://github.com/spring-projects/spring-integration/issues/8711

* Add `SKIP LOCKED` into `MySqlChannelMessageStoreQueryProvider` and `PostgresChannelMessageStoreQueryProvider`
* Stop the `postgresChannelMessageTableSubscriber` after the latch waiting 10s has been passed in `testMessagesDispatchedInTransaction` test case
2023-08-31 12:51:39 -04:00
Adama Sorho
32eba4ecb9 GH-8692 Add createIndexes to MongoDbMessageStore
Fixes https://github.com/spring-projects/spring-integration/issues/8692

* Added `createIndexes` in `AbstractConfigurableMongoDbMessageStore`
* Added Javadoc for `setCreateIndex()` method
* Removed `afterPropertiesSet()` in `MongoDbChannelMessageStore` and update `whats-new.adoc` and `mongodb.adoc` files

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-08-30 11:55:13 -04:00
Artem Bilan
3f5f32b1e0 Bring back provider for modifiedFiles
Turns out Antora build fails on `grgit.status()`
therefore we have to defer its evaluation until the
`updateCopyrights` is called

* Fix some typos in `whats-new.adoc`
2023-08-28 16:11:31 -04:00
Spring Builds
205a6dcd43 [artifactory-release] Next development version 2023-08-22 23:16:20 +00:00
Spring Builds
bf7d4a2015 [artifactory-release] Release version 6.2.0-M2 2023-08-22 23:16:17 +00:00
Artem Bilan
603c7cdad5 Upgrade deps; prepare for release 2023-08-22 15:12:48 -07:00
Artem Bilan
4e310bbad6 GH-8705 Expose errorOnTimeout on MessagingGateway
Fixes https://github.com/spring-projects/spring-integration/issues/8705

an internal `MethodInvocationGateway` is a `MessagingGatewaySupport`
extension with all the logic available there.
One of the option introduced in `5.2.2` to be able to throw a `MessageTimeoutException`
instead of returning `null` when no reply received in time from downstream flow

* Expose an `errorOnTimeout` on the `@MessagingGateway` and `GatewayEndpointSpec`
* Propagate this option from a `GatewayProxyFactoryBean` down to its internal
`MethodInvocationGateway` implementation
* Modify couple tests to react for `errorOnTimeout` set to `true`
* Document the feature

Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-08-22 15:11:47 -07:00
Artem Bilan
d85c5e3a0a GH-8704: Add global property for defaultTimeout (#8706)
* GH-8704: Add global property for `defaultTimeout`

Fixes https://github.com/spring-projects/spring-integration/issues/8704

The default timeout for requests and replies in the integration endpoints
is 30 seconds to avoid indefinite blocking in threads.
Sometime those 30 seconds is not enough.

* Introduce a `spring.integration.endpoints.defaultTimeout` global property
to allow overriding all the timeouts to desired value.
The negative number indicates an indefinite waiting time: similar to what
was there before introducing 30 seconds by default

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-08-22 18:04:31 -04:00
Artem Bilan
e4bacc3e92 GH-8708: Fix concurrency around SFTP client (#8709)
Fixes https://github.com/spring-projects/spring-integration/issues/8708

According to the `org.apache.sshd.common.channel.ChannelAsyncOutputStream.writeBuffer()` JavaDocs cannot be used concurrently.

* Introduce internal `DefaultSftpSessionFactory.ConcurrentSftpClient` extension
of the `DefaultSftpClient` to set a `Lock` around `super.send(cmd, buffer);`
* Remove lock from the `SftpSession` since it now is managed by the mentioned `ConcurrentSftpClient`

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-08-22 14:00:34 -04:00
Artem Bilan
e900a53fc8 Fix JavaDoc in the DefaultSftpSessionFactory 2023-08-18 16:50:21 -04:00
EddieChoCho
ef5db3059d GH-8703: Fix MessagingAnnotationPP for AOT
Fixes https://github.com/spring-projects/spring-integration/issues/8703

* Instantiate a `MessagingAnnotationBeanPostProcessor` via factory method from `MessagingAnnotationPostProcessor`
avoiding extra code generation on an explicitly provided complex `Map` for bean definition property
* Fix test to react properly to a new logic of `MessagingAnnotationBeanPostProcessor` bean registration
2023-08-16 16:31:44 -04:00
Eddie Cho
ba6d35d123 GH-8699: Fix Issue of removeLockKey()
In the current code, an `IllegalStateException` might be thrown from the try block while invoking the `removeLockKeyInnerUnlink()` method, especially when caused by key expiration (resulting in `unlinkResult == false`).

This triggers the check block, which incorrectly sets the `unlinkAvailable` flag to `false`, even if the Redis server supports the unlink operation.

As a consequence, the subsequent `removeLockKeyInnerDelete()` method is invoked when it should not be.

* `IllegalStateException` should not be thrown from try block
* Add a comment and fix Checkstyle violations

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-08-15 13:48:03 -04:00
EddieChoCho
0450a323ec GH-8699: Atomic Redis script for unlock()
Expected Behavior

Using a single Lua script to verify ownership of the lock and remove it.

Current Behavior

`unlock()` method of `RedisLock` uses two separate Redis operations:

    * `isAcquiredInThisProcess()`` method executes a `GET` operation to verify if the lock is owned by the process.
    * `removeLockKey()`` method executes `UNLINK/DEL` operation to remove the lock.

* The `removeLockKeyInnerUnlink()`, and `removeLockKeyInnerDelete()`
methods will execute a script both verify ownership of the lock and remove it.

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-08-14 10:44:57 -04:00
Artem Bilan
92fb0d3bc8 Fix failed tests according to new SF requirements 2023-08-14 09:55:38 -04:00
Artem Bilan
62b60cfe85 Fix AggregatorTests for deprecations 2023-08-12 12:58:04 -04:00
Artem Bilan
c157b9b094 Upgrade some deps; fix deprecations 2023-08-12 10:27:48 -04:00
Artem Bilan
50bf00e6a1 Various docs fixes
* Remove unused from now on assets
* Fix `nav.adoc` order
* Fix some typos in the docs
* Fix links to SF docs on Antora
* Modify Gradle build files according to tasks registration recommendations
* Fix `antora.yml` removing `jvmargs` option since this cause the build failure on Windows
2023-08-11 12:22:21 -04:00
Rob Winch
eb6ddfe429 Migrate to Antora Documentation
* Use `providers.provider` for `modifiedFiles`.
This defers the looking up of `modifiedFiles` until it is used. It
also allows other tasks to be ran from the non-primary work tree.
Working in another work tree is essential for Antora which will clone
each branch/tag into a new work tree to process it.

* Migrate Structure

* Insert explicit ids for headers

* Fix image::image

* Copy default antora files

* Fix indentation for all pages

* Split files

* Generate a default navigation

* Remove includes

* Fix cross references

* Enable Section Summary TOC for small pages

* Polish nav.adoc

* Fix duplicate ids

* Escape {firstname} and {lastname}

* Remove links to htmlsingle and pdf Formats

* Fix broken links

* Disable attributes for ${debezium-version}

* Migrate to Asciidoctor Tabs

* Move to src/reference/antora

* Remove Asciidoctor Build

* deploy-docs only spring-projects
2023-08-10 15:10:47 -04:00
Artem Bilan
f9dc75c739 GH-8697: No autowire for Int ConversionService
Fixes https://github.com/spring-projects/spring-integration/issues/8697

The `IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME` is not designed
for end-user code autowire, rather internal framework requirement

* Mark `BeanDefinition` for `IntegrationUtils.INTEGRATION_CONVERSION_SERVICE_BEAN_NAME`
as `setAutowireCandidate(false)`
2023-08-08 16:08:06 -04:00
Gary Russell
27e71abeca TCP Events Doc Polishing 2023-08-03 10:58:51 -04:00
Artem Bilan
7dcc0bb125 GH-8680: Check DB for table on start (#8690)
* GH-8680: Check DB for table on start

Fixes https://github.com/spring-projects/spring-integration/issues/8680

If database is not initialized properly before application start, we may lose messages
at runtime when we fail to insert data into DB

* Implement `SmartLifecycle` on `JdbcMessageStore`, `JdbcChannelMessageStore`,
`JdbcMetadataStore`, and `DefaultLockRepository` to perform `SELECT COUNT` query in `start()`
to fail fast if no required table is present.
* Refactor `AbstractJdbcChannelMessageStoreTests` into JUnit 5 and use `MySqlContainerTest`
for more coverage
* Fix newly failed tests which had DB not initialized before
* Exclude `commons-logging` from `commons-dbcp2` dependency to avoid
classpath conflict
* Document the new feature

* * Fix HTTP URL in the `DataSource-mysql-context.xml`

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Add `setCheckDatabaseOnStart(false)` to disable the check query for all the SI JDBC components

* Fix language in Javadocs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-07-28 14:47:49 -04:00
abilan
0c7d40d1e2 Fix EnricherSpec for method choice ambiguity 2023-07-28 12:33:55 -04:00
abilan
b6fe685873 Fix some Kotlin <-> Java DSL interoperability
* Remove generic argument from the `EnricherSpec.property()` which simply does nothing, but noise
- `Object` is enough
* Do the same for `EnricherSpec.header()`
* Fix `KotlinEnricherSpec`, respectively: no generic argument - just `Any`
* Make `KotlinIntegrationFlowDefinition.route()` to expect non-null for `router` arg
* Add `& Any` to generic params of the `KotlinRouterSpec` methods - according compiler warning requirements
2023-07-28 11:54:59 -04:00
abilan
abcc115c21 Upgrade to Gradle 8.2.1 2023-07-27 17:16:20 -04:00
Artem Bilan
b6d39e429b GH-3696: DeserializationEx support for KafkaMS (#8689)
* GH-3696: DeserializationEx support for KafkaMS

Fixes https://github.com/spring-projects/spring-integration/issues/3696

* Add an internal logic into `KafkaMessageSource` to react properly for the
`ErrorHandlingDeserializer` configuration and re-throw `DeserializationException`

* * Use `ErrorHandlingUtils` from `spring-kafka` to determine a `deserializer` from consumer properties

* Revise the test in favor of just single `await().untilAsserted()` -
no reason to wait for assignment or check for returned record.
We just need to be sure that `DeserializationException` is thrown eventually
2023-07-26 16:19:44 -04:00
abilan
2efaecdf95 Remove Gitter badge from README
We don't support Gitter chat anymore.
Everyone is advised to raise questions on StackOverflow
2023-07-26 15:53:32 -04:00
Eric Haag
53710fb804 Connect build to ge.spring.io
This change publishes a build scan to ge.spring.io for every local
build from an authenticated Spring committer and for CI where
appropriate access tokens are available. The build will not fail if
publishing fails.

This change also allows the build to benefit from remote build caching,
providing faster builds for all contributors.

Additionally, the project will have access to all features of Gradle
Enterprise such as:

- Dashboards to view all historical build scans, along with performance
   trends over time
- Build failure analytics for enhanced investigation and diagnosis of
   build failures
- Test failure analytics to better understand trends and causes around
   slow, failing, and flaky tests
2023-07-26 14:33:00 -04:00
ColoredCarrot
bd013e095c GH-8681: Expose send-buffer-overflow-strategy
Fixes https://github.com/spring-projects/spring-integration/issues/8681

* Expose `send-buffer-overflow-strategy` attribute for WebSocket XML configuration
* GH-8681: Add send-buffer-overflow-strategy to documentation
2023-07-24 11:03:50 -04:00
Artem Bilan
f4212d8df4 GH-8685: Re-fetch group after setting condition (#8686)
* GH-8685: Re-fetch group after setting condition

Fixes https://github.com/spring-projects/spring-integration/issues/8685

The `AbstractCorrelatingMessageHandler` updates the group metadata in DB
not only for provided `condition`, but also a `lastModified` field.
A subsequent scheduling for group timeout takes the `lastModified`
to compare with the value in the store after re-fetching group in task.
This does not reflect reality since adding `condition` modifies the data in DB,
but in-memory state remains the same.

* Re-fetch a group from the store in the `AbstractCorrelatingMessageHandler.setGroupConditionIfAny()`.
* Verify expected behavior via new `ConfigurableMongoDbMessageGroupStoreTests.groupIsForceReleaseAfterTimeoutWhenGroupConditionIsSet()`

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**

* * Fix Checkstyle violation in the test
2023-07-24 09:10:44 -04:00
abilan
4e464e9ce5 Try GH Actions on ubuntu22-8-32 2023-07-18 16:31:54 -04:00
abilan
5533bc86af Try upload-parallel: 10 for Maven deploy 2023-07-18 15:54:35 -04:00
abilan
79139915e8 Update GH workflows to the latest versions 2023-07-18 15:37:41 -04:00
Spring Builds
39304e399a [artifactory-release] Next development version 2023-07-18 18:47:45 +00:00
Spring Builds
9c298663bf [artifactory-release] Release version 6.2.0-M1 2023-07-18 18:47:42 +00:00
abilan
674a68def4 Upgrade to Spring Security 6.2.0 & GraphQL 1.2.2 2023-07-18 12:43:36 -04:00
Julian Koch
be53593af9 GH-8678: Add BufferOverflowStrategy for WebScoket
Within Spring Integration's WebSocket support, a `ConcurrentWebSocketSessionDecorator`,
which buffers outbound messages if sending is slow,
is used to decorate all websocket sessions in `IntegrationWebSocketContainer`,
the standard entrypoint for using websockets with Integration.

* Expose a `ConcurrentWebSocketSessionDecorator.OverflowStrategy` option on the `IntegrationWebSocketContainer`

**Cherry-pick to `5.5.x`, `6.0.x` & `6.1.x`**
2023-07-18 11:06:17 -04:00
abilan
790b8f999f Upgrade to the latest deps
* Fix `CamelMessageHandlerTests` for a new Apache Camel API
2023-07-17 16:20:09 -04:00
Artem Bilan
152666116a Upgrade to MINA SSHD 2.10.0 (#8676)
* Remove trailing slashes from remote dirs to create in SFTP tests
2023-07-17 16:19:06 -04:00
Artem Bilan
d4535cab7d GH-8674: Fix ServerKeyVerifier impl for key type (#8675)
Fixes https://github.com/spring-projects/spring-integration/issues/8674

The `ResourceKnownHostsServerKeyVerifier` does not take into account that several different
keys can be present in the known hosts resource for the same host/port

* Fix `ResourceKnownHostsServerKeyVerifier` to find a list of knows host for the requested session.
Then iterate of this result to match the key type first and then compare keys and their `revoked` marker

**Cherry-pick to `6.1.x` & `6.0.x`**
2023-07-17 14:26:39 -04:00
abilan
f527025256 Fix placeholder in JmsChannelParser for cache
Looks like spring JMS does not uppercase the `cacheLevelName` before assigning
anymore.

* Remove `toUpperCase()` for `cache` in the `JmsChannelParser` logic.
* Perform `toUpperCase()` in the `JmsChannelFactoryBean.setCacheLevelName()` instead
2023-07-17 11:07:24 -04:00
Artem Bilan
20d5f628bc GH-8644: WebSocketHandlerReg: Remove ThreadLocal (#8667)
Fixes https://github.com/spring-projects/spring-integration/issues/8644

* Introduce an inner `DynamicHandlerRegistrationProxy` class
to create an `IntegrationDynamicWebSocketHandlerRegistration`
and map it into a provided `WebSocketHandler` when a `dynamicHandlerMapping`
is in action.

Co-authored-by: pziobron <64628007+pziobron@users.noreply.github.com>
2023-07-13 10:47:00 -04:00
Gary Russell
fc3c8d2c8d GH-8668: AMQP Docs for Exclusive and S-A Consumers
Resolves https://github.com/spring-projects/spring-integration/issues/8668

**cherry-pick to all supported branches**
2023-07-12 16:50:03 -04:00
Artem Bilan
eb944335ac GH-8664: Do not use broken observeWithContext() (#8665)
Fixes https://github.com/spring-projects/spring-integration/issues/8664

When an `Observation` is turned to `NoopObservation`,
the `observeWithContext()` fails with `ClassCastException`
since `NoopObservation` serves just plain `Context` not the one we supplied

* Fix `MessagingGatewaySupport.sendAndReceiveWithObservation()` same way
as it is in version `6.0.x`
* Modify `IntegrationObservabilityZipkinTests` to reject some `Observation`
via `observationPredicate()` configuration

**Cherry-pick to `6.1.x`**
2023-07-12 14:51:24 -04:00
abilan
076b34dd6f Upgrade some dependencies to latest Milestones 2023-07-12 12:19:38 -04:00
Artem Bilan
e4c1851c07 Fix ImapIdleChA for scheduling race condition (#8670)
* Fix ImapIdleChA for scheduling race condition

The IMAP IDLE is long-lived process and can be blocked
waiting for any reply from the server.
This way it is not suited to be used in a `TaskScheduler`
especially when it has only one thread in its pool in Spring Boot by default.
Another concurrent scheduled task is exactly an `ImapMailReceiver.IdleCanceller`.
With a single thread in a `TaskScheduler` pool it cannot be reached
therefore we never cancel and IDLE task and cannot react to the connection loss properly

* Rework the `ImapIdleChannelAdapter` logic to use a regular `Executor` and `while()` loop
with a `Thread.sleep()` when we lose connection
* Clean up the `ImapMailReceiverTests` from `TaskScheduler` not used anymore.
* Expose new `taskExecutor` option in the `ImapIdleChannelAdapterSpec` for Java DSL
* Enable `ImapMailReceiverTests.testIdleWithMessageMapping()` with an attempt to see
if this fix covers an unclear problem exposed before

* * The `testIdleWithMessageMapping()` still fails on GH actions
2023-07-12 11:00:40 -04:00