Commit Graph

11116 Commits

Author SHA1 Message Date
Artem Bilan
beef2e68c7 Use unique consumer groups in KafkaDslKotlinTests 2022-10-18 13:20:38 -04:00
Artem Bilan
0951738ab3 Upgrade dependencies to be ready for release
* Revert `getMethodValue()` for `CookieTests` since it is removed in the SF SNAPSHOT
* Add `@Suppress("UNCHECKED_CAST")` to `FunctionsTests.kt` to suppress compilation warning
2022-10-18 12:30:44 -04:00
Artem Bilan
64f0e738a9 GH-3897: Deprecate ChannelSecurityInterceptor (#3915)
* GH-3897: Deprecate `ChannelSecurityInterceptor`

Fixes https://github.com/spring-projects/spring-integration/issues/3897

Spring Security has deprecated `AccessDecisionManager` and all its infrastructure
in favor of `AuthorizationManager`

* Deprecate and AOP `ChannelSecurityInterceptor` and all its infrastructure,
including `@SecuredChannel` and respective XML configuration.
The `AuthorizationChannelInterceptor` added to respective channels for security
or configured as a global channel interceptor fully covers the previous AOP configuration
* Fix deprecation warnings in other tests with security

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Remove `forRemoval` attr from `@Deprecated` markers for Security classes:
looks like to mark `@Deprecated` and even `@SuppressWarnings("deprecation")`
don't silence warnings on compilation

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-10-17 18:32:17 -04:00
Artem Bilan
ff076b6a19 GH-3902: Add Kotlin Coroutines Support (#3905)
* GH-3902: Add Kotlin Coroutines Support

Fixes https://github.com/spring-projects/spring-integration/issues/3902

* Add `isAsync()` propagation from the `MessagingMethodInvokerHelper`
to the `AbstractMessageProducingHandler` to set into its `async` property.
The logic is based on a `CompletableFuture`, `Publisher` or Kotlin `suspend`
return types of the POJO method
* Introduce `IntegrationMessageHandlerMethodFactory` and `IntegrationInvocableHandlerMethod`
to extend the logic to newly introduced `ContinuationHandlerMethodArgumentResolver`
and call for Kotlin suspend functions.
* Remove `MessageHandlerMethodFactoryCreatingFactoryBean` since its logic now is covered with the
`IntegrationMessageHandlerMethodFactory`
* Kotlin suspend functions are essentially reactive, so use `CoroutinesUtils.invokeSuspendingFunction()`
and existing logic in the `AbstractMessageProducingHandler` to deal with `Publisher` reply

* Fix `GroovySplitterTests` for the current code base

* Add `kotlinx.coroutines.flow.Flow` support
The `Flow` is essentially a multi-value reactive `Publisher`,
so use `ReactiveAdapterRegistry` to convert any custom reactive streams result to `Flux` and `Mono`
which we already support as reply types

* Add docs for `Kotlin Coroutines`
Rearrange the doc a bit extracting Kotlin support to individual `kotlin-functions.adoc` file

* Fix missed link to `reactive-streams.adoc` from the `index-single.adoc`
* Fix unintended Javadocs formatting in the `AbstractMessageProducingHandler`

* Add suspend functions support for Messaging Gateway
* Add convenient `CoroutinesUtils` for Coroutines types and `Continuation` argument fulfilling via `Mono`
* Treat `suspend fun` in the `GatewayProxyFactoryBean` as a `Mono` return
* Convert `Mono` to the `Continuation` resuming in the end of gateway call

* Document `suspend fun` for `@MessagingGateway`

* * Make `async` implicitly only for `suspend fun`

* * Remove unused imports

* * Verify sync and async `Flow` processing
* Mention default sync behavior in the docs

* * Improve reflection in the `CoroutinesUtils`

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Rebase and revert blank lines around `include` in docs

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-10-17 17:53:55 -04:00
Artem Bilan
cfeaecaae8 GH-3421: Resolve throws Exception; in the API
Fixes https://github.com/spring-projects/spring-integration/issues/3421

Remove `throws Exception;` from production code to honor
the rule `Generic exceptions should never be thrown` which is enabled on SonarQube

* Rework affected usages to `try..catch` with throwing respective runtime exception
or just logging
* Some other refactoring for the affected classes
2022-10-17 16:57:24 -04:00
Artem Bilan
a08713fe87 Migrate Cassandra extension project (#3913)
* Migrate Cassandra extension project

* Add `spring-integration-cassandra` module based on the extension project
* Add Java DSL for Cassandra module
* Add documentation
* Add `CassandraContainerTest` based on a Testcontainers

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Fix `reactive-streams.adoc` for a proper link to
the new `spring-integration-cassandra` module

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-10-17 16:03:37 -04:00
Artem Bilan
83f2a9c246 Migrate Groovy DSL extension project (#3914)
* Migrate Groovy DSL extension project

* Upgrade to Groovy `4.0.5`
* Apply `groovy` plugin for Groovy module to be able to compile Groovy classes for DSL
* Document new feature

* * Restore `lambdaWrapper` for the `GroovyIntegrationFlowDefinition.handle()`
to be able to preserve a generic argument type

* * Migrate more Spock test methods to JUnit style

* * Add missed `assert` to Groovy tests

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Fix Copyright and `@since` in new classes
* Remove deprecated `IntegrationFlows` class mentions in the `groovy-dsl.adoc`

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-10-17 15:41:58 -04:00
Artem Bilan
fe06fbc241 Fix XmppConnFactoryBeanTests config for parser
The `xmppDomain` has two setters on the `XMPPTCPConnectionConfiguration.Builder`,
so the Java Bean specification may choose a wrong one by name and the
provided value may not fit into expectations

* Rework `XmppConnectionFactoryBeanTests-context.xml` to avoid ambiguity
with target setters presence
2022-10-13 15:59:18 -04:00
Artem Bilan
b7dd7d1379 Fix RetryAdviceParser for the latest spring-retry
The `FixedBackOffPolicy` and `ExponentialBackOffPolicy` have now overloaded
setters for `Supplier` of value.
The XML parser relies on the Java Bean specification and cannot determine
the proper setter by type.
Looks like it is just resolved by name and there is no guarantee which one
will win

* Add inner `FixedBackOffPolicy` and `ExponentialBackOffPolicy` adapters
to expose `*Simple` setters for the values we get from the XML configuration
2022-10-13 15:17:56 -04:00
Artem Bilan
e1d62b5478 Observation docsL: single DocsGeneratorCommand
The `micrometer-docs-generator` project now provides
an artifact which can generate all the Observation docs
with only one Gradle task
2022-10-11 16:53:14 -04:00
Artem Bilan
5ccfcbd96f Fix SftpSession for host:port resolution
The plain `SocketAddress.toString()` may resolve
to `hostName/IP` pair which is not parsed properly via `URI.getHost()`
downstream leaving `/IP` part for `uri.getPath()` request.
In the end this may lead to wrong file name to be used in the SFTP logic

* Fix `SftpSession.getHostPort()` to use `SshdSocketAddress` utility
to resolve host and port properly from the provided `SocketAddress`
2022-10-11 15:09:57 -04:00
Artem Bilan
f7dd876be2 INT-3333: Return empty list as is from DB gateway (#3907)
* INT-3333: Return empty list as is from DB gateway

Fixes https://jira.spring.io/browse/INT-3333

In `JdbcOutboundGateway` and `JpaOutboundGateway` the empty result list
is treated as "no reply" and therefore `null` is returned cause
the flow to stop at this point.
It is better to return such a result as is and the target application to
decided what to do with it, e.g. a `discardChannel` on a downstream splitter
configuration.

NOTE: the `MongoDbOutboundGateway` doesn't treat an empty result as a `null`

* * Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-10-11 14:39:29 -04:00
Artem Bilan
6641b1f545 GH-3664: Re-enable JavaScript support via GraalVM (#3911)
* GH-3664: Re-enable JavaScript support via GraalVM

Fixes https://github.com/spring-projects/spring-integration/issues/3664

The JavaScript JSR223 engine has been removed from Java.

Migrate JavaScript support into GraalVM Polyglot API:
* Implement `PolyglotScriptExecutor` which can also support other GraalVM languages
* Change the `ScriptExecutorFactory` to use a `PolyglotScriptExecutor` when it encounters JavaScript
* Re-enable tests for JavaScript
* Add GraalVM Polyglot support into docs

* * Reinstate `@EnabledIfSystemProperty` for GraalVM JS system property
2022-10-11 14:38:46 -04:00
Artem Bilan
1dd8b6bed5 Add jackson-databind to AMQP module for tests
The latest changes in Spring AMQP about migrating from RabbitMQ Hop
project to WebFlux has lead to dropping a Jackson deps which were
before transitive from Hop

* Add `com.fasterxml.jackson.core:jackson-databind` as tests dependency into AMQP module
2022-10-11 11:32:26 -04:00
Artem Bilan
c68ec19f89 Fix HttpDslTests for latest Spring Security 2022-10-11 11:21:35 -04:00
Artem Bilan
f4d7c4f5d3 INT-2086: JMS TemporaryTopic when replyPubSub
Fixes https://jira.spring.io/browse/INT-2086

Some JMS vendors handle differently a `TemporaryTopic` and `TemporaryQueue`,
so, change a `JmsOutboundGateway` to create respective temporary destination
according the `replyPubSubDomain` option set
2022-10-11 10:28:11 -04:00
Artem Bilan
a30dc10447 Improve messaging gateway mapping
The `#args` and `#gatewayMethod` SpEL variables have been deprecated for a while

* Remove their population and usage in favor of `MethodArgsHolder` `root` of the evaluation context
This change optimize a gateway mapping logic the way that there is no need in evaluation context
for every call: we can just reuse a global one
* Some other `GatewayMethodInboundMessageMapper` code style refactoring
* Fix effected test classes and their configs
2022-10-11 10:05:23 -04:00
Artem Bilan
a9f511c170 Remove reflection handling for kotlin.Unit class
There is just enough to check the value type name against "kotlin.Unit" literal.
This way we don't need extra reflection bits to be exposed into a native image
2022-10-07 12:18:32 -04:00
Artem Bilan
daaa30e67f Make replyContainer as bean in Kafka tests
It looks like `replyContainer()` is not registered as a bean,
so its lifecycle is somehow out of application context control

* Mark `replyContainer()` method as a `@Bean` in the `KafkaDslTests` and `KafkaDslKotlinTests`
to see if this fixes flaky state of the test suite
* Upgrade to Kotlin `1.7.20` and fix `KafkaDslKotlinTests` according to its requirements
2022-10-06 16:01:08 -04:00
Artem Bilan
f24fbd992b Add documentation for Observability (#3896)
* Add documentation for Observability

* Adapt Observation code to the latest dependencies
* Add doc generation tasks for meters and spans
* Document new Observation API features
* Include generated meters and spans docs to a general `metrics.adoc` chapter

* * Adapt `ObservationPropagationChannelInterceptorTests` for the latest `SpansAssert` API

* * Adjust to the latest Micrometer SNAPSHOT
* Make Observation doc generation tasks only as local.
We don't need ambiguous changes to source code on CI

* * Automate metrics/spans docs generation as a part of `reference` build phase
* Replace 'org.springframework.integration' content in the generated files with a 'o.s.i'
to make it easier to read, especially in the tables
* Break `DefaultMessageReceiverObservationConvention <=> IntegrationObservation` classes tangle
using literal for `KeyValues` in the `DefaultMessageReceiverObservationConvention`
instead of nested enums from the `IntegrationObservation`
* Some other minor build script clean up

* Fix indent in `build.gradle` for `micrometerVersion` property code line

* Add new line after observation section in whats-new.adoc

* * Adapt to the latest Micrometer changes

* * Use Reactor `2022.0.0-SNAPSHOT` version
2022-10-06 12:42:26 -04:00
Artem Bilan
a667171c4f Use unique Kafka consumer groups in tests
so, it doesn't cause unexpected rebalances in the global shared embedded Kafka broker
2022-10-06 10:50:01 -04:00
Artem Bilan
6b31d970de Remove spring-integration-gemfire module
Starting with Spring Data 2022.0.0 there is not going to be GemFire (Geode) support
directly from Spring team

A source code of the `spring-integration-gemfire` is moving to Spring Integration Extensions
from where community may consider to pull it and support in their own manner
2022-10-05 10:35:19 -04:00
Artem Bilan
fb1c89f680 AsyncGatewayTests: Fix to currentThread() method 2022-10-04 16:07:12 -04:00
Artem Bilan
5fadaf533d GH-3635: Add Future<Void> & Mono<Void> to gateway (#3899)
* GH-3635: Add Future<Void> & Mono<Void> to gateway

Fixes https://github.com/spring-projects/spring-integration/issues/3635

When `Future<Void>` & `Mono<Void>` is used as a messaging gateway return type,
the application hangs out on this barrier which may lead to the out of memory eventually

* Add support for the `Future<Void>` & `Mono<Void>` messaging gateway return type
and ensure an asynchronous call for the `gateway.send(Message)` operation and
its exception handling.
In case of successful call, the `Future` is fulfilled with `null` and `Mono` is completed as empty

* * Check for `void.class` as well in the `GatewayProxyFactoryBean.isVoidReturnType`

* * Allow `Future<Void>` as a reply type of the gateway request-reply operation

* * Fix  Checkstyle violations

* * Resolve `System.err.println()` in the test code

* Add `Thread.currentThread.interrupt()` to the `InterruptedException` block in the test
2022-10-04 16:03:04 -04:00
Artem Bilan
0eb6ae172e Fix new classes and packages tangles (#3898)
* Fix new classes and packages tangles

* Move `MessagingAnnotationPostProcessor` and `MethodAnnotationPostProcessor` impls
out of the `config.annotation` package due to usage of the `FactoryBean` configs
* Move `GenericHandler` and `GenericTransformer` to the `core` package to break
a tangle with a `LambdaMessageProcessor`
* Clean up affected tests and docs

* * Fix Checkstyle violation for imports order
2022-10-03 15:07:43 -04:00
Alexander Münch
6246c9f489 Remove superfluous space in IntegrationFlowBuilder (#3901) 2022-10-01 20:41:22 -04:00
Artem Bilan
6ce61ed7f2 Disable ResKnownHostsSerKeyVerifierTests on CI
Some server keys don't pass the verifier
2022-09-27 15:51:59 -04:00
Artem Bilan
82e3073fc9 Remove unused import from known_hosts test class 2022-09-27 14:26:32 -04:00
Artem Bilan
e32a5024ad Use hashed host for known_hosts tests
The `192.168.1.61` is Private Use IP.
It is an intranet IP address, which is usually allocated to mobile phones, desktops, laptops, TVs, smart speakers and other devices.

The logic is taken from a similar test class in the Apache MINA project
2022-09-27 13:46:24 -04:00
Artem Bilan
4aa2f91bd9 GH-3572: Migrate SFTP from jsch to sshd-sftp (#3892)
* GH-3572: Migrate SFTP from `jsch` to `sshd-sftp`

Fixes https://github.com/spring-projects/spring-integration/issues/3572

* Rework SFTP module from the JSch API to more modern `sshd-sftp`
* Migrate generics of most API from `ChannelSftp.LsEntry` to the `SftpClient.DirEntry`
* Rework `DefaultSftpSessionFactory` to deal with an `SshClient` and create `SftpClient`
wrapped to the `SftpSession`
* Implement a `ResourceKnownHostsServerKeyVerifier` to load `known-hosts` from any possible resource
* Implement an expected `SftpSession.list()` with just file name to take or pattern matching
* Remove some unused tests and their config
* Remove tests for custom `UserInfo` since we don't provide any custom out-of-the-box
* Test a new `ResourceKnownHostsServerKeyVerifier` against default `known-hosts` file

* * Some tests improvements

* * Improve generics handling for `FileUtils`
2022-09-27 13:10:11 -04:00
Artem Bilan
69176632c0 Adapt to the latest SF changes 2022-09-27 12:15:37 -04:00
Gary Russell
26816a3eef GH-3626: RabbitMQ Stream Support (#3895)
Resolves https://github.com/spring-projects/spring-integration/issues/3626

- spec for stream listener container
- move message handler from scst to here; add spec

* Fix test.

* Polishing and Docs.

* Fix anchors in doc.

* Don't Extend `AbstractMessageProducingHandler`
2022-09-26 17:43:37 -04:00
Gary Russell
cbfa150dfa GH-3661: Fix Javadocs for RetryingMLA
Use of the `RetryingMessageListenerAdapter` was removed in
0550380704
but the javadocs were not corrected.
2022-09-21 16:04:20 -04:00
Spring Builds
d825adeeda [artifactory-release] Next development version 2022-09-21 00:50:49 +00:00
Spring Builds
3304cd4bcc [artifactory-release] Release version 6.0.0-M5 2022-09-21 00:50:46 +00:00
Rafael Winterhalter
145edb238e GH-3872: Docs for PostgresSubscribableChannel
Fixes https://github.com/spring-projects/spring-integration/issues/3872

* Add documentation for `PostgresSubscribableChannel` and related components
* Add links, example and what's new section.
* Docs clean up
2022-09-20 20:12:38 -04:00
Rafael Winterhalter
14d85977d7 GH-3872: Add PostgresSubscribableChannel implementation
Fixes https://github.com/spring-projects/spring-integration/issues/3872

Adds an implementation for a Postres-compatible notification listener for a `JdbcChannelMessageStore`.

* Introduce `PostgresChannelMessageTableSubscriber.Subscription` contract
* Implement a `PostgresSubscribableChannel`

* Add Javadoc and fix code formatting issues.
* Handle temporary connection loss.
* Add tests for `PostgresChannelMessageTableSubscriber`

* Replace NOTIFY command with pg_notify function call.

* Code style clean up
* Fix some Javadocs
* Use `DataSourceInitializer` in the `PostgresChannelMessageTableSubscriberTests` to populate scripts
* Use Java text block for DB scripts
2022-09-20 16:18:48 -04:00
Artem Bilan
695e1563ed GH-3888: Fix NPE in the RedisLockRegistry.destroy() (#3889)
* GH-3888: Fix NPE in the `RedisLockRegistry.destroy()`

Fixed https://github.com/spring-projects/spring-integration/issues/3888

When `RedisLockType.SPIN_LOCK` (default), the `RedisLockRegistry.destroy()`
causes an NPE on the `redisMessageListenerContainer` since pub-sub is not used
in a busy-spin mode

* Check for `redisMessageListenerContainer` before calling its `destroy()`

**Cherry-pick to 5.5.x**

* * Reset properties in the `RedisLockRegistry` after `destroy()`
2022-09-20 14:17:37 -04:00
Artem Bilan
ab7b57939c Use Testcontainers BOM
* Rework `MongoDbContainerTest` to explicit `MongoDBContainer`
2022-09-20 13:32:48 -04:00
Artem Bilan
036b319dfd Remove unused import in IntObservZipkinTests 2022-09-20 12:52:39 -04:00
Artem Bilan
9a54c2d6cc Remove saaj dep: it comes from spring-ws-core 2022-09-20 12:44:17 -04:00
Artem Bilan
145c845178 Fix race condition in the IntObservZipkinTests
The `QueueChannel.receive()` may be fulfilled before
an observation is stopped in the `MessageHandler`

* Rework `IntegrationObservabilityZipkinTests` configuration to add
a `HandleMessageAdvice` to wait on the `CountDownLatch` before asserting spans
* Exclude an `adviceChain` attribute from the `ServiceActivatorAnnotationPostProcessor`
since an advice can be applied for the consumer endpoint, not the MH directly.
The `ConsumerEndpointFactoryBean` does the proper decision to apply advice onto MH
or just around its `handleMessage()` method
2022-09-20 12:33:59 -04:00
Artem Bilan
91a8b14b00 Move dependencies to Milestones for release
* Adapt code to the changes in upstream libs
* Change consumer groups in Kafka tests to unique values to avoid clashes
2022-09-20 11:43:36 -04:00
Artem Bilan
8c73a2d0cd Add some infrastructure for Observation (#3879)
* Add some infrastructure for Observation

* Populate an `ObservationRegistry` bean from the `IntegrationManagementConfigurer`
into all the `IntegrationManagement` components
* Introduce `MessageReceiverContext` and `MessageSenderContext` for easier usage
in the target code
* Implement `Observation` handling in the `AbstractMessageHandler`
* Modify `ObservationPropagationChannelInterceptorTests` for new `MessageSenderContext`
* Use `BridgeHandler` to ensure that `Observation` is propagated and handled properly
* Verify that tags from the `AbstractMessageHandler` are preset on the consumer span

* * Add a `DocumentedObservation` infrastructure

* * Add `Timer` verification to the propagation test

* * Update to the latest Observation API

* * Add custom observation convention support for the `AbstractMessageHandler`
* Use more meaningful prefix for Spring Integration tags

* * Move singleton instance for `DefaultMessageReceiverObservationConvention`
into `DefaultMessageReceiverObservationConvention` per se as an `INSTANCE` constant
* Use `MeterRegistryAssert` in the `ObservationPropagationChannelInterceptorTests`
to verify meters emitted

* * And an integration test with Zipkin based on the `SampleTestRunner`
2022-09-19 10:22:04 -04:00
Artem Bilan
5ece0e0dfd Add Apache Camel support (#3887)
* Add Apache Camel support

* Implement `CamelMessageHandler` to perform send and send-n-reply
operations to the Apache Camel routes
* Fix `AbstractMessageProducingHandler` to handle async errors even if reply is
not expected

* * Narrow Camel dependency to `api`
2022-09-15 16:41:25 -04:00
Artem Bilan
f39ad6f558 ZKMetadataStore: Ignore the event for root node
Turns out Linux emits a `node created` event for the node a bit later.
Therefore, even if we add a listened after creating a root node,
we still may catch it in the listener
2022-09-15 14:22:10 -04:00
Artem Bilan
878f8cd3d9 Improve some ZK tests 2022-09-15 12:35:19 -04:00
Artem Bilan
a4997c67a0 Remove extra root creation in ZKMetadataStore 2022-09-15 11:33:39 -04:00
Artem Bilan
e527c6e321 ZKMetadataStore: don't react on the root creation
There is a race condition when we add a listener to the cache and then
immediately create a root for the tree: this event may be caught or not.
Either way we don't to react for such an event
2022-09-15 10:24:41 -04:00
Artem Bilan
abd94bd58a GH-3435: Upgrade to curator-recipes 5.3.0
Fixes https://github.com/spring-projects/spring-integration/issues/3435

* Migrate `ZookeeperMetadataStore` from deprecated `PathChildrenCache` to the `CuratorCache`
and respective `CuratorCacheListener`
2022-09-14 17:13:36 -04:00