Commit Graph

11098 Commits

Author SHA1 Message Date
Artem Bilan
daaa30e67f Make replyContainer as bean in Kafka tests
It looks like `replyContainer()` is not registered as a bean,
so its lifecycle is somehow out of application context control

* Mark `replyContainer()` method as a `@Bean` in the `KafkaDslTests` and `KafkaDslKotlinTests`
to see if this fixes flaky state of the test suite
* Upgrade to Kotlin `1.7.20` and fix `KafkaDslKotlinTests` according to its requirements
2022-10-06 16:01:08 -04:00
Artem Bilan
f24fbd992b Add documentation for Observability (#3896)
* Add documentation for Observability

* Adapt Observation code to the latest dependencies
* Add doc generation tasks for meters and spans
* Document new Observation API features
* Include generated meters and spans docs to a general `metrics.adoc` chapter

* * Adapt `ObservationPropagationChannelInterceptorTests` for the latest `SpansAssert` API

* * Adjust to the latest Micrometer SNAPSHOT
* Make Observation doc generation tasks only as local.
We don't need ambiguous changes to source code on CI

* * Automate metrics/spans docs generation as a part of `reference` build phase
* Replace 'org.springframework.integration' content in the generated files with a 'o.s.i'
to make it easier to read, especially in the tables
* Break `DefaultMessageReceiverObservationConvention <=> IntegrationObservation` classes tangle
using literal for `KeyValues` in the `DefaultMessageReceiverObservationConvention`
instead of nested enums from the `IntegrationObservation`
* Some other minor build script clean up

* Fix indent in `build.gradle` for `micrometerVersion` property code line

* Add new line after observation section in whats-new.adoc

* * Adapt to the latest Micrometer changes

* * Use Reactor `2022.0.0-SNAPSHOT` version
2022-10-06 12:42:26 -04:00
Artem Bilan
a667171c4f Use unique Kafka consumer groups in tests
so, it doesn't cause unexpected rebalances in the global shared embedded Kafka broker
2022-10-06 10:50:01 -04:00
Artem Bilan
6b31d970de Remove spring-integration-gemfire module
Starting with Spring Data 2022.0.0 there is not going to be GemFire (Geode) support
directly from Spring team

A source code of the `spring-integration-gemfire` is moving to Spring Integration Extensions
from where community may consider to pull it and support in their own manner
2022-10-05 10:35:19 -04:00
Artem Bilan
fb1c89f680 AsyncGatewayTests: Fix to currentThread() method 2022-10-04 16:07:12 -04:00
Artem Bilan
5fadaf533d GH-3635: Add Future<Void> & Mono<Void> to gateway (#3899)
* GH-3635: Add Future<Void> & Mono<Void> to gateway

Fixes https://github.com/spring-projects/spring-integration/issues/3635

When `Future<Void>` & `Mono<Void>` is used as a messaging gateway return type,
the application hangs out on this barrier which may lead to the out of memory eventually

* Add support for the `Future<Void>` & `Mono<Void>` messaging gateway return type
and ensure an asynchronous call for the `gateway.send(Message)` operation and
its exception handling.
In case of successful call, the `Future` is fulfilled with `null` and `Mono` is completed as empty

* * Check for `void.class` as well in the `GatewayProxyFactoryBean.isVoidReturnType`

* * Allow `Future<Void>` as a reply type of the gateway request-reply operation

* * Fix  Checkstyle violations

* * Resolve `System.err.println()` in the test code

* Add `Thread.currentThread.interrupt()` to the `InterruptedException` block in the test
2022-10-04 16:03:04 -04:00
Artem Bilan
0eb6ae172e Fix new classes and packages tangles (#3898)
* Fix new classes and packages tangles

* Move `MessagingAnnotationPostProcessor` and `MethodAnnotationPostProcessor` impls
out of the `config.annotation` package due to usage of the `FactoryBean` configs
* Move `GenericHandler` and `GenericTransformer` to the `core` package to break
a tangle with a `LambdaMessageProcessor`
* Clean up affected tests and docs

* * Fix Checkstyle violation for imports order
2022-10-03 15:07:43 -04:00
Alexander Münch
6246c9f489 Remove superfluous space in IntegrationFlowBuilder (#3901) 2022-10-01 20:41:22 -04:00
Artem Bilan
6ce61ed7f2 Disable ResKnownHostsSerKeyVerifierTests on CI
Some server keys don't pass the verifier
2022-09-27 15:51:59 -04:00
Artem Bilan
82e3073fc9 Remove unused import from known_hosts test class 2022-09-27 14:26:32 -04:00
Artem Bilan
e32a5024ad Use hashed host for known_hosts tests
The `192.168.1.61` is Private Use IP.
It is an intranet IP address, which is usually allocated to mobile phones, desktops, laptops, TVs, smart speakers and other devices.

The logic is taken from a similar test class in the Apache MINA project
2022-09-27 13:46:24 -04:00
Artem Bilan
4aa2f91bd9 GH-3572: Migrate SFTP from jsch to sshd-sftp (#3892)
* GH-3572: Migrate SFTP from `jsch` to `sshd-sftp`

Fixes https://github.com/spring-projects/spring-integration/issues/3572

* Rework SFTP module from the JSch API to more modern `sshd-sftp`
* Migrate generics of most API from `ChannelSftp.LsEntry` to the `SftpClient.DirEntry`
* Rework `DefaultSftpSessionFactory` to deal with an `SshClient` and create `SftpClient`
wrapped to the `SftpSession`
* Implement a `ResourceKnownHostsServerKeyVerifier` to load `known-hosts` from any possible resource
* Implement an expected `SftpSession.list()` with just file name to take or pattern matching
* Remove some unused tests and their config
* Remove tests for custom `UserInfo` since we don't provide any custom out-of-the-box
* Test a new `ResourceKnownHostsServerKeyVerifier` against default `known-hosts` file

* * Some tests improvements

* * Improve generics handling for `FileUtils`
2022-09-27 13:10:11 -04:00
Artem Bilan
69176632c0 Adapt to the latest SF changes 2022-09-27 12:15:37 -04:00
Gary Russell
26816a3eef GH-3626: RabbitMQ Stream Support (#3895)
Resolves https://github.com/spring-projects/spring-integration/issues/3626

- spec for stream listener container
- move message handler from scst to here; add spec

* Fix test.

* Polishing and Docs.

* Fix anchors in doc.

* Don't Extend `AbstractMessageProducingHandler`
2022-09-26 17:43:37 -04:00
Gary Russell
cbfa150dfa GH-3661: Fix Javadocs for RetryingMLA
Use of the `RetryingMessageListenerAdapter` was removed in
0550380704
but the javadocs were not corrected.
2022-09-21 16:04:20 -04:00
Spring Builds
d825adeeda [artifactory-release] Next development version 2022-09-21 00:50:49 +00:00
Spring Builds
3304cd4bcc [artifactory-release] Release version 6.0.0-M5 2022-09-21 00:50:46 +00:00
Rafael Winterhalter
145edb238e GH-3872: Docs for PostgresSubscribableChannel
Fixes https://github.com/spring-projects/spring-integration/issues/3872

* Add documentation for `PostgresSubscribableChannel` and related components
* Add links, example and what's new section.
* Docs clean up
2022-09-20 20:12:38 -04:00
Rafael Winterhalter
14d85977d7 GH-3872: Add PostgresSubscribableChannel implementation
Fixes https://github.com/spring-projects/spring-integration/issues/3872

Adds an implementation for a Postres-compatible notification listener for a `JdbcChannelMessageStore`.

* Introduce `PostgresChannelMessageTableSubscriber.Subscription` contract
* Implement a `PostgresSubscribableChannel`

* Add Javadoc and fix code formatting issues.
* Handle temporary connection loss.
* Add tests for `PostgresChannelMessageTableSubscriber`

* Replace NOTIFY command with pg_notify function call.

* Code style clean up
* Fix some Javadocs
* Use `DataSourceInitializer` in the `PostgresChannelMessageTableSubscriberTests` to populate scripts
* Use Java text block for DB scripts
2022-09-20 16:18:48 -04:00
Artem Bilan
695e1563ed GH-3888: Fix NPE in the RedisLockRegistry.destroy() (#3889)
* GH-3888: Fix NPE in the `RedisLockRegistry.destroy()`

Fixed https://github.com/spring-projects/spring-integration/issues/3888

When `RedisLockType.SPIN_LOCK` (default), the `RedisLockRegistry.destroy()`
causes an NPE on the `redisMessageListenerContainer` since pub-sub is not used
in a busy-spin mode

* Check for `redisMessageListenerContainer` before calling its `destroy()`

**Cherry-pick to 5.5.x**

* * Reset properties in the `RedisLockRegistry` after `destroy()`
2022-09-20 14:17:37 -04:00
Artem Bilan
ab7b57939c Use Testcontainers BOM
* Rework `MongoDbContainerTest` to explicit `MongoDBContainer`
2022-09-20 13:32:48 -04:00
Artem Bilan
036b319dfd Remove unused import in IntObservZipkinTests 2022-09-20 12:52:39 -04:00
Artem Bilan
9a54c2d6cc Remove saaj dep: it comes from spring-ws-core 2022-09-20 12:44:17 -04:00
Artem Bilan
145c845178 Fix race condition in the IntObservZipkinTests
The `QueueChannel.receive()` may be fulfilled before
an observation is stopped in the `MessageHandler`

* Rework `IntegrationObservabilityZipkinTests` configuration to add
a `HandleMessageAdvice` to wait on the `CountDownLatch` before asserting spans
* Exclude an `adviceChain` attribute from the `ServiceActivatorAnnotationPostProcessor`
since an advice can be applied for the consumer endpoint, not the MH directly.
The `ConsumerEndpointFactoryBean` does the proper decision to apply advice onto MH
or just around its `handleMessage()` method
2022-09-20 12:33:59 -04:00
Artem Bilan
91a8b14b00 Move dependencies to Milestones for release
* Adapt code to the changes in upstream libs
* Change consumer groups in Kafka tests to unique values to avoid clashes
2022-09-20 11:43:36 -04:00
Artem Bilan
8c73a2d0cd Add some infrastructure for Observation (#3879)
* Add some infrastructure for Observation

* Populate an `ObservationRegistry` bean from the `IntegrationManagementConfigurer`
into all the `IntegrationManagement` components
* Introduce `MessageReceiverContext` and `MessageSenderContext` for easier usage
in the target code
* Implement `Observation` handling in the `AbstractMessageHandler`
* Modify `ObservationPropagationChannelInterceptorTests` for new `MessageSenderContext`
* Use `BridgeHandler` to ensure that `Observation` is propagated and handled properly
* Verify that tags from the `AbstractMessageHandler` are preset on the consumer span

* * Add a `DocumentedObservation` infrastructure

* * Add `Timer` verification to the propagation test

* * Update to the latest Observation API

* * Add custom observation convention support for the `AbstractMessageHandler`
* Use more meaningful prefix for Spring Integration tags

* * Move singleton instance for `DefaultMessageReceiverObservationConvention`
into `DefaultMessageReceiverObservationConvention` per se as an `INSTANCE` constant
* Use `MeterRegistryAssert` in the `ObservationPropagationChannelInterceptorTests`
to verify meters emitted

* * And an integration test with Zipkin based on the `SampleTestRunner`
2022-09-19 10:22:04 -04:00
Artem Bilan
5ece0e0dfd Add Apache Camel support (#3887)
* Add Apache Camel support

* Implement `CamelMessageHandler` to perform send and send-n-reply
operations to the Apache Camel routes
* Fix `AbstractMessageProducingHandler` to handle async errors even if reply is
not expected

* * Narrow Camel dependency to `api`
2022-09-15 16:41:25 -04:00
Artem Bilan
f39ad6f558 ZKMetadataStore: Ignore the event for root node
Turns out Linux emits a `node created` event for the node a bit later.
Therefore, even if we add a listened after creating a root node,
we still may catch it in the listener
2022-09-15 14:22:10 -04:00
Artem Bilan
878f8cd3d9 Improve some ZK tests 2022-09-15 12:35:19 -04:00
Artem Bilan
a4997c67a0 Remove extra root creation in ZKMetadataStore 2022-09-15 11:33:39 -04:00
Artem Bilan
e527c6e321 ZKMetadataStore: don't react on the root creation
There is a race condition when we add a listener to the cache and then
immediately create a root for the tree: this event may be caught or not.
Either way we don't to react for such an event
2022-09-15 10:24:41 -04:00
Artem Bilan
abd94bd58a GH-3435: Upgrade to curator-recipes 5.3.0
Fixes https://github.com/spring-projects/spring-integration/issues/3435

* Migrate `ZookeeperMetadataStore` from deprecated `PathChildrenCache` to the `CuratorCache`
and respective `CuratorCacheListener`
2022-09-14 17:13:36 -04:00
Artem Bilan
0f7b3e66dc Remove redundant dokkaHtml from build.gradle
The `build.gradle` contains two `dokkaHtmlPartial` configs
where the first one is redundant and apparently is overridden
by the second one.
The second one is correct any way.

* Remove the first `dokkaHtmlPartial` definition since it is
misleading and redundant
2022-09-14 14:38:41 -04:00
Artem Bilan
4758fa2473 Fix STOMP module for latest SF changes 2022-09-12 12:18:04 -04:00
Artem Bilan
26a4c4dfac Fix WebSocket module for latest SF 2022-09-12 11:36:31 -04:00
Artem Bilan
62619804b6 fix HTTP module according latest SF changes 2022-09-12 10:52:18 -04:00
Artem Bilan
df1b8b4422 GH-3885: Change LockRepository to LocalDateTime
Fixed https://github.com/spring-projects/spring-integration/issues/3885

Turns out the `Date` is stored into DB without time zone, so instances
in different zones may not react properly for updates

* Change the logic in the `DefaultLockRepository` to store time via
`LocalDateTime.now(ZoneOffset.UTC)` to ensure that all instance deals with the same
time representation without time zone offsets
2022-09-12 10:44:30 -04:00
Artem Bilan
e2a5b7d9a3 Improve AOT hints code 2022-09-12 10:04:57 -04:00
Artem Bilan
47cfae9d14 Close volatile resource in some Kafka tests 2022-09-07 17:27:39 -04:00
Artem Bilan
96d3b005fb Add @DirtiesContext to a couple Kafka tests 2022-09-07 16:05:48 -04:00
Artem Bilan
f36181bc5a GH-3660: Invoke well-known lambdas explicitly
Fixes https://github.com/spring-projects/spring-integration/pull/3660

* Change the `LambdaMessageProcessor` to invoke `Function`, `GenericHandler`,
`GenericSelector` & `GenericTransformer` explicitly without reflection.
This allows to avoid `--add-opens` for Java types to be used from reflections.
Plus this removes some overhead with native images.
And in general it is faster than reflection invocation.
2022-09-07 15:11:38 -04:00
Artem Bilan
1fbfad2d2a Fix unused import in the ObjToMapTransformerTests 2022-09-07 11:40:41 -04:00
Artem Bilan
fa85d99214 Disable ObjectToMapTransformerTests.cycle test
The `ObjectToMapTransformerTests.testObjectToSpelMapTransformerWithCycle()`
causes a `StackOverflowError` according to the parent-child-parent cycle in the model under test.
This ends up with an error in the Gradle logs:
```
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at s\src\java.instrument\share\native\libinstrument\JPLISAgent.c line: 873
```

* Disable this test to avoid memory overhead and CPU time
to let Java determine stack overflow and avoid a build error
2022-09-07 11:26:14 -04:00
Artem Bilan
0e6b70c850 Disable ObjToMapTransformerParserTests.cycle test
The `ObjectToMapTransformerParserTests.testObjectToSpelMapTransformerWithCycle()`
causes a `StackOverflowError` according to the parent-child-parent cycle in the model under test.
This ends up with an error in the Gradle logs:
```
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at s\src\java.instrument\share\native\libinstrument\JPLISAgent.c line: 873
```

* Disable this test to avoid memory overhead and CPU time
to let Java to determine stack overflow and avoid a build error
2022-09-07 10:02:50 -04:00
Artem Bilan
2ecc33e9c3 Improve AOT for some infrastructure
* Add more proxy hints for interfaces we use in the framework code
* Implement `BeanRegistrationExcludeFilter` to exlude `BeanFactoryPostProcessor` beans
* Remove `AotDetector.useGeneratedArtifacts()` from those `BFPP`s
2022-09-06 12:51:00 -04:00
Artem Bilan
15e89c3b62 Rework infra logic onto IntegrationProperties (#3881)
* Rework infra logic onto IntegrationProperties

The `Properties` bean has been deprecated since `5.5`

* Register a default `integrationGlobalProperties` as a `IntegrationProperties`
* Fix respective usages from the `Properties`

* * Remove unused constant from the `IntegrationContextUtils`

* * Fix language in the `whats-new.adoc`

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-09-06 10:50:34 -04:00
Artem Bilan
8d241c60d2 Rename CONDITION column to GROUP_CONDITION
The `CONDITION` word is reserved one in some RDBMS vendors

* Rename the `CONDITION` column to `GROUP_CONDITION` to avoid possible failures
and get rid of the conditional code on the `JdbcMessageStore`
2022-09-06 10:34:06 -04:00
Artem Bilan
8c9662a45e Expose more AOT hints
* Add `@Reflective` to methods of `Pausable` and `ManageableLifecycle`
to give them access from Control Bus which uses SpEL invocation engine -
reflection, essentially
* Add proxy hint for the `RequestReplyExchanger`, which is used for
`gateway()` definition in Java DSL
2022-09-01 15:56:34 -04:00
Artem Bilan
5080fc2f45 GH-3828: Initial Spring AOT support (#3832)
* GH-3828: Initial Spring AOT support

Fixes https://github.com/spring-projects/spring-integration/issues/3828

* Provide an infrastructure based on a new Spring AOT engine in the latest Spring Framework
* Introduce `RuntimeHintsRegistrar` impls into modules which require some reflection,
resources or proxies and serialization available in the native image
* Mark some framework method with the `@Reflective` to make their reflection
info available in the native image, for example for SpEL or JMX invocations
* Add a `GatewayProxyBeanRegistrationAotProcessor` to register proxy interfaces
info for messaging gateways (either instance of the `GatewayProxyFactoryBean`)
* Rework `ConverterRegistrar` to not use a `beanFactory.getBeansWithAnnotation()`
since it is not available after AOT phase.
Instead, register an intermediate `IntegrationConverterRegistration` bean definition
from the `IntegrationConverterInitializer`
* Refactor `GlobalChannelInterceptorInitializer` a bit according to a new logic in the
`IntegrationConverterInitializer`
* Remove `JsonNodeWrapperToJsonNodeConverter` bean registration from the
`DefaultConfiguringBeanFactoryPostProcessor` - it is added by the `ConverterRegistrar`
into the target `ConversionService`
* Fix `ParentContextTests` respectively a `JsonNodeWrapperToJsonNodeConverter` bean removal
* Refactor `XsltPayloadTransformer` to not load a `ServletContextResource`, but just use its
name for the `xslResource` condition

* * Rework AOT support according latest changes and requirements
* Remove `@Bean` reflection since it is not needed any more
* Add `AotDetector.useGeneratedArtifacts()` condition to not register beans
one more time at runtime after AOT build phase
* Fix deprecation in the WebFlux test from the latest SF
2022-09-01 13:59:07 -04:00
Artem Bilan
80eea1508e Ignore type in the MessageJacksonDeserializer
The `GenericJackson2JsonRedisSerializer` now resolves the target type
before calling `ObjectMapper.readValue()` causing a `ClassCast` in the
`StdNodeBasedDeserializer.deserializeWithType()`

* Override `MessageJacksonDeserializer.deserializeWithType()` with delegating
to the plain `deserialize()` ignoring the `TypeDeserializer`

Related to https://github.com/spring-projects/spring-data-redis/issues/2322
2022-09-01 11:40:34 -04:00