Commit Graph

11075 Commits

Author SHA1 Message Date
Artem Bilan
145c845178 Fix race condition in the IntObservZipkinTests
The `QueueChannel.receive()` may be fulfilled before
an observation is stopped in the `MessageHandler`

* Rework `IntegrationObservabilityZipkinTests` configuration to add
a `HandleMessageAdvice` to wait on the `CountDownLatch` before asserting spans
* Exclude an `adviceChain` attribute from the `ServiceActivatorAnnotationPostProcessor`
since an advice can be applied for the consumer endpoint, not the MH directly.
The `ConsumerEndpointFactoryBean` does the proper decision to apply advice onto MH
or just around its `handleMessage()` method
2022-09-20 12:33:59 -04:00
Artem Bilan
91a8b14b00 Move dependencies to Milestones for release
* Adapt code to the changes in upstream libs
* Change consumer groups in Kafka tests to unique values to avoid clashes
2022-09-20 11:43:36 -04:00
Artem Bilan
8c73a2d0cd Add some infrastructure for Observation (#3879)
* Add some infrastructure for Observation

* Populate an `ObservationRegistry` bean from the `IntegrationManagementConfigurer`
into all the `IntegrationManagement` components
* Introduce `MessageReceiverContext` and `MessageSenderContext` for easier usage
in the target code
* Implement `Observation` handling in the `AbstractMessageHandler`
* Modify `ObservationPropagationChannelInterceptorTests` for new `MessageSenderContext`
* Use `BridgeHandler` to ensure that `Observation` is propagated and handled properly
* Verify that tags from the `AbstractMessageHandler` are preset on the consumer span

* * Add a `DocumentedObservation` infrastructure

* * Add `Timer` verification to the propagation test

* * Update to the latest Observation API

* * Add custom observation convention support for the `AbstractMessageHandler`
* Use more meaningful prefix for Spring Integration tags

* * Move singleton instance for `DefaultMessageReceiverObservationConvention`
into `DefaultMessageReceiverObservationConvention` per se as an `INSTANCE` constant
* Use `MeterRegistryAssert` in the `ObservationPropagationChannelInterceptorTests`
to verify meters emitted

* * And an integration test with Zipkin based on the `SampleTestRunner`
2022-09-19 10:22:04 -04:00
Artem Bilan
5ece0e0dfd Add Apache Camel support (#3887)
* Add Apache Camel support

* Implement `CamelMessageHandler` to perform send and send-n-reply
operations to the Apache Camel routes
* Fix `AbstractMessageProducingHandler` to handle async errors even if reply is
not expected

* * Narrow Camel dependency to `api`
2022-09-15 16:41:25 -04:00
Artem Bilan
f39ad6f558 ZKMetadataStore: Ignore the event for root node
Turns out Linux emits a `node created` event for the node a bit later.
Therefore, even if we add a listened after creating a root node,
we still may catch it in the listener
2022-09-15 14:22:10 -04:00
Artem Bilan
878f8cd3d9 Improve some ZK tests 2022-09-15 12:35:19 -04:00
Artem Bilan
a4997c67a0 Remove extra root creation in ZKMetadataStore 2022-09-15 11:33:39 -04:00
Artem Bilan
e527c6e321 ZKMetadataStore: don't react on the root creation
There is a race condition when we add a listener to the cache and then
immediately create a root for the tree: this event may be caught or not.
Either way we don't to react for such an event
2022-09-15 10:24:41 -04:00
Artem Bilan
abd94bd58a GH-3435: Upgrade to curator-recipes 5.3.0
Fixes https://github.com/spring-projects/spring-integration/issues/3435

* Migrate `ZookeeperMetadataStore` from deprecated `PathChildrenCache` to the `CuratorCache`
and respective `CuratorCacheListener`
2022-09-14 17:13:36 -04:00
Artem Bilan
0f7b3e66dc Remove redundant dokkaHtml from build.gradle
The `build.gradle` contains two `dokkaHtmlPartial` configs
where the first one is redundant and apparently is overridden
by the second one.
The second one is correct any way.

* Remove the first `dokkaHtmlPartial` definition since it is
misleading and redundant
2022-09-14 14:38:41 -04:00
Artem Bilan
4758fa2473 Fix STOMP module for latest SF changes 2022-09-12 12:18:04 -04:00
Artem Bilan
26a4c4dfac Fix WebSocket module for latest SF 2022-09-12 11:36:31 -04:00
Artem Bilan
62619804b6 fix HTTP module according latest SF changes 2022-09-12 10:52:18 -04:00
Artem Bilan
df1b8b4422 GH-3885: Change LockRepository to LocalDateTime
Fixed https://github.com/spring-projects/spring-integration/issues/3885

Turns out the `Date` is stored into DB without time zone, so instances
in different zones may not react properly for updates

* Change the logic in the `DefaultLockRepository` to store time via
`LocalDateTime.now(ZoneOffset.UTC)` to ensure that all instance deals with the same
time representation without time zone offsets
2022-09-12 10:44:30 -04:00
Artem Bilan
e2a5b7d9a3 Improve AOT hints code 2022-09-12 10:04:57 -04:00
Artem Bilan
47cfae9d14 Close volatile resource in some Kafka tests 2022-09-07 17:27:39 -04:00
Artem Bilan
96d3b005fb Add @DirtiesContext to a couple Kafka tests 2022-09-07 16:05:48 -04:00
Artem Bilan
f36181bc5a GH-3660: Invoke well-known lambdas explicitly
Fixes https://github.com/spring-projects/spring-integration/pull/3660

* Change the `LambdaMessageProcessor` to invoke `Function`, `GenericHandler`,
`GenericSelector` & `GenericTransformer` explicitly without reflection.
This allows to avoid `--add-opens` for Java types to be used from reflections.
Plus this removes some overhead with native images.
And in general it is faster than reflection invocation.
2022-09-07 15:11:38 -04:00
Artem Bilan
1fbfad2d2a Fix unused import in the ObjToMapTransformerTests 2022-09-07 11:40:41 -04:00
Artem Bilan
fa85d99214 Disable ObjectToMapTransformerTests.cycle test
The `ObjectToMapTransformerTests.testObjectToSpelMapTransformerWithCycle()`
causes a `StackOverflowError` according to the parent-child-parent cycle in the model under test.
This ends up with an error in the Gradle logs:
```
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at s\src\java.instrument\share\native\libinstrument\JPLISAgent.c line: 873
```

* Disable this test to avoid memory overhead and CPU time
to let Java determine stack overflow and avoid a build error
2022-09-07 11:26:14 -04:00
Artem Bilan
0e6b70c850 Disable ObjToMapTransformerParserTests.cycle test
The `ObjectToMapTransformerParserTests.testObjectToSpelMapTransformerWithCycle()`
causes a `StackOverflowError` according to the parent-child-parent cycle in the model under test.
This ends up with an error in the Gradle logs:
```
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at s\src\java.instrument\share\native\libinstrument\JPLISAgent.c line: 873
```

* Disable this test to avoid memory overhead and CPU time
to let Java to determine stack overflow and avoid a build error
2022-09-07 10:02:50 -04:00
Artem Bilan
2ecc33e9c3 Improve AOT for some infrastructure
* Add more proxy hints for interfaces we use in the framework code
* Implement `BeanRegistrationExcludeFilter` to exlude `BeanFactoryPostProcessor` beans
* Remove `AotDetector.useGeneratedArtifacts()` from those `BFPP`s
2022-09-06 12:51:00 -04:00
Artem Bilan
15e89c3b62 Rework infra logic onto IntegrationProperties (#3881)
* Rework infra logic onto IntegrationProperties

The `Properties` bean has been deprecated since `5.5`

* Register a default `integrationGlobalProperties` as a `IntegrationProperties`
* Fix respective usages from the `Properties`

* * Remove unused constant from the `IntegrationContextUtils`

* * Fix language in the `whats-new.adoc`

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-09-06 10:50:34 -04:00
Artem Bilan
8d241c60d2 Rename CONDITION column to GROUP_CONDITION
The `CONDITION` word is reserved one in some RDBMS vendors

* Rename the `CONDITION` column to `GROUP_CONDITION` to avoid possible failures
and get rid of the conditional code on the `JdbcMessageStore`
2022-09-06 10:34:06 -04:00
Artem Bilan
8c9662a45e Expose more AOT hints
* Add `@Reflective` to methods of `Pausable` and `ManageableLifecycle`
to give them access from Control Bus which uses SpEL invocation engine -
reflection, essentially
* Add proxy hint for the `RequestReplyExchanger`, which is used for
`gateway()` definition in Java DSL
2022-09-01 15:56:34 -04:00
Artem Bilan
5080fc2f45 GH-3828: Initial Spring AOT support (#3832)
* GH-3828: Initial Spring AOT support

Fixes https://github.com/spring-projects/spring-integration/issues/3828

* Provide an infrastructure based on a new Spring AOT engine in the latest Spring Framework
* Introduce `RuntimeHintsRegistrar` impls into modules which require some reflection,
resources or proxies and serialization available in the native image
* Mark some framework method with the `@Reflective` to make their reflection
info available in the native image, for example for SpEL or JMX invocations
* Add a `GatewayProxyBeanRegistrationAotProcessor` to register proxy interfaces
info for messaging gateways (either instance of the `GatewayProxyFactoryBean`)
* Rework `ConverterRegistrar` to not use a `beanFactory.getBeansWithAnnotation()`
since it is not available after AOT phase.
Instead, register an intermediate `IntegrationConverterRegistration` bean definition
from the `IntegrationConverterInitializer`
* Refactor `GlobalChannelInterceptorInitializer` a bit according to a new logic in the
`IntegrationConverterInitializer`
* Remove `JsonNodeWrapperToJsonNodeConverter` bean registration from the
`DefaultConfiguringBeanFactoryPostProcessor` - it is added by the `ConverterRegistrar`
into the target `ConversionService`
* Fix `ParentContextTests` respectively a `JsonNodeWrapperToJsonNodeConverter` bean removal
* Refactor `XsltPayloadTransformer` to not load a `ServletContextResource`, but just use its
name for the `xslResource` condition

* * Rework AOT support according latest changes and requirements
* Remove `@Bean` reflection since it is not needed any more
* Add `AotDetector.useGeneratedArtifacts()` condition to not register beans
one more time at runtime after AOT build phase
* Fix deprecation in the WebFlux test from the latest SF
2022-09-01 13:59:07 -04:00
Artem Bilan
80eea1508e Ignore type in the MessageJacksonDeserializer
The `GenericJackson2JsonRedisSerializer` now resolves the target type
before calling `ObjectMapper.readValue()` causing a `ClassCast` in the
`StdNodeBasedDeserializer.deserializeWithType()`

* Override `MessageJacksonDeserializer.deserializeWithType()` with delegating
to the plain `deserialize()` ignoring the `TypeDeserializer`

Related to https://github.com/spring-projects/spring-data-redis/issues/2322
2022-09-01 11:40:34 -04:00
Artem Bilan
ccf41d1480 RedisChMessageStore: Add JSON serialization test 2022-09-01 11:11:57 -04:00
Artem Bilan
56aaa4a87a Fix Observation test race condition
The latch is counted down before the `Observation` is stopped
2022-08-31 16:43:54 -04:00
Gary Russell
9c4c5283ba Fix .gitignore 2022-08-31 10:28:07 -04:00
Gary Russell
ef2813c8a4 GH-3878: Fix Javadocs for JdbcChannelMessageStore
Resolves https://github.com/spring-projects/spring-integration/issues/3878

**cherry-pick to 5.5.x**
2022-08-31 10:25:03 -04:00
Artem Bilan
abac00b341 Fix flowFromSupplier sample in kotlin-dsl.adoc
**Cherry-pick to 5.5.x**
2022-08-28 09:54:57 -04:00
Artem Bilan
ca138c0c06 GH-3844: Rework messaging annotation with @Bean (#3877)
* GH-3844: Rework messaging annotation with @Bean

Fixes https://github.com/spring-projects/spring-integration/issues/3844

* Make `MessagingAnnotationPostProcessor` as a `BeanDefinitionRegistryPostProcessor`
to process bean definitions as early as possible and register respective messaging
components at that early phase
* Make bean definitions parsing logic optional for AOT and native mode since beans
have bean parsed during AOT building phase
* Introduce a `BeanDefinitionPropertiesMapper` for easier mapping
of the annotation attributes to the target `BeanDefinition`
* Remove `@Bean`-related logic from method parsing process
* Change the logic for `@Bean`-based endpoint bean names:
since we don't deal with methods on the bean definition phase, then method name
does not make sense.
It even may mislead if we `@Bean` name is based on a method by default, so we end up
with duplicated word in the target endpoint bean name.
Now we don't
* Fix `configuration.adoc` respectively for a new endpoint bean name logic
* In the end the new logic in the `AbstractMethodAnnotationPostProcessor`
is similar to XML parsers: we feed annotation attributes to the
`AbstractStandardMessageHandlerFactoryBean` impls

* * Fix language in docs and exception message
2022-08-22 12:38:23 -04:00
Artem Bilan
c1dbb02c51 GH-3875: Initialize ClassUtils as early as possible
Fixes https://github.com/spring-projects/spring-integration/issues/3875

The class `static` variables and initialization block are performed on first class access.

* Add "fake" `ClassUtils.resolvePrimitiveType(Integer.class)` call to the `IntegrationRegistrar`
to trigger `ClassUtils` initialization with the current Spring `ClassLoader`

**Cherry-pick to `5.5.x`**
2022-08-17 13:21:52 -04:00
Artem Bilan
abce38cccb Fix raw types warn in JmsOutGatewayParserTests 2022-08-17 10:07:23 -04:00
Artem Vozhdayenko
20cebfca11 GH-3685: Add docs for shared MQTT client feature
Related to https://github.com/spring-projects/spring-integration/issues/3685

* Add documentation for a new MQTT shared client feature

Add an overview with reason for the feature as well as basic
capabilities listing. Give an example with Java DSL usage for several
adapters.

* Fill `whats-new.adoc` with MQTT changes

Add a reference to MQTT documentation with info about shared MQTT client

* Couple of code review changes
2022-08-16 13:00:18 -04:00
Artem Bilan
2bfcb32628 Initial support for Micrometer Observation (#3845)
* Initial support for Micrometer Observation

* Add respective Observation dependencies
* Refactor an `AbstractMessageHandler` logic for potential Observation hooks
* Introduce an `ObservationPropagationChannelInterceptor` to propagate
an `Observation` from one thread to another through message channels

* Adds an example of propagation

* Fixed the user code and receiving spans

* * Clean up for Tracing unit test
* Make `micrometer-observation` as an `api` dep - non-optional for direct API usage

Co-authored-by: Marcin Grzejszczak <marcin@grzejszczak.pl>
2022-08-16 10:31:14 -04:00
Artem Bilan
78fa2970fa GH-3873: Remove JMS test for private method
Fixes https://github.com/spring-projects/spring-integration/issues/3873

Testing for `private` method is considered as a bad practice
and better to verify with the public method a private one is called

* Remove `JmsOutboundGatewayParserTests.gatewayWithDestExpression()`
in favor of `replyDestinationExpression` resolution in the
`JmsOutboundGatewayTests.testConnectionBreakOnReplyMessageIdCorrelation()`.
2022-08-11 16:47:20 -04:00
Artem Vozhdayenko
5f12729ed7 GH-3685: Share MQTT connection across components
Fixes https://github.com/spring-projects/spring-integration/issues/3685

Introduce some initial design.
Add a new interface `ClientManager` which will manage clients and
connections.
Use this manager in v3 topic adapter and message handler.
Add a new interface `ClientManager` which will manage clients and
connections.
Add different implementations for v3 and v5 MQTT clients.
Use this manager in v3/v5 topic adapters and message handlers.
Add a couple of unit/integration tests to cover client manager usage.
Several small code improvements after the code review:
* Improve client manager usage via providing several mutual exclusive
constructors, whether the users provides `url` or `connectionOptions`
or `clientFactory` for v3.
* Move the logger to `AbstractMqttClientManager`
* Do not inject TaskScheduler in constructor for v3 client manager
but use lazy init via `BeanFactory` and `IntegrationContextUtils`
* Other smaller code readability improvements

Add new tests with reconnect cases.
Other code improvements after the code review:
* Adjust javadocs according to standards
* Remove `setClientManager` and use exclusive ctors
* Make automatic reconnects using the v3 client instead of manually
using task scheduler

Some fixes and improvements after another code review iteration:
* Rearrange the code according to the code style guides
* Move client instance to `AbstractClientManager` with `isRunning`
method
* Fix abstract adapter/handler fields visibility and `final`ize them
where we can
* Send application event if automatic reconnect is not enabled for the
client manager

Other fixes and improvements after code review:
* Changes around fields, methods, ctors visibility
* Removed contradictory ctors
* Reduce amount of unnecessary `getClientManager() != null` checks
in logic and make it as similar as possible for client manager and the
old approach
* Use auto-reconnect where possible
* Remove manual reconnect trigger and rely on events instead to know
where to subscribe
* Do not close the connection in adapter to be able to use reconnect
logic without lose of subscriptions
* Make `ClientManager` extend `MqttComponent` so that it knows about
connection options as part of its contract
* Remove not relevant auto test cases (relying on connection close or
manual reconnect)
* Other code style smaller changes

Other fixes and improvements after code review:
* Get manual `reconnect` invocation back for v3/v5 adapters and client
managers (see bug GH-3822 for a reasoning)
* Remove unnecessary getters/setter for a listener and use adapter
class as listener instead
* Optimize MessageListener: remove redundant inner class and use a
single method reference instead of N instances per each subscribe
* Javadocs improvements

* Add Javadocs to abstract client manager
* Extract common callback add/rm logic to abstract adapter class
* Small code cleanups/fixes related to code style & simplicity, ctor
inits and unnecessary methods; eliminate unnecessary logs noise
* Remove `@LongRunningTest` for `ClientManagerBackToBackTests` as test
run time is ~6-7 secs

* Remove client factory as dependency for v3 client manager and use
plain connection properties and client persistence instead
* Add missed javadocs
* Other code style & cleanup improvements
* More code cleanup
* More Javadocs
2022-08-11 14:47:54 -04:00
Jay Bryant
99967772ed Use spring-asciidoctor-backends
Rather than `spring-doc-resources`, which is to be retired.

* Fix some problems with tabs in docs
2022-08-04 16:22:16 -04:00
Artem Bilan
690743ee0c Use Global Embedded Kafka whenever possible
For better test suite lifecycle (higher performance)
reuse one global embedded Kafka broker introduced in Spring for Apache Kafka `3.0`

Some tests have left with their own `EmbeddedKafkaBroker` definitions since they
rely on different partitions
2022-08-01 12:27:58 -04:00
Artem Bilan
5a178de7e7 Fix AMQP module for ListenableFuture deprecation
* Realign API for `CompletableFuture`
* Fix for Spring AMQP classes extracted to top level
2022-07-28 17:05:10 -04:00
Artem Bilan
5572c2161d Fix deprecations around ListenableFuture (#3865)
* Fix deprecations around ListenableFuture

SF has deprecated a `ListenableFuture` and API around it

* Migrate to `CompletableFuture` everywhere a `ListenableFuture` has been used
* Suppress a deprecation for `ListenableFuture` keeping the functionality until the next version
* Resolve deprecations nad removals from the latest Spring for Apache Kafka
* Fix documentation for the `ListenableFuture` in favor of `CompletableFuture`

NOTE: the AMQP module is left as is until `ListenableFuture` deprecation is resolved in Spring AMQP

* * Restore some `ListenableFuture` test for messaging gateway
2022-07-28 09:32:01 -04:00
freyan929
2e9beada0b GH-3858: Add ImapIdleChASpec.reconnectDelay()
Fixes https://github.com/spring-projects/spring-integration/issues/3858
2022-07-25 21:46:14 -04:00
Artem Bilan
573c29793a Fix SIK module for latest SK compatibility
Spring for Apache Kafka does not produce `ListenableFuture` anymore.

* Rework `KafkaProducerMessageHandler` to deal with the `CompletableFuture` from now on
* Add support for `CompletableFuture` replies handling into `AbstractMessageProducingHandler`
* Remove redundant cast in the `DispatcherHasNoSubscribersTests`
2022-07-25 17:19:19 -04:00
Artem Bilan
0422486409 Fix for latest compatibility 2022-07-25 12:27:17 -04:00
Spring Builds
31907ec1d9 [artifactory-release] Next development version 2022-07-20 18:20:22 +00:00
Spring Builds
31cd7a16a8 [artifactory-release] Release version 6.0.0-M4 2022-07-20 18:20:18 +00:00
Artem Bilan
821f09995d Upgrade dependencies; prepare for release 2022-07-20 13:38:43 -04:00
Artem Vozhdayenko
50769d0350 GH-3839: Migrate MongoDB tests to Testcontainers
Fixes https://github.com/spring-projects/spring-integration/issues/3839

* Create a new base interface for Mongo container-based tests
* Migrate all tests to JUnit5
* Remove obsolete classes related to JUnit4 Mongo rule
* Fix code style & readability here and there, a few Sonar issues
* Fix failing test `validateWithConfiguredPollerFlow`
* * This test was failing with Mongo error
`$and/$or/$nor must be a nonempty array`. The cause was too small
polling ratio of the update query, the consecutive read queries
failed as no 'Oleg' document was in collection after ~100 ms.

Couple of changes after the pull request review:
* Get back the blank lines in test classes
* Get back the blank lines for some inner classes
* Eliminate bad renaming consequences
* Remove a couple of garbage todos
* Couple of places with code cleanup
2022-07-19 16:56:18 -04:00