Commit Graph

2543 Commits

Author SHA1 Message Date
Artem Bilan
7433e41036 Add more Kotlin DSL for sub-flows (#3233)
* Add more Kotlin DSL for sub-flows

* Introduce `KotlinSplitterEndpointSpec`, `KotlinEnricherSpec` &
`KotlinFilterEndpointSpec` to avoid an explicit use of `IntegrationFlow` usage
from Kotlin DSL
* Fix `AbstractKotlinRouterSpec` to extend `ConsumerEndpointSpec` for access
to more endpoint options as it is with Java DSL for similar specs
* Introduce a `KotlinIntegrationFlowDefinition.publishSubscribe()`
for a `BroadcastCapableChannel` and `vararg` of `KotlinIntegrationFlowDefinition` builders.
This allows us to not introduce a `BroadcastPublishSubscribeSpec` wrapper for Kotlin
and let us to remove `publishSubscribeChannel()` DSL methods for `PublishSubscribeSpec`
since it is covered with the `publishSubscribe(PublishSubscribeChannel())` and
respective set of sub-flows in Kotlin DSL as subscribers
* Use new Kotlin specs in the `KotlinIntegrationFlowDefinition` instead of their
Java variants

* * Mention `kotlin-dsl.adoc` in the ToC
2020-04-01 11:06:54 -04:00
Artem Bilan
9c46b3420e Fix JsonToObjectTransformer for ClassNotFoundEx (#3228)
* Fix JsonToObjectTransformer for ClassNotFoundEx

Related to https://github.com/spring-projects/spring-integration/issues/3223

The `JsonToObjectTransformer` consults `JsonHeaders` first and tries to build
a `ResolvableType` from other type headers which may be just a string identifications.
In this case a `ClassNotFoundException` could be thrown if a `ResolvableType` cannot
be build against non-class identificators

* Add `valueTypeExpression` option to the `JsonToObjectTransformer` to let to
build a `ResolvableType` using any possible custom logic, e.g. resolving
target classes from some registry using their ids from the mentioned headers

**Cherry-pick to 5.2.x**

* * Fix English language mistakes
2020-03-27 15:42:24 -04:00
Artem Bilan
bff891b7a9 Fix new Sonar smells 2020-03-26 10:31:32 -04:00
Artem Bilan
b6bd83d548 Fix deprecation for TX; fix JMS tests; Moore-SR6
* Upgrade to Spring Data Moore-SR6
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
for deprecations in the `TransactionInterceptor`
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
consumers to expose new `TransactionManager`-based options and
deprecate `PlatformTransactionManager`-based
* Fix failing JMS tests to reuse an ActiveMQ Connection Factory with a
`trustedPackaged(*)`

**Cherry-pick to master**

# Conflicts:
#	build.gradle
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/ConsumerEndpointSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/PollerSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionHandleMessageAdvice.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionInterceptorBuilder.java
2020-03-25 13:56:37 -04:00
Gary Russell
fbf06e8bb5 GH-3181: MQTT: Support MANUAL Acks
Resolves https://github.com/spring-projects/spring-integration/issues/3181

* Doc polishing

* Rework acknowledgment into the existing `AcknowledgmentCallback`.

* Fix javadocs and doc linFix javadocs and doc linkk

* Doc polishing; explain uses of `ACKNOWLEDGMENT_CALLBACK` header.
2020-03-19 15:20:13 -04:00
Artem Bilan
3eedda6c9d GH- 3223: Build ResolvableType after mapping
Fixes https://github.com/spring-projects/spring-integration/issues/3223

The `json__TypeId__` header may have a value which cannot be resolved to the `Class` in the current classpath
So, skip `ResolvableType` building logic until we really sure that end-user wants to map JSON headers
* WARN a build exception that we can't load a class for the `json__TypeId__` when we try to build a `ResolvableType`
in the  `DefaultAmqpHeaderMapper`
* Document the negation feature for JSON headers

**Cherry-pick to 5.2.x**
2020-03-19 14:51:32 -04:00
Artem Bilan
92421f7575 Add Kotlin wrappers for router specs (#3218)
* Add Kotlin wrappers for router specs

To avoid casting and extra logic logic in the end-user code,
it is better to provide Kotlin-specific API to let end-users to
do whatever is really dictated by API and don't think about specific types to cast

* * Fix typos; code clean up
2020-03-17 12:56:45 -04:00
Artem Bilan
e9a3580ef5 Upgrade dependencies; fix new Sonar smells 2020-03-11 10:22:51 -04:00
Artem Bilan
36f96ec2c6 Document Kotlin DSL (#3210)
* Document Kotlin DSL

* Remove unused imports in `FunctionsTests.kt`

* * Fix typos in Docs
2020-03-10 16:47:41 -04:00
Artem Bilan
1511dd8748 GH-3204: Add DSL intercept() operator
Fixes https://github.com/spring-projects/spring-integration/issues/3204

* Add an `intercept(ChannelInterceptor...)` method into `BaseIntegrationFlowDefinition` 
to register one or more channel interceptors at the current flow position.
* refactor to reuse `InterceptableChannel` creation from `wireTap`
* document the new operator
2020-03-05 16:15:52 -05:00
Gary Russell
c84d264294 GH-3183: Add ReactiveRequestHandlerAdvice (#3197)
* GH-3183: Add ReactiveRequestHandlerAdvice

Fixes https://github.com/spring-projects/spring-integration/issues/3183

* Introduce a `ReactiveRequestHandlerAdvice` with a `BiFunction<Message<?>, Mono<?>, Publisher<?>>`
logic to apply a `Mono.transform()` operator for a returned from the handler `Mono` reply
* Fix `WebFluxRequestExecutingMessageHandler` to return a `Mono.then()` instead of an explicit subscription -
it happens downstream anyway during reply producing with a proper error handling, too
* Demonstrate `ReactiveRequestHandlerAdvice` in the `RSocketDslTests` - without `retry()` it fails
* Add `ConsumerEndpointSpec.customizeMonoReply()` for convenience
* Document `ReactiveRequestHandlerAdvice` feature

* * Fix language in docs
2020-03-05 13:37:54 -05:00
Artem Bilan
8a38a5e829 Merge spring-integration-kotlin-dsl project
* Migrate all the Kotlin tests to use new Kotlin DSL
* Upgrade to the latest Kotlin
* Generate KDocs
2020-03-05 13:10:32 -05:00
Artem Bilan
45a91b25de Add Reactive TX Manager support (#3201)
* Add Reactive TX Manager support

* Change `TransactionInterceptorBuilder` and `TransactionHandleMessageAdvice` to reply
on a generic `TransactionManager`, so we can configure reactive one as well
* Change a `<transactional>` XML element to support a generic `TransactionManager` reference,
so we can configure reactive one as well
* Support `adviceChain` configuration for the `ReactiveMessageHandler` in the `ConsumerEndpointFactoryBean`
* Attemp to apply reactive transaction for the Reactive MongoDB channel adapter

* * Revert `MongoDbTests` change -
to support transactions we need the latest MongoDb server with replica enabled
* Document reactive transactions

* * Address PR reviews
2020-03-03 11:54:29 -05:00
Artem Bilan
e6e9f45164 Use Mono.create instead of Mono.fromCallable
It is better to poll message from `QueueChannel`
when ever a reactive request happens.
The `Mono.fromCallable()` operator does poll on subscription, not request
and caches the value.
However we may lose such a value in between.

* Use `Mono.create()` with its `monoSink.onRequest()` callback in the
`adaptPollableChannelToPublisher()` implementation to defer `inputChannel.receive()`
until an on demand request downstream
2020-03-02 16:57:29 -05:00
Tim Feuerbach
c6cf261e25 GH-3202: Fix tests failing with non-English locales
Fixes https://github.com/spring-projects/spring-integration/issues/3202

Set the locale to en-US for tests that assert error messages
2020-03-02 14:03:27 -05:00
Artem Bilan
0e5258a661 Add log() to test Publisher for diagnostics
https://build.spring.io/browse/INT-MASTER-1916/
2020-02-27 18:12:31 -05:00
Artem Bilan
87c8e47a88 GH-3192: pub-sub DSL for broker-backed channels (#3193)
* GH-3192: pub-sub DSL for broker-backed channels

Fixes https://github.com/spring-projects/spring-integration/issues/3192

* Introduce a `BroadcastCapableChannel` abstract to indicate those `SubscribableChannel`
implementations which can provide a pub-sub functionality
* Implement a `BroadcastCapableChannel` in broker-baked channels with pub-sub option
* Introduce a `BaseIntegrationFlowDefinition.publishSubscribeChannel()` based
on the `BroadcastCapableChannel` and `BroadcastPublishSubscribeSpec` to let to
configure sub-flow subscribers in fluent manner

* * Add some JavaDocs and document new feature

* * Show the channel bean definition in the doc
* Fix typo
2020-02-25 13:01:37 -05:00
Artem Bilan
d61a247f4b Fix ReactiveStreamsConsumer for PollableChannel
https://build.spring.io/browse/INT-MASTERSPRING40-985/

* Move `onSubscribe()` callback to the `subscribe()` operator
to honor a `request` contract from the target subscriber
2020-02-18 12:37:24 -05:00
Artem Bilan
6d936d9774 Fix some Sonar smells 2020-02-14 17:57:58 -05:00
Artem Bilan
8c4675aa83 Fix new Sonar smell
* Make `adaptPollableChannelToPublisher()` as non-blocking
2020-02-13 14:11:21 -05:00
Artem Bilan
a0cdfd90ee Change adaptPollableChannelToPublisher to Mono
We can use a `Mono.fromCallable()` in combination with `repeat()`
instead of possible race condition deal in the `Flux.create()`
2020-02-12 12:09:31 -05:00
Artem Bilan
ede9528c19 Fix some tests for not closed application context 2020-02-11 12:43:33 -05:00
Artem Bilan
a8c471c061 Fix ReactiveStreamsConsumer for error handling
To support `onErrorContinue()` logic for the plain `Subscriber`
we need to wrap its `onNext()` into a `try..catch` and respective
`errorHandler` in the `ReactiveStreamsConsumer`
2020-02-10 16:27:59 -05:00
Artem Bilan
6591ce90f6 Fix ReactiveStreamsConsumer for plain subscriber
https://build.spring.io/browse/INT-MASTERSPRING40-978

Investigate a behaviour for `ReactiveStreamsConsumer`
when we use a plain `Subscriber` directly instead of `doOn...` callbacks.
It looks like there is some race condition when the data can be consumed
upstream, but there is no consumer downstream ready yet
2020-02-10 16:00:14 -05:00
Artem Bilan
867a8cf108 GH-3155: Add support for Java DSL extensions (#3167)
* GH-3155: Add support for Java DSL extensions

Fixes https://github.com/spring-projects/spring-integration/issues/3155

Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.

* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature

* * Fix language and typos in docs

* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods

* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
2020-02-07 13:40:39 -05:00
Jayadev Sirimamilla
1d9a818efe GH-3157: pop sequence for message as well
Fixes https://github.com/spring-projects/spring-integration/issues/3157

* Fixed issue with sequences not being popped when Output processor returns a message.

* Updated documents and added additional Checks to the testcase to ensure proper `popSequence` is done.

* Updated documents and added corrected code formatting

* Updated documents and used headerAccessor Constants for matching headers
* Clean up code style
* Improve docs for the feature on the matter
2020-02-04 11:37:19 -05:00
Artem Bilan
e5740f253c Upgrade dependencies
* Gradle 6.1.1
* MongoDb 4.0
* Other Spring dependencies to `BUILD-SNAPSHOT`
* Fix MongoDb tests according latest Spring Data
* State in the docs that both MongoDb driver are `optional` now in the dependencies
2020-02-03 13:12:50 -05:00
Artem Bilan
d792915be6 Add debug log for ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTER-1883/
2020-01-31 15:16:57 -05:00
Artem Bilan
2f4394d8b5 Populate a JsonHeaders.RESOLVABLE_TYPE on reply (#3163)
* Populate a JsonHeaders.RESOLVABLE_TYPE on reply

Fixes https://github.com/spring-projects/spring-integration-samples/issues/277

In the `AbstractAmqpOutboundEndpoint` the `JsonHeaders.RESOLVABLE_TYPE`
from request message is copied to reply message making inconsistency downstream.
The `JsonToObjectTransformer` consults first a `JsonHeaders.RESOLVABLE_TYPE`
and deserialize payload to wrong type

* Fix `DefaultAmqpHeaderMapper` to populate a `JsonHeaders.RESOLVABLE_TYPE`
alongside with other `JsonHeaders` populated from the reply AMQP message.
This way a `JsonHeaders.RESOLVABLE_TYPE` from request message won't have effect
* To get access to classes, supply `AbstractHeaderMapper` with a bean factory `ClassLoader`
* Introduce a couple utility methods into `JsonHeaders` for building a `ResolvableType`

**Cherry-pick to 5.2.x**

* * Fix code formatting for arguments wrapping
2020-01-31 14:17:19 -05:00
Artem Bilan
34bc8116df Add @LogLevels to ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTERSPRING40-967/
2020-01-30 17:58:40 -05:00
Jayadev Sirimamilla
9f07803abf GH-3152: Fix for nested Scatter Gather
Fixes https://github.com/spring-projects/spring-integration/issues/3152

The upstream `gatherResultChannel` header has been missed when we produced a reply from nested scatter-gather

Added Test Case for Nested Scatter Gather test

Simplified the the test cases and added author in changed cases

Corrected codestyle issue in Travis CI

Removed additional OriginalReplyChannel and originalErrorChannel in Headers.
Added additional not to be executed line of code in test case.

Restored OriginalErrorChannel Header and removed error handling related fixes

* Clean up code style and improve readability

**Cherry-pick to 5.1.x & master**

* Migrate `RouterTests` to JUnit 5
2020-01-27 15:12:15 -05:00
Artem Bilan
290c803f85 Fix new Sonar smell 2020-01-27 11:25:41 -05:00
Artem Bilan
e70fd0d227 Fix new Sonar smells 2020-01-26 10:38:27 -05:00
Artem Bilan
a9cb80afae Fix some Sonar smells 2020-01-25 19:23:04 -05:00
Artem Bilan
bfe28dba9a Revert NPE check in the AbstMappingMessageRouter 2020-01-25 09:19:59 -05:00
Artem Bilan
990294b048 Fix AbstractMappingMessageRouter for NPE 2020-01-24 19:17:08 -05:00
Artem Bilan
d2df9dde9f Fix Checkstyle violations 2020-01-24 16:53:11 -05:00
Artem Bilan
ad5496fa4b Fix some Sonar smells 2020-01-24 16:46:32 -05:00
Artem Bilan
c7c7422fa3 Fix unused imports violation 2020-01-22 15:57:20 -05:00
Artem Bilan
8bd9464b74 Use Overflow.BUFFER for PollableChPublisherAdapter
The `ReactiveStreamsConsumerTests.testReactiveStreamsConsumerPollableChannel()`
fails periodically.

Change the logic to rely on the `FluxSink.OverflowStrategy.BUFFER` for
possible future subscribers, meanwhile with `IGNORE` we just lose a message
altogether
2020-01-22 15:47:00 -05:00
Artem Bilan
9b7b0d91ac FluxMessageChannel: try.catch not onErrorContinue
It turns out that some upstream fluxes may come with the
`onErrorResume()` logic.
The `onErrorContinue()` here downstream in the `FluxMessageChannel.java`
eliminates an `onErrorResume()` purpose.

* Change the logic to `try..catch` in the `doOnNext()` instead and let
that upstream `onErrorResume()` to do its job
2020-01-22 13:15:56 -05:00
Artem Bilan
4b051684ef Stop endpoints in the end of tests
https://build.spring.io/browse/INT-MASTERSPRING40-954/

The `ReactiveStreamsConsumerTests` fails with some race condition from time to time.
Possible reason that no one endpoint is stopped in the end of test.
Even if we step out from the test method, background threads are still active
2020-01-17 09:55:11 -05:00
sergebg
dba307adf6 GH-3143: Fix simple pool resizing
Resolves https://github.com/spring-projects/spring-amqp/issues/1142

Resize down does not work when not enough already allocated to match the reduction.

Improved SimplePool test cases and fixed code style

Fixed code style

GH-3143: Test Polishing

- use preferred assertJ expected exception checking
- add a test for reducing a partially allocated pool
- check active counts
2020-01-16 13:03:25 -05:00
Artem Bilan
7a3a8a4bee Restore IntegrationFlowDefinition from previous
It turns out that some downstream projects are going to
support both Spring Boot 2.2 and 2.3 versions (Spring Cloud).
This way they are going to compile against Spring Integration 5.2,
but possible have a runtime against Spring Integration 5.3.

* Restore `IntegrationFlowDefinition` from the previous version
to keep binary compatibility at runtime
2020-01-16 12:49:58 -05:00
Artem Bilan
5cd36c7af9 Fix new Sonar smells 2020-01-16 09:05:23 -05:00
Artem Bilan
442c4fa4d5 GH-3111: Document @Lazy for messaging annotations (#3142)
* GH-3111: Document @Lazy for messaging annotations

Fixes https://github.com/spring-projects/spring-integration/issues/3111
Fixes https://github.com/spring-projects/spring-integration/issues/3130

* Fix some typos in docs
* Ensure in tests that announced `@Lazy` works as expected

* Doc Polishing

Co-authored-by: Gary Russell <grussell@pivotal.io>
2020-01-15 10:16:25 -05:00
Artem Bilan
f93a740065 Messaging Annotations: process ReactiveMH (#3141)
* Messaging Annotations: process ReactiveMH

* Add support for `ReactiveMessageHandler` as a `@ServiceActivator` bean
* Wrap `ReactiveMessageHandler` in the `ReactiveMessageHandlerAdapter`
in the `ServiceActivatorAnnotationPostProcessor`
* Unwrap `ReactiveMessageHandlerAdapter` in the `AbstractMethodAnnotationPostProcessor`
for `ReactiveStreamsConsumer`

* * Add documentation for the `ReactiveMessageHandler`
& Reactive MongoDb channel adapters
2020-01-14 16:17:40 -05:00
Artem Bilan
d13752b40b Infrastructure for ReactiveMessageHandler (#3137)
* Infrastructure for ReactiveMessageHandler

We have now a `ReactiveMongoDbStoringMessageHandler` which implements
a `ReactiveMessageHandler`, but not a `MessageHandler` for possible
deferred subscriptions to the returned Reactor type

We don't have a proper application context processing for this
new type of message handlers

* Change a  `ConsumerEndpointFactoryBean` to apply an `MH` and `RMH`
as possible types for handler
* Introduce a `ReactiveMessageHandlerAdapter` to wrap an `RMH`
into a `MH` for synchronous calls in the regular consumer endpoints
* Wrap an `RMH` into a `ReactiveMessageHandlerAdapter` for regular
endpoints and unwrap for `ReactiveStreamsConsumer`
* Add `RMH`-based ctor into `ReactiveStreamsConsumer` for target
reactive streams composition (`flatMap()` on the `RMH`)
* Remove a `DelegatingSubscriber` from the `ReactiveStreamsConsumer`
in favor of direct calls from the `doOnSubscribe()`, `doOnComplete()`
& `doOnNext()`
* Add an `onErrorContinue()` to handle per-message errors, but don't
cancel the whole source `Publisher`
* Use `Disposable` from the `subscribe()` to cancel in the `stop()`
- recommended way in Reactor
* Use `onErrorContinue()` in the `FluxMessageChannel` instead of
`try..catch` in the `doOnNext()` - for possible `onErrorStop()`
in the provided upstream `Publisher`
* Handle `RMH` in the `ServiceActivatorFactoryBean` as a direct handler
as well with wrapping into `ReactiveMessageHandlerAdapter` for return.
The `ConsumerEndpointFactoryBean` extracts an `RMH` from the adapter
for the `ReactiveStreamsConsumer` anyway
* Add XML parsing test for `ReactiveMongoDbStoringMessageHandler`
* Add `log4j-slf4j-impl` for all the test runtime since `slf4j-api`
comes as a transitive dependency from many places

* * Fix conflicts after rebasing to master

* * Fix typo in warn message
* Change `Assert.state()` to `Assert.isTrue()`
for `ConsumerEndpointFactoryBean.setHandler()`

* * Fix `ConsumerEndpointFactoryBean` when reactive and no advice-chain
* Fix race condition in the
`ReactiveMongoDbStoringMessageHandlerTests.testReactiveMongoMessageHandlerFromApplicationContext()`

* * Handle `ReactiveMessageHandler` in Java DSL.
Essentially request a wrapping into `ReactiveMessageHandlerAdapter`.
Describe such a requirements in the `ReactiveMessageHandlerAdapter` JavaDocs
* Some Java DSL test polishing
* Add Java DSL for `ReactiveMongoDbStoringMessageHandler`
* Propagate missed `ApplicationContext` population into an internally
created `ReactiveMongoTemplate` in the `ReactiveMongoDbStoringMessageHandler`
2020-01-13 08:41:41 -05:00
Artem Bilan
e0509cc339 Fix new Sonar smell 2020-01-10 10:16:50 -05:00
Artem Bilan
d74f9e8b86 Fix new Sonar smell 2020-01-09 16:46:53 -05:00