Commit Graph

2483 Commits

Author SHA1 Message Date
Artem Bilan
bf0260a616 GH-3276: reactive inbound: Fix onErrorResume
Fixes https://github.com/spring-projects/spring-integration/issues/3276

The `onErrorResume` for the `MessagingGatewaySupport.doSendAndReceiveMessageReactive()`
was in wrong place: only for the `buildReplyMono` which works only
when an outbound flow is fully based on reactive channels.
With a regular direct channel we can get an exception from the
`sendMessageForReactiveFlow` which is not covered with the mentioned
`onErrorResume` for the error handling on the configured `errorChannel`

Cherry-pick to `5.2.x & 5.1.x`
2020-05-13 15:24:18 -04:00
Artem Bilan
79056581f0 Fix RESOLVABLE_TYPE header population
* We should not build a RESOLVABLE_TYPE header when we map requests.
Only for replies. See: https://github.com/spring-projects/spring-integration-samples/issues/277
* We should log `ClassNotFoundException` only at debug level - to noise with info or warn

**Cherry-pick to 5.2.x**
2020-05-13 13:06:46 -04:00
Artem Bilan
c264e12deb Fix new Sonar smells 2020-04-29 09:27:54 -04:00
Artem Bilan
dc15910f2c GH-3266: Don't override props in AnnGWProxyFB
Fixes https://github.com/spring-projects/spring-integration/issues/3266

It turns out that `AnnotationGatewayProxyFactoryBean.onInit()` implementation
parses a `@MessagingGateway` attributes ignoring possible properties
population by setters.
This way a Java DSL `GatewayProxySpec` becomes useless since all
its options are overridden by default values from a synthesized
`@MessagingGateway`.
Also it is inconsistency when we declare an `AnnotationGatewayProxyFactoryBean`
as regular bean, but then called setters are ignored

* Add `protected` getters into `GatewayProxyFactoryBean` for all
the properties which can be overridden by annotation attributes
* Fix `AnnotationGatewayProxyFactoryBean` to consult with those getters
before populating a property with value from the annotation

**Cherry-pick to 5.2.x**
2020-04-28 17:27:17 -04:00
Artem Bilan
2d9a5f60f4 Introduce a ReceiveMessageAdvice (#3265)
* Introduce a `ReceiveMessageAdvice`

* Deprecate an `AbstractMessageSourceAdvice` in favor of
`default` method in the `MessageSourceMutator`
* Move a `applyReceiveOnlyAdviceChain()` logic into the `AbstractPollingEndpoint`:
now both `PollingConsumer` and `SourcePollingChannelAdapter` can use
`ReceiveMessageAdvice`
* Introduce a `SimpleActiveIdleReceiveMessageAdvice` based already
on the `ReceiveMessageAdvice` and deprecate a `SimpleActiveIdleMessageSourceAdvice`
which is fully replaceable with newly introduced `SimpleActiveIdleReceiveMessageAdvice`
* Add `@SuppressWarnings("deprecation")` for those out-of-the-box `ReceiveMessageAdvice`
implementation which still use an `AbstractMessageSourceAdvice` for
backward compatibility
* Document a new feature and give the `MessageSourceMutator` a new meaning

* * Fix language in the `polling-consumer.adoc`
2020-04-28 13:03:30 -04:00
Artem Bilan
cfd03f89a0 Use a SimpleJsonSerializer in the FileSplitter (#3262)
* Use a SimpleJsonSerializer in the FileSplitter

* To avoid extra dependency for Jackson when we
serialize `FileSplitter.FileMarker` to JSON, use
a `SimpleJsonSerializer` instead.
* Fix `SimpleJsonSerializer` to escape a `\` symbol from property values
since it is used for quoting string values in the final JSON

* * Document the change
2020-04-27 15:42:45 -04:00
Artem Bilan
6b57e16440 GH-3253: Scan BF hierarchy for BeanDefinition
Fixes https://github.com/spring-projects/spring-integration/issues/3253

The `IntegrationFlowBeanPostProcessor` uses a `containsBean()`
and then `getBeanDefinition()` to be sure that we don't override already
existing bean even if it is created in the parent(s) context.
The problem that `containsBean()` check the hierarchy, but `getBeanDefinition()`
doesn't.
So, we fail with `NoSuchBeanDefinitionException` if bean exists in the parent ctx

* Introduce an utility `IntegrationContextUtils.getBeanDefinition()` to
scan `BeanFactory` recursively for `BeanDefinition` for the requested `name`
* Use this tool in the `IntegrationFlowBeanPostProcessor` logic

**Cherry-pick to 5.2.x & 5.1.x**
2020-04-24 14:01:03 -04:00
Artem Bilan
02407f7dff Add ReactiveMessageSourceProducer (#3254)
* Add `ReactiveMessageSourceProducer`

The `ReactiveMessageSourceProducer` wraps a provided `MessageSource`
into a `Flux` for subscription in the `subscribeToPublisher(Publisher<? extends Message<?>>)`
to make a source polling feature fully based on a reactive, on demand solution

* Introduce a `IntegrationReactiveUtils` replacing existing `MessageChannelReactiveUtils`
with more functionality
* Replace a deprecated `MessageChannelReactiveUtils` with a new `IntegrationReactiveUtils`
* Test and document the feature

* * Fix Docs typos

* * Remove unused imports from `MessageChannelReactiveUtils`

* * Fix JavaDoc copy/paste artifact
2020-04-23 15:28:16 -04:00
Gary Russell
9414765b28 Fix Sonar issue 2020-04-23 12:14:02 -04:00
Gary Russell
c2babe4ba2 Add remaining deprecations for legacy metrics
- add a module to suppress legacy metrics from the Integration Graph
- move graph tests package to match src/main

Fix missed deprecation and checkstyle.

More deprecations and remove GraphLegacyStatsNullModule.

* Revert GRAPH_VERSION.
* Fix JavaDocs warnings
* Fix `MBeanExporterParserTests`
2020-04-22 17:29:36 -04:00
Artem Bilan
8505da163d Increase a block timeout in Gateway Mono test
* It turns out that 1 seconds is not enough on CI machine under the stress
2020-04-18 11:13:53 -04:00
Artem Bilan
97702ae712 Fix some Sonar smells 2020-04-08 13:01:49 -04:00
Artem Bilan
d8c378bd28 GH-2788: Add MongoDbChangeStreamMessageProducer
Fixes https://github.com/spring-projects/spring-integration/issues/2788

* Introduce a `MessageProducerSupport.subscribeToPublisher(Publisher<Message<?>>)`
for components which produces `Flux` for data from their source
* Such a component is auto-stopped when subscription to that `Publisher` is canceled
* Implement a `MongoDbChangeStreamMessageProducer` based on the reactive support for
in Spring Data MongoDb
* Implement a Java DSL for `MongoDbChangeStreamMessageProducer`
* Disable a test for change stream since it requires server of version 4.x started with 'replSet' option
* Add `MongoHeaders` for change stream events

* Change `MessageProducerSupport` to use a `takeWhile((message) -> isRunning())`
instead of storing a `subscription` from a callback
* Document new features

* Remove trailing whitespaces

* Doc Polishing.
2020-04-07 16:51:55 -04:00
Artem Bilan
d24c2c8431 Fix Checkstyle violations & RMI tests
* Increase latch timeouts for `RedisLockRegistryLeaderInitiatorTests`
* Some RMI tests fail with different outcome because an RMI registry might be
available on default `1099` port.
Fix them to not rely on that port
2020-04-07 13:21:46 -04:00
Artem Bilan
3a846bad29 Fix some Sonar smells 2020-04-07 11:26:13 -04:00
Gary Russell
cc0c2fd2fe GH-3241: Fix Sonar issue 2020-04-07 10:32:01 -04:00
Gary Russell
7d7c273515 GH-3241: MetadataStoreSelector - compare old/new (#3242)
Resolves: https://github.com/spring-projects/spring-integration/issues/3241

 * Docs and XML namespace support.
2020-04-06 18:12:36 -04:00
Gary Russell
fcca7a7b26 GH-3230: MMIH: Fix Pausable/Lifecycle Methods
Resolves https://github.com/spring-projects/spring-integration/issues/3230

Previously, methods with the same name as `Lifecycle` methods were unconditionally ignored.

- Use logic similar to `ControlBusMethodFilter` to explicitly compare the candidate Method
  to those on `Paused` and `Lifecycle` and only ignore those.
- Explicitly disallow these methods when named - previously the check was only performed
  if no method name was supplied.

* - reinstate `Pausable/Lifecycle` methods if explicitly requested.
- log `Pausable` methods that are not candidates.

* - Fix typo in test method.

**Cherry-pick to 5.2.x**
2020-04-01 12:17:26 -04:00
Artem Bilan
76e79012fe GH-3219: Add HandleMessageAdviceAdapter (#3234)
* GH-3219: Add `HandleMessageAdviceAdapter`

Fixes https://github.com/spring-projects/spring-integration/issues/3219

* * Fix language in doc

Co-Authored-By: Gary Russell <grussell@pivotal.io>

Co-authored-by: Gary Russell <grussell@pivotal.io>
2020-04-01 11:10:33 -04:00
Artem Bilan
7433e41036 Add more Kotlin DSL for sub-flows (#3233)
* Add more Kotlin DSL for sub-flows

* Introduce `KotlinSplitterEndpointSpec`, `KotlinEnricherSpec` &
`KotlinFilterEndpointSpec` to avoid an explicit use of `IntegrationFlow` usage
from Kotlin DSL
* Fix `AbstractKotlinRouterSpec` to extend `ConsumerEndpointSpec` for access
to more endpoint options as it is with Java DSL for similar specs
* Introduce a `KotlinIntegrationFlowDefinition.publishSubscribe()`
for a `BroadcastCapableChannel` and `vararg` of `KotlinIntegrationFlowDefinition` builders.
This allows us to not introduce a `BroadcastPublishSubscribeSpec` wrapper for Kotlin
and let us to remove `publishSubscribeChannel()` DSL methods for `PublishSubscribeSpec`
since it is covered with the `publishSubscribe(PublishSubscribeChannel())` and
respective set of sub-flows in Kotlin DSL as subscribers
* Use new Kotlin specs in the `KotlinIntegrationFlowDefinition` instead of their
Java variants

* * Mention `kotlin-dsl.adoc` in the ToC
2020-04-01 11:06:54 -04:00
Artem Bilan
9c46b3420e Fix JsonToObjectTransformer for ClassNotFoundEx (#3228)
* Fix JsonToObjectTransformer for ClassNotFoundEx

Related to https://github.com/spring-projects/spring-integration/issues/3223

The `JsonToObjectTransformer` consults `JsonHeaders` first and tries to build
a `ResolvableType` from other type headers which may be just a string identifications.
In this case a `ClassNotFoundException` could be thrown if a `ResolvableType` cannot
be build against non-class identificators

* Add `valueTypeExpression` option to the `JsonToObjectTransformer` to let to
build a `ResolvableType` using any possible custom logic, e.g. resolving
target classes from some registry using their ids from the mentioned headers

**Cherry-pick to 5.2.x**

* * Fix English language mistakes
2020-03-27 15:42:24 -04:00
Artem Bilan
bff891b7a9 Fix new Sonar smells 2020-03-26 10:31:32 -04:00
Artem Bilan
b6bd83d548 Fix deprecation for TX; fix JMS tests; Moore-SR6
* Upgrade to Spring Data Moore-SR6
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
for deprecations in the `TransactionInterceptor`
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
consumers to expose new `TransactionManager`-based options and
deprecate `PlatformTransactionManager`-based
* Fix failing JMS tests to reuse an ActiveMQ Connection Factory with a
`trustedPackaged(*)`

**Cherry-pick to master**

# Conflicts:
#	build.gradle
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/ConsumerEndpointSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/PollerSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionHandleMessageAdvice.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionInterceptorBuilder.java
2020-03-25 13:56:37 -04:00
Gary Russell
fbf06e8bb5 GH-3181: MQTT: Support MANUAL Acks
Resolves https://github.com/spring-projects/spring-integration/issues/3181

* Doc polishing

* Rework acknowledgment into the existing `AcknowledgmentCallback`.

* Fix javadocs and doc linFix javadocs and doc linkk

* Doc polishing; explain uses of `ACKNOWLEDGMENT_CALLBACK` header.
2020-03-19 15:20:13 -04:00
Artem Bilan
3eedda6c9d GH- 3223: Build ResolvableType after mapping
Fixes https://github.com/spring-projects/spring-integration/issues/3223

The `json__TypeId__` header may have a value which cannot be resolved to the `Class` in the current classpath
So, skip `ResolvableType` building logic until we really sure that end-user wants to map JSON headers
* WARN a build exception that we can't load a class for the `json__TypeId__` when we try to build a `ResolvableType`
in the  `DefaultAmqpHeaderMapper`
* Document the negation feature for JSON headers

**Cherry-pick to 5.2.x**
2020-03-19 14:51:32 -04:00
Artem Bilan
92421f7575 Add Kotlin wrappers for router specs (#3218)
* Add Kotlin wrappers for router specs

To avoid casting and extra logic logic in the end-user code,
it is better to provide Kotlin-specific API to let end-users to
do whatever is really dictated by API and don't think about specific types to cast

* * Fix typos; code clean up
2020-03-17 12:56:45 -04:00
Artem Bilan
e9a3580ef5 Upgrade dependencies; fix new Sonar smells 2020-03-11 10:22:51 -04:00
Artem Bilan
36f96ec2c6 Document Kotlin DSL (#3210)
* Document Kotlin DSL

* Remove unused imports in `FunctionsTests.kt`

* * Fix typos in Docs
2020-03-10 16:47:41 -04:00
Artem Bilan
1511dd8748 GH-3204: Add DSL intercept() operator
Fixes https://github.com/spring-projects/spring-integration/issues/3204

* Add an `intercept(ChannelInterceptor...)` method into `BaseIntegrationFlowDefinition` 
to register one or more channel interceptors at the current flow position.
* refactor to reuse `InterceptableChannel` creation from `wireTap`
* document the new operator
2020-03-05 16:15:52 -05:00
Gary Russell
c84d264294 GH-3183: Add ReactiveRequestHandlerAdvice (#3197)
* GH-3183: Add ReactiveRequestHandlerAdvice

Fixes https://github.com/spring-projects/spring-integration/issues/3183

* Introduce a `ReactiveRequestHandlerAdvice` with a `BiFunction<Message<?>, Mono<?>, Publisher<?>>`
logic to apply a `Mono.transform()` operator for a returned from the handler `Mono` reply
* Fix `WebFluxRequestExecutingMessageHandler` to return a `Mono.then()` instead of an explicit subscription -
it happens downstream anyway during reply producing with a proper error handling, too
* Demonstrate `ReactiveRequestHandlerAdvice` in the `RSocketDslTests` - without `retry()` it fails
* Add `ConsumerEndpointSpec.customizeMonoReply()` for convenience
* Document `ReactiveRequestHandlerAdvice` feature

* * Fix language in docs
2020-03-05 13:37:54 -05:00
Artem Bilan
8a38a5e829 Merge spring-integration-kotlin-dsl project
* Migrate all the Kotlin tests to use new Kotlin DSL
* Upgrade to the latest Kotlin
* Generate KDocs
2020-03-05 13:10:32 -05:00
Artem Bilan
45a91b25de Add Reactive TX Manager support (#3201)
* Add Reactive TX Manager support

* Change `TransactionInterceptorBuilder` and `TransactionHandleMessageAdvice` to reply
on a generic `TransactionManager`, so we can configure reactive one as well
* Change a `<transactional>` XML element to support a generic `TransactionManager` reference,
so we can configure reactive one as well
* Support `adviceChain` configuration for the `ReactiveMessageHandler` in the `ConsumerEndpointFactoryBean`
* Attemp to apply reactive transaction for the Reactive MongoDB channel adapter

* * Revert `MongoDbTests` change -
to support transactions we need the latest MongoDb server with replica enabled
* Document reactive transactions

* * Address PR reviews
2020-03-03 11:54:29 -05:00
Artem Bilan
e6e9f45164 Use Mono.create instead of Mono.fromCallable
It is better to poll message from `QueueChannel`
when ever a reactive request happens.
The `Mono.fromCallable()` operator does poll on subscription, not request
and caches the value.
However we may lose such a value in between.

* Use `Mono.create()` with its `monoSink.onRequest()` callback in the
`adaptPollableChannelToPublisher()` implementation to defer `inputChannel.receive()`
until an on demand request downstream
2020-03-02 16:57:29 -05:00
Tim Feuerbach
c6cf261e25 GH-3202: Fix tests failing with non-English locales
Fixes https://github.com/spring-projects/spring-integration/issues/3202

Set the locale to en-US for tests that assert error messages
2020-03-02 14:03:27 -05:00
Artem Bilan
0e5258a661 Add log() to test Publisher for diagnostics
https://build.spring.io/browse/INT-MASTER-1916/
2020-02-27 18:12:31 -05:00
Artem Bilan
87c8e47a88 GH-3192: pub-sub DSL for broker-backed channels (#3193)
* GH-3192: pub-sub DSL for broker-backed channels

Fixes https://github.com/spring-projects/spring-integration/issues/3192

* Introduce a `BroadcastCapableChannel` abstract to indicate those `SubscribableChannel`
implementations which can provide a pub-sub functionality
* Implement a `BroadcastCapableChannel` in broker-baked channels with pub-sub option
* Introduce a `BaseIntegrationFlowDefinition.publishSubscribeChannel()` based
on the `BroadcastCapableChannel` and `BroadcastPublishSubscribeSpec` to let to
configure sub-flow subscribers in fluent manner

* * Add some JavaDocs and document new feature

* * Show the channel bean definition in the doc
* Fix typo
2020-02-25 13:01:37 -05:00
Artem Bilan
d61a247f4b Fix ReactiveStreamsConsumer for PollableChannel
https://build.spring.io/browse/INT-MASTERSPRING40-985/

* Move `onSubscribe()` callback to the `subscribe()` operator
to honor a `request` contract from the target subscriber
2020-02-18 12:37:24 -05:00
Artem Bilan
6d936d9774 Fix some Sonar smells 2020-02-14 17:57:58 -05:00
Artem Bilan
8c4675aa83 Fix new Sonar smell
* Make `adaptPollableChannelToPublisher()` as non-blocking
2020-02-13 14:11:21 -05:00
Artem Bilan
a0cdfd90ee Change adaptPollableChannelToPublisher to Mono
We can use a `Mono.fromCallable()` in combination with `repeat()`
instead of possible race condition deal in the `Flux.create()`
2020-02-12 12:09:31 -05:00
Artem Bilan
ede9528c19 Fix some tests for not closed application context 2020-02-11 12:43:33 -05:00
Artem Bilan
a8c471c061 Fix ReactiveStreamsConsumer for error handling
To support `onErrorContinue()` logic for the plain `Subscriber`
we need to wrap its `onNext()` into a `try..catch` and respective
`errorHandler` in the `ReactiveStreamsConsumer`
2020-02-10 16:27:59 -05:00
Artem Bilan
6591ce90f6 Fix ReactiveStreamsConsumer for plain subscriber
https://build.spring.io/browse/INT-MASTERSPRING40-978

Investigate a behaviour for `ReactiveStreamsConsumer`
when we use a plain `Subscriber` directly instead of `doOn...` callbacks.
It looks like there is some race condition when the data can be consumed
upstream, but there is no consumer downstream ready yet
2020-02-10 16:00:14 -05:00
Artem Bilan
867a8cf108 GH-3155: Add support for Java DSL extensions (#3167)
* GH-3155: Add support for Java DSL extensions

Fixes https://github.com/spring-projects/spring-integration/issues/3155

Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.

* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature

* * Fix language and typos in docs

* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods

* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
2020-02-07 13:40:39 -05:00
Jayadev Sirimamilla
1d9a818efe GH-3157: pop sequence for message as well
Fixes https://github.com/spring-projects/spring-integration/issues/3157

* Fixed issue with sequences not being popped when Output processor returns a message.

* Updated documents and added additional Checks to the testcase to ensure proper `popSequence` is done.

* Updated documents and added corrected code formatting

* Updated documents and used headerAccessor Constants for matching headers
* Clean up code style
* Improve docs for the feature on the matter
2020-02-04 11:37:19 -05:00
Artem Bilan
e5740f253c Upgrade dependencies
* Gradle 6.1.1
* MongoDb 4.0
* Other Spring dependencies to `BUILD-SNAPSHOT`
* Fix MongoDb tests according latest Spring Data
* State in the docs that both MongoDb driver are `optional` now in the dependencies
2020-02-03 13:12:50 -05:00
Artem Bilan
d792915be6 Add debug log for ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTER-1883/
2020-01-31 15:16:57 -05:00
Artem Bilan
2f4394d8b5 Populate a JsonHeaders.RESOLVABLE_TYPE on reply (#3163)
* Populate a JsonHeaders.RESOLVABLE_TYPE on reply

Fixes https://github.com/spring-projects/spring-integration-samples/issues/277

In the `AbstractAmqpOutboundEndpoint` the `JsonHeaders.RESOLVABLE_TYPE`
from request message is copied to reply message making inconsistency downstream.
The `JsonToObjectTransformer` consults first a `JsonHeaders.RESOLVABLE_TYPE`
and deserialize payload to wrong type

* Fix `DefaultAmqpHeaderMapper` to populate a `JsonHeaders.RESOLVABLE_TYPE`
alongside with other `JsonHeaders` populated from the reply AMQP message.
This way a `JsonHeaders.RESOLVABLE_TYPE` from request message won't have effect
* To get access to classes, supply `AbstractHeaderMapper` with a bean factory `ClassLoader`
* Introduce a couple utility methods into `JsonHeaders` for building a `ResolvableType`

**Cherry-pick to 5.2.x**

* * Fix code formatting for arguments wrapping
2020-01-31 14:17:19 -05:00
Artem Bilan
34bc8116df Add @LogLevels to ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTERSPRING40-967/
2020-01-30 17:58:40 -05:00
Jayadev Sirimamilla
9f07803abf GH-3152: Fix for nested Scatter Gather
Fixes https://github.com/spring-projects/spring-integration/issues/3152

The upstream `gatherResultChannel` header has been missed when we produced a reply from nested scatter-gather

Added Test Case for Nested Scatter Gather test

Simplified the the test cases and added author in changed cases

Corrected codestyle issue in Travis CI

Removed additional OriginalReplyChannel and originalErrorChannel in Headers.
Added additional not to be executed line of code in test case.

Restored OriginalErrorChannel Header and removed error handling related fixes

* Clean up code style and improve readability

**Cherry-pick to 5.1.x & master**

* Migrate `RouterTests` to JUnit 5
2020-01-27 15:12:15 -05:00