Commit Graph

1561 Commits

Author SHA1 Message Date
Artem Bilan
481d5eb2a5 Fix new Sonar smells
* Use `tryEmitNext` on Reactor `Sink` since `emitNext` is deprecated
* Add `MessageDeliveryException` emission when `send()` returns `false`
in the `FluxMessageChannel` for `subscribeTo` provided `Publisher`
2020-09-11 12:37:27 -04:00
Artem Bilan
383af8ceb9 GH-3374: Fix scan for BF propagation (#3378)
* GH-3374: Fix scan for BF propagation

Fixes https://github.com/spring-projects/spring-integration/issues/3374

An internal `ClassPathScanningCandidateComponentProvider` instance in the `IntegrationComponentScanRegistrar`
does not propagate a provided `registry`.

* Implement `getRegistry()` on the internal `ClassPathScanningCandidateComponentProvider` to propagate
a provided into the `registerBeanDefinitions()` a `BeanDefinitionRegistry`
* Add `@Conditional` on some scanned `@MessagingGateway` in the `EnableIntegrationTests`

**Cherry-pick to 5.3.x & 5.2.x**

* * Remove unused import

* Restore `unused` warning on the unused registry arg
2020-09-11 09:48:36 -04:00
Artem Bilan
af5bcdf4d7 Honor Emission result in FluxMessageChannel
* Implement `Emission` handling in the `FluxMessageChannel` and `IntegrationReactiveUtils`
* Upgrade to Spring Kafka `2.6.0`
* Fix R2DBC components for deprecation in the Spring Data R2DBC
* Implement `StatementMapper.SelectSpec` for query expression
* Clean up for some sporadic test failures
2020-09-09 12:16:00 -04:00
Artem Bilan
52070259b3 Fix compatibility with the latest SF 2020-08-26 12:43:47 -04:00
Michel Jung
6780bbd5c8 GH-3367: Separate timeouts in BarrierMH
Fixes https://github.com/spring-projects/spring-integration/issues/3367

Introduce a `requestTimeout` and `triggerTimeout` for `BarrierMessageHandler`
For instance, if an HTTP request sends a message to the barrier,
it should time out after 1min if no trigger message is received.
If the trigger message then arrives late and the HTTP request is no longer waiting,
it shouldn't wait for 1min before discarding the request but do so immediately.
2020-08-21 12:50:45 -04:00
Artem Bilan
17fc4eae99 Fix IntReactiveUtils for the proper emission
* Enable `MessageChannelReactiveUtilsTests.testOverproducingWithSubscribableChannel()` back
* Disable `StompServerIntegrationTests` again since build on CI is stalled again
2020-08-10 12:32:47 -04:00
Gary Russell
0bae11c60e GH-3359: Add Timer to MessagingGatewaySupport
Resolves https://github.com/spring-projects/spring-integration/issues/3359

- replace removed simple counter with a `Timer`
- register DSL GPFB as beans so they can be discovered for wiring metrics

* Apply suggestions from code review
* Remove left over deprecation suppression
* Fix import
2020-08-10 10:46:41 -04:00
Artem Bilan
9d557426b5 * Fix new Sonar smells
* Remove redundant `@SuppressWarnings("deprecation")`
* Add `Duration.ofSeconds(10)` to `StepVerifier.verify()`
to avoid infinite wait and lose failing text context on the hang CI build
2020-08-08 12:29:24 -04:00
Gary Russell
1beb854fb4 Remove Legacy Metrics
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
  implement `ManageableLifecycle`. Register an additional MBean for
  polled endpoints to control the lifecycle.

* Polishing

- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
  to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
2020-08-07 12:56:57 -04:00
Artem Bilan
cd63763bb4 Fix compatibility with Reactor & Spring Data 2020-08-05 15:33:24 -04:00
Gary Russell
0e2652fdf6 More Cron test fixes 2020-08-03 16:51:55 -04:00
Gary Russell
2356324e9b Fix for internal changes to CronTrigger 2020-08-03 16:30:27 -04:00
Artem Bilan
f23f3ea63d Add gauges for queue channel size (#3349)
* Add gauges for queue channel size

The `QueueChannel` provides a current size and remaining capacity metrics

* Add Micrometer gauges into `QueueChannel` to expose the current values
of the size and remaining capacity

**Cherry-pick to 5.3.x, 5.2.x & 5.1.x**

* * Revert `@SuppressWarnings("unchecked")` for test
* Document new gauges for queue channel

* * Fix IntegrationManagementConfigurer for NPE on `metricsCaptor`

* Fix wording in meter descriptions

Co-authored-by: Michel Jung <michel.jung89@gmail.com>

Co-authored-by: Michel Jung <michel.jung89@gmail.com>
2020-07-28 16:59:35 -04:00
Artem Bilan
5e45b09cd5 Fix typo in KryoCodecTests 2020-07-22 12:33:24 -04:00
Artem Bilan
6a02b5abe9 Improve Kryo Codec for registrations 2020-07-22 12:07:05 -04:00
Artem Bilan
492f0bfee3 GH-3334: Add "embedded reaper" into CorrelationMH (#3342)
* GH-3334: Add "embedded reaper" into CorrelationMH

Fixes https://github.com/spring-projects/spring-integration/issues/3334

* Add `expireTimeout` property into `AbstractCorrelatingMessageHandler`
to call newly introduced `purgeOrphanedGroups()` API for removing old
groups from the store
* Add `expireDuration` to perform `purgeOrphanedGroups()` task periodically

* * Add Java DSL and XML support for `expireTimeout` and `expireDuration` options
* Document the new feature

* * Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-07-20 16:31:24 -04:00
Artem Bilan
6a865b55b9 GH-3344: Treat kotlin.Unit return as null in MMIH (#3346)
* GH-3344: Treat kotlin.Unit return as null in MMIH

Fixes https://github.com/spring-projects/spring-integration/issues/3344

When function lambda doesn't return anything (e.g. a `void` method call is the last one),
Kotlin produces a `kotlin.Unit` instance as a return value which is not null and produced
as a reply message payload.

* Fix `MessagingMethodInvokerHelper` to treat a `kotlin.Unit` as `null` for reply
making Kotlin lambdas working the same way as Java lambdas when we don't return anything
from from there

**Cherry-pick to `5.3.x`**

* * Introduce `ClassUtils.isKotlinUnit(Class)` API;
use it in the `MessagingMethodInvokerHelper` instead of
`.getName().equals()`

* * Fix since on new `isKotlinUnit()` API
2020-07-20 13:51:54 -04:00
Artem Bilan
7a09358582 SO-62761903: Inject BF into Gateway's correlator
Related to https://stackoverflow.com/questions/62761903/spring-integration-reactive-streams-support-exception-in-creating-a-reactive

The `MessagingGatewaySupport` creates an internal endpoint
for consuming messages from the provided `replyChannel`.
The endpoint type depends on the channel type.
The `ReactiveStreamsConsumer` was missing a `BeanFactory` injection
causing an error when `ReactiveStreamsConsumer` tries to extract a default `ErrorHandler`
from `BeanFactory`

* Refactor `MessagingGatewaySupport` to inject a `BeanFactory` to all
the possible correlator endpoints.
Also always call `afterPropertiesSet()` for all of them

**Cherry-pick to 5.3.x & 5.2.x**
2020-07-08 14:58:38 -04:00
Gary Russell
b45d690a45 Fix CachedSessionFactory Race
Close the pool so that any sessions returned after the factory is
`destroy()`ed are closed.

* Call `removeAllIdleItems()` in `close()`.

* Close sessions in `SftpStreamingMessageSourceTests`.

**cherry-pick to all supported branches**
2020-07-08 14:19:57 -04:00
Artem Bilan
146b0af1f4 GH-3320: Refine lifecycle control in StdIntFlow (#3322)
* GH-3320: Refine lifecycle control in StdIntFlow

Fixes https://github.com/spring-projects/spring-integration/issues/3320

Turns out that lifecycle control for the whole bunch of components
in one `IntegrationFlow` is useful in fields.

* Change the logic in the `StandardIntegrationFlow` to let to call
`start()` and `stop()` independently how the flow was registered in
the application context.
This way it can be autowired as a `Lifecycle` to let end-user to
avoid the search for proper component in the flow to stop or start
manually - all the components registered with the flow are going
to be stopped or started respectively

* * Add `this.` to class property usage to satisfy Checkstyle

Co-authored-by: Artem Bilan <abilan@vmware.com>
2020-07-02 15:29:43 -04:00
Artem Bilan
3bb445e133 Fix deprecations from Reactor 2020-06-22 13:32:06 -04:00
Artem Bilan
498f42d480 Fix deprecations from SF
* Remove `whitelist` words
* Resolve Sonar smells
* Add `await()` for FTP file removal test: looks like this operation may fail under the stress build
2020-06-19 12:07:58 -04:00
Artem Bilan
3f5aba2cb9 Support Java 14 (#3310)
* Support Java 14

* Provide changes to avoid deprecated Java API
and have a compatibility back to Java 8
* Change affected test classes to JUnit 5 whenever it is possible
* Ignore/Disable some TCP/IP tests which don't pass on Java 14

* Fix (some) TCP tests on JRE 14

* Fix SSL Handshake test - client side handshake is successful with java 14

- change the badClient cert to a badServer cert to force an error on the client side

Co-authored-by: artembilan <raven666>
Co-authored-by: Gary Russell <grussell@pivotal.io>
2020-06-17 14:00:06 -04:00
Artem Bilan
2419c03b25 Replace whitelist with allowlist 2020-06-15 14:20:51 -04:00
Artem Bilan
b8e2dfe4e1 GH-3280: NullChannel as reply for void gateways (#3292)
* GH-3280: NullChannel as reply for void gateways

Fixes https://github.com/spring-projects/spring-integration/issues/3280

To properly support a one-way gateway (`void` return type), it is
better to ignore any possible replies from downstream instead of
unexpected `no output-channel or replyChannel header available`
error

* Populate a `nullChannel` into a `replyChannel` header
for `void` gateway methods when `replyChannel` is not set explicitly
* Remove redundant `GatewayProxyFactoryBean.PARSER` in favor of
similar `EXPRESSION_PARSER` in the super class
* Test and document the feature

* * Fix language in the `whats-new.adoc`

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Fix language in gateway.adoc

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-05-27 16:10:05 -04:00
Artem Bilan
e98169146b GH-3288: Add Kotlin DSL transform(Transformer)
Fixes https://github.com/spring-projects/spring-integration/issues/3288

* For better end-user experience with Kotlin DSL and get a gain from
existing `Transformer` implementations add a `transform(Transformer)`
EI-method into the `KotlinIntegrationFlowDefinition`
* Also add `filter(MessageSelector)` for any out-of-the-box `MessageSelector`

**Cherry-pick to 5.3.x**
2020-05-27 13:39:54 -04:00
Artem Bilan
79056581f0 Fix RESOLVABLE_TYPE header population
* We should not build a RESOLVABLE_TYPE header when we map requests.
Only for replies. See: https://github.com/spring-projects/spring-integration-samples/issues/277
* We should log `ClassNotFoundException` only at debug level - to noise with info or warn

**Cherry-pick to 5.2.x**
2020-05-13 13:06:46 -04:00
Artem Bilan
dc15910f2c GH-3266: Don't override props in AnnGWProxyFB
Fixes https://github.com/spring-projects/spring-integration/issues/3266

It turns out that `AnnotationGatewayProxyFactoryBean.onInit()` implementation
parses a `@MessagingGateway` attributes ignoring possible properties
population by setters.
This way a Java DSL `GatewayProxySpec` becomes useless since all
its options are overridden by default values from a synthesized
`@MessagingGateway`.
Also it is inconsistency when we declare an `AnnotationGatewayProxyFactoryBean`
as regular bean, but then called setters are ignored

* Add `protected` getters into `GatewayProxyFactoryBean` for all
the properties which can be overridden by annotation attributes
* Fix `AnnotationGatewayProxyFactoryBean` to consult with those getters
before populating a property with value from the annotation

**Cherry-pick to 5.2.x**
2020-04-28 17:27:17 -04:00
Artem Bilan
2d9a5f60f4 Introduce a ReceiveMessageAdvice (#3265)
* Introduce a `ReceiveMessageAdvice`

* Deprecate an `AbstractMessageSourceAdvice` in favor of
`default` method in the `MessageSourceMutator`
* Move a `applyReceiveOnlyAdviceChain()` logic into the `AbstractPollingEndpoint`:
now both `PollingConsumer` and `SourcePollingChannelAdapter` can use
`ReceiveMessageAdvice`
* Introduce a `SimpleActiveIdleReceiveMessageAdvice` based already
on the `ReceiveMessageAdvice` and deprecate a `SimpleActiveIdleMessageSourceAdvice`
which is fully replaceable with newly introduced `SimpleActiveIdleReceiveMessageAdvice`
* Add `@SuppressWarnings("deprecation")` for those out-of-the-box `ReceiveMessageAdvice`
implementation which still use an `AbstractMessageSourceAdvice` for
backward compatibility
* Document a new feature and give the `MessageSourceMutator` a new meaning

* * Fix language in the `polling-consumer.adoc`
2020-04-28 13:03:30 -04:00
Artem Bilan
02407f7dff Add ReactiveMessageSourceProducer (#3254)
* Add `ReactiveMessageSourceProducer`

The `ReactiveMessageSourceProducer` wraps a provided `MessageSource`
into a `Flux` for subscription in the `subscribeToPublisher(Publisher<? extends Message<?>>)`
to make a source polling feature fully based on a reactive, on demand solution

* Introduce a `IntegrationReactiveUtils` replacing existing `MessageChannelReactiveUtils`
with more functionality
* Replace a deprecated `MessageChannelReactiveUtils` with a new `IntegrationReactiveUtils`
* Test and document the feature

* * Fix Docs typos

* * Remove unused imports from `MessageChannelReactiveUtils`

* * Fix JavaDoc copy/paste artifact
2020-04-23 15:28:16 -04:00
Gary Russell
c2babe4ba2 Add remaining deprecations for legacy metrics
- add a module to suppress legacy metrics from the Integration Graph
- move graph tests package to match src/main

Fix missed deprecation and checkstyle.

More deprecations and remove GraphLegacyStatsNullModule.

* Revert GRAPH_VERSION.
* Fix JavaDocs warnings
* Fix `MBeanExporterParserTests`
2020-04-22 17:29:36 -04:00
Artem Bilan
8505da163d Increase a block timeout in Gateway Mono test
* It turns out that 1 seconds is not enough on CI machine under the stress
2020-04-18 11:13:53 -04:00
Artem Bilan
d8c378bd28 GH-2788: Add MongoDbChangeStreamMessageProducer
Fixes https://github.com/spring-projects/spring-integration/issues/2788

* Introduce a `MessageProducerSupport.subscribeToPublisher(Publisher<Message<?>>)`
for components which produces `Flux` for data from their source
* Such a component is auto-stopped when subscription to that `Publisher` is canceled
* Implement a `MongoDbChangeStreamMessageProducer` based on the reactive support for
in Spring Data MongoDb
* Implement a Java DSL for `MongoDbChangeStreamMessageProducer`
* Disable a test for change stream since it requires server of version 4.x started with 'replSet' option
* Add `MongoHeaders` for change stream events

* Change `MessageProducerSupport` to use a `takeWhile((message) -> isRunning())`
instead of storing a `subscription` from a callback
* Document new features

* Remove trailing whitespaces

* Doc Polishing.
2020-04-07 16:51:55 -04:00
Gary Russell
7d7c273515 GH-3241: MetadataStoreSelector - compare old/new (#3242)
Resolves: https://github.com/spring-projects/spring-integration/issues/3241

 * Docs and XML namespace support.
2020-04-06 18:12:36 -04:00
Gary Russell
fcca7a7b26 GH-3230: MMIH: Fix Pausable/Lifecycle Methods
Resolves https://github.com/spring-projects/spring-integration/issues/3230

Previously, methods with the same name as `Lifecycle` methods were unconditionally ignored.

- Use logic similar to `ControlBusMethodFilter` to explicitly compare the candidate Method
  to those on `Paused` and `Lifecycle` and only ignore those.
- Explicitly disallow these methods when named - previously the check was only performed
  if no method name was supplied.

* - reinstate `Pausable/Lifecycle` methods if explicitly requested.
- log `Pausable` methods that are not candidates.

* - Fix typo in test method.

**Cherry-pick to 5.2.x**
2020-04-01 12:17:26 -04:00
Artem Bilan
7433e41036 Add more Kotlin DSL for sub-flows (#3233)
* Add more Kotlin DSL for sub-flows

* Introduce `KotlinSplitterEndpointSpec`, `KotlinEnricherSpec` &
`KotlinFilterEndpointSpec` to avoid an explicit use of `IntegrationFlow` usage
from Kotlin DSL
* Fix `AbstractKotlinRouterSpec` to extend `ConsumerEndpointSpec` for access
to more endpoint options as it is with Java DSL for similar specs
* Introduce a `KotlinIntegrationFlowDefinition.publishSubscribe()`
for a `BroadcastCapableChannel` and `vararg` of `KotlinIntegrationFlowDefinition` builders.
This allows us to not introduce a `BroadcastPublishSubscribeSpec` wrapper for Kotlin
and let us to remove `publishSubscribeChannel()` DSL methods for `PublishSubscribeSpec`
since it is covered with the `publishSubscribe(PublishSubscribeChannel())` and
respective set of sub-flows in Kotlin DSL as subscribers
* Use new Kotlin specs in the `KotlinIntegrationFlowDefinition` instead of their
Java variants

* * Mention `kotlin-dsl.adoc` in the ToC
2020-04-01 11:06:54 -04:00
Artem Bilan
9c46b3420e Fix JsonToObjectTransformer for ClassNotFoundEx (#3228)
* Fix JsonToObjectTransformer for ClassNotFoundEx

Related to https://github.com/spring-projects/spring-integration/issues/3223

The `JsonToObjectTransformer` consults `JsonHeaders` first and tries to build
a `ResolvableType` from other type headers which may be just a string identifications.
In this case a `ClassNotFoundException` could be thrown if a `ResolvableType` cannot
be build against non-class identificators

* Add `valueTypeExpression` option to the `JsonToObjectTransformer` to let to
build a `ResolvableType` using any possible custom logic, e.g. resolving
target classes from some registry using their ids from the mentioned headers

**Cherry-pick to 5.2.x**

* * Fix English language mistakes
2020-03-27 15:42:24 -04:00
Artem Bilan
b6bd83d548 Fix deprecation for TX; fix JMS tests; Moore-SR6
* Upgrade to Spring Data Moore-SR6
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
for deprecations in the `TransactionInterceptor`
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
consumers to expose new `TransactionManager`-based options and
deprecate `PlatformTransactionManager`-based
* Fix failing JMS tests to reuse an ActiveMQ Connection Factory with a
`trustedPackaged(*)`

**Cherry-pick to master**

# Conflicts:
#	build.gradle
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/ConsumerEndpointSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/dsl/PollerSpec.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionHandleMessageAdvice.java
#	spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionInterceptorBuilder.java
2020-03-25 13:56:37 -04:00
Artem Bilan
92421f7575 Add Kotlin wrappers for router specs (#3218)
* Add Kotlin wrappers for router specs

To avoid casting and extra logic logic in the end-user code,
it is better to provide Kotlin-specific API to let end-users to
do whatever is really dictated by API and don't think about specific types to cast

* * Fix typos; code clean up
2020-03-17 12:56:45 -04:00
Artem Bilan
36f96ec2c6 Document Kotlin DSL (#3210)
* Document Kotlin DSL

* Remove unused imports in `FunctionsTests.kt`

* * Fix typos in Docs
2020-03-10 16:47:41 -04:00
Artem Bilan
1511dd8748 GH-3204: Add DSL intercept() operator
Fixes https://github.com/spring-projects/spring-integration/issues/3204

* Add an `intercept(ChannelInterceptor...)` method into `BaseIntegrationFlowDefinition` 
to register one or more channel interceptors at the current flow position.
* refactor to reuse `InterceptableChannel` creation from `wireTap`
* document the new operator
2020-03-05 16:15:52 -05:00
Artem Bilan
8a38a5e829 Merge spring-integration-kotlin-dsl project
* Migrate all the Kotlin tests to use new Kotlin DSL
* Upgrade to the latest Kotlin
* Generate KDocs
2020-03-05 13:10:32 -05:00
Tim Feuerbach
c6cf261e25 GH-3202: Fix tests failing with non-English locales
Fixes https://github.com/spring-projects/spring-integration/issues/3202

Set the locale to en-US for tests that assert error messages
2020-03-02 14:03:27 -05:00
Artem Bilan
0e5258a661 Add log() to test Publisher for diagnostics
https://build.spring.io/browse/INT-MASTER-1916/
2020-02-27 18:12:31 -05:00
Artem Bilan
ede9528c19 Fix some tests for not closed application context 2020-02-11 12:43:33 -05:00
Artem Bilan
867a8cf108 GH-3155: Add support for Java DSL extensions (#3167)
* GH-3155: Add support for Java DSL extensions

Fixes https://github.com/spring-projects/spring-integration/issues/3155

Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.

* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature

* * Fix language and typos in docs

* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods

* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
2020-02-07 13:40:39 -05:00
Jayadev Sirimamilla
1d9a818efe GH-3157: pop sequence for message as well
Fixes https://github.com/spring-projects/spring-integration/issues/3157

* Fixed issue with sequences not being popped when Output processor returns a message.

* Updated documents and added additional Checks to the testcase to ensure proper `popSequence` is done.

* Updated documents and added corrected code formatting

* Updated documents and used headerAccessor Constants for matching headers
* Clean up code style
* Improve docs for the feature on the matter
2020-02-04 11:37:19 -05:00
Artem Bilan
e5740f253c Upgrade dependencies
* Gradle 6.1.1
* MongoDb 4.0
* Other Spring dependencies to `BUILD-SNAPSHOT`
* Fix MongoDb tests according latest Spring Data
* State in the docs that both MongoDb driver are `optional` now in the dependencies
2020-02-03 13:12:50 -05:00
Artem Bilan
d792915be6 Add debug log for ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTER-1883/
2020-01-31 15:16:57 -05:00
Artem Bilan
34bc8116df Add @LogLevels to ReactiveStreamsConsumerTests
https://build.spring.io/browse/INT-MASTERSPRING40-967/
2020-01-30 17:58:40 -05:00