Commit Graph

10664 Commits

Author SHA1 Message Date
Artem Bilan
010ce8b01f Fail JmsMDEndpoint if container has a listener
Right now the `JmsMessageDrivenEndpoint` only logs a warn about
already provided listener.
Turns out this is not enough for end-users and they may miss
the warn message in the logs somehow having some problems at
runtime trying to determine find out the root of unexpected behavior

* Change the warn message to the `Assert.isNull()` to fail fast.
Same is done in the `KafkaMessageDrivenChannelAdapter` & `AmqpInboundChannelAdapter`
2021-03-10 16:22:53 -05:00
Artem Bilan
a1a4341655 Add logging to CachingClientConnFactoryTests 2021-03-10 11:52:00 -05:00
Artem Bilan
207de60c0a Add logging diagnostics to UdpChannelAdapterTests 2021-03-10 11:16:18 -05:00
Artem Bilan
61153a96fc Resolve new Sonar smells 2021-03-10 10:42:43 -05:00
Artem Bilan
b2a4e67db7 INT-4377: aggregator groupTimeout as Date (#3505)
* INT-4377: aggregator groupTimeout as Date

JIRA: https://jira.spring.io/browse/INT-4377

Change the `groupTimeoutExpression` logic to let it to be evaluated to `Date`
instance for some fine-grained scheduling use-case, e.g. to determine a
scheduling moment from the group creation time (`timestamp`) instead of a
current message arrival

* Fix language in docs accoridng PR review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-03-08 16:24:29 -05:00
Artem Bilan
e9f234683e INT-4444: Introduce @Reactive & reactive() (#3503)
* INT-4444: Introduce `@Reactive` & `reactive()`

JIRA: https://jira.spring.io/browse/INT-4444

Right now the high-level API creates a `ReactiveStreamsConsumer`
only when the input channel is a `Publisher<?>` impl or target handler
is a `ReactiveMessageHandler`

* Add `@Reactive[] reactive()` attribute to messaging annotations
* Add `ConsumerEndpointSpec.reactive()`
Both options point to the same `ConsumerEndpointFactoryBean.setReactiveCustomizer()`
making the target endpoint always as a `ReactiveStreamsConsumer` independently of
the input channel and target handler
* Use the `Function` to customize a source `Flux` from the channel
* Test and document a new feature

* * Fix links in docs

* * Fix `ReactiveStreamsTests`

* * Rework `reactive()` attribute of messaging annotations ot a single `@Reactive` value
with default as `@Reactive(ValueConstants.DEFAULT_NONE)`
* Fix language in docs
* Fix `MessagingAnnotationUtils.resolveAttribute()` to use `requiredType.isInstance()`
instead of comparing classes since annotation instances are `Proxy` at runtime
2021-03-08 12:02:31 -05:00
Artem Bilan
f154088935 Rework most of JMS tests to JUnit 5
* Optimize some resources usage relying on a shared object
* Remove some redundant `ActiveMqTestUtils`
2021-03-05 16:51:49 -05:00
Artem Bilan
9761c7f0b7 Make custom converter compatible with latest S-D
The latest changes in the `MappingMongoConverter` resolves the target type from the document.
That type is a `GenericMessage`.
The `GenericConversionService` doesn't match such a target type for the
`MessageReadConverter implements Converter<Document, Message<?>>` and fail.

* Change the `MessageReadConverter` generic type into the `GenericMessage` for matching data in
the collection
2021-03-03 17:35:23 -05:00
Artem Bilan
9d8393eb3e Miscellaneous improvement for end-user experience (#3500)
* Miscellaneous improvement for end-user experience

Related to https://stackoverflow.com/questions/66428239/ineligiblemethodexception-found-more-than-one-parameter-type-candidate-spring
Related to https://stackoverflow.com/questions/66397350/spring-integration-how-do-i-call-the-spring-integration-from-spring-controller

The exception thrown from gateway method call for `null` arg is not clear that it cannot be `null`
* Improve `GatewayMethodInboundMessageMapper.mapArgumentsToMessage()` exception message to show args
and explain that it cannot be resolved to `null`

When more than one params are candidates for method invocation, the `MessagingMethodInvokerHelper`
throws an `IneligibleMethodException` which outcome is not clear what to do
* Add `@Payload` annotation recommendation into an exception message to make the reason more cleaner
* Some other code style clean up in the `MessagingMethodInvokerHelper`

* * Fix `PseudoTransactionalMessageSourceTests.testCommit()` for better coverage

* Apply suggestions from code review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-03-03 13:48:32 -05:00
Artem Bilan
cb74a9494c Remove unused imports from tests 2021-03-01 14:00:55 -05:00
Artem Bilan
f285e81d08 Refactor scripting module
Since Nashorn JavaScript Engine is deprecated in Java 11 (https://openjdk.java.net/jeps/335)
it is better do nothing with JavaScript in the project any more

* Replace JS script tests to some other languages
* Mentioned deprecation in the Docs
* Rework `scripting.adoc` for code snippet switches
2021-03-01 13:38:03 -05:00
Artem Bilan
df3fc4c9a6 Fix race condition in MixedDispatchConfigTests 2021-02-26 15:06:30 -05:00
Artem Bilan
be34fb6866 Some test improvement; fix new Sonar smell 2021-02-26 14:20:49 -05:00
Artem Bilan
40464ca4cf Optimize OrderedAwareCopyOnWriteArraySetTests
It looks like `OrderedAwareCopyOnWriteArraySetTests` uses
extra repetitions in its tests for nothing relevant to the
tested functionality.
Only the problem that it spawns too many threads
2021-02-25 14:30:38 -05:00
Artem Bilan
19d7fb7df4 Improve IdGenerator tests
When one of the IdGenerator tests fails, it affect the rest
because the ctx is not closed and therefore headers field
for IdGenerator is not reset
2021-02-25 13:56:56 -05:00
Artem Bilan
c023816c7e Various tests fixes
* Use a tmp file in the `FileTests.testFileReadingFlow()`
for writing content and then rename it to the target file.
It looks like we may already have a file just with a short content
and it is picked up by the watch service for processing in the flow
* Rework `AggregatorWithCustomReleaseStrategyTests` to JUnit 5 and remove
extra looping logic since it just does not add any extra coverage just
performance overhead via crating and destroying the same application ctx
* Rework `ChannelAdapterParserTests` to `@SpringJUnitConfig` managed test
2021-02-25 11:33:16 -05:00
Artem Bilan
51b6ba8de8 INT-3585: JsonPropertyAccessor: skip failed parse
JIRA: https://jira.spring.io/browse/INT-3585

Even if this fix doesn't provide an order support,
it does cover an original premise of an issue
With this fix the order of accessors doesn't matter.

* Catch an `AccessException` in the `JsonPropertyAccessor.canRead()` and return `false`
to let the rest accessors to deal with the property requested
2021-02-25 11:28:51 -05:00
Artem Bilan
eef31d4b34 Fix new Sonar smells; optimize some tests 2021-02-24 11:34:29 -05:00
Artem Bilan
a0093f016d Clean up some schedulers and contexts in tests
* Migrate affected test to JUnit 5

All the work is done with an attempt to figure out an OOM errors on CI
2021-02-23 16:59:16 -05:00
Artem Bilan
f200849eda Move deps back to SNAPSHOTs
* Increase a `remoteTimeout` for `IpIntegrationTests.testCloseStream()`
2021-02-23 14:06:57 -05:00
Artem Bilan
6682c7db57 Increase JVM heap for test and HeapDumpOnOOMError 2021-02-23 12:30:03 -05:00
Artem Bilan
9c686ddcb7 More JVM memory for Gradle
* Fix race condition in the `KafkaOutboundAdapterParserTests`
2021-02-23 11:28:04 -05:00
Artem Bilan
ba3c9bf00f Use correct checksum for Gradle wrapper 2021-02-23 10:12:46 -05:00
Artem Bilan
4277cd713a Upgrade to Gradle 6.8.3 2021-02-23 10:05:26 -05:00
Artem Bilan
6c50419418 Increase Xmx for JVM in Gradle builds 2021-02-22 14:08:29 -05:00
Artem Bilan
64489e1aaa GH-3497: ReplyProducerCleaner: check only MPs
Fixes https://github.com/spring-projects/spring-integration/issues/3497

The `BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS` is filled up
only with the `MessageProducer` instances.
Therefore no reason to calculate a hash code for every single bean passed
to the `ReplyProducerCleaner.requiresDestruction()`

* Check for the `MessageProducer` instance before passing the bean to the
`BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS.contains()`
* Check for the `MessageProducer` in the `ReplyProducerCleaner.postProcessBeforeDestruction()`,
too.
It can be called independently of the `requiresDestruction()` and will calculate a hash code
from the bean again for nothing

**Cherry-pick to 5.4.x & 5.3.x**
2021-02-22 13:15:38 -05:00
Artem Bilan
6cad6edf8c Fix Checkstyle violation 2021-02-18 12:43:32 -05:00
Artem Bilan
a2081137f6 Fix new Sonar smells 2021-02-18 12:26:37 -05:00
Artem Bilan
e2e01db9a2 Add SFTP remote copy test-case 2021-02-18 11:31:08 -05:00
Spring Buildmaster
ee3ccc4493 [artifactory-release] Next development version 2021-02-17 23:35:20 +00:00
Spring Buildmaster
a395dd0ac5 [artifactory-release] Release version 5.5.0-M2 2021-02-17 23:35:16 +00:00
Artem Bilan
d1438e8a74 Upgrade dependencies; prepare for release
* Clean up code style in Gradle configs
* Fix JavaDoc errors
* Make JUnit 4 as `optional` dependency for `spring-integration-test-support`
* Remove redundant `javax.activation-api` from some modules
2021-02-17 16:49:52 -05:00
Artem Bilan
b97bedfc88 FileInbound DSL: Add recursive for convenience (#3495)
* FileInbound DSL: Add `recursive` for convenience

Related to https://stackoverflow.com/questions/66171881/how-to-read-nested-txt-file-from-spring-integration-file

The `FileInboundChannelAdapterSpec` can be configured with an external `DirectoryScanner`,
but it sometimes becomes burden for end-users to extract a scanner bean and configure it
with all the required file filters

* Expose a `recursive(boolean)` option for better end-user experience
* Rework `FileTests` for JUnit 5
* Mention a new option in the docs

* * Restore accidentally removed code
* Restore special symbols in the `FileTests`
* Fix language in the docs according review
2021-02-17 14:48:56 -05:00
Artem Bilan
281d8d5bf6 Deprecate Properties as bean for int global props
Having a bean as a `java.util.Properties` is not confusing
and may lead to some conflicts in the real application.
Plus it is not so easy to configure: need to know all
the possible integration properties - bad end-user experience

* Make an `IntegrationProperties` as a public POJO
for easy configuration of global properties
* Deprecate a presence of the `java.util.Properties`
* Leave framework-created `integrationGlobalProperties` as
a `Properties` instance for backward compatibility
* Fix tests to expose an `IntegrationProperties` bean instead
of deprecated `Properties`
* Fix docs according the change and current recommendations
2021-02-17 13:56:03 -05:00
Artem Bilan
aeb43f3069 JMS-in-gateway: destination to requestDestination (#3494)
* JMS-in-gateway: destination to requestDestination

Related to https://stackoverflow.com/questions/66174779/what-is-the-dsl-version-of-int-jmsinbound-gateway-request-destination

In Java DSL, the `JmsInboundGatewaySpec` expose a `destination()` option which is not correlated
what we have in XML with a `request-destination` and what we have with a `replyQueue` option

* Deprecate an existing `destination()` option in the `JmsInboundGatewaySpec`
in favor of newly introduced `requestDestination()`
* Clean up a bit `jms.adoc` and add some code block-switch sections to show
Java & Kotlin DSLs along side with existing XML configs

* Fix typo.

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-16 17:08:24 -05:00
Artem Bilan
b1cb9069fe Add MongoDbMessageSource UPDATE option (#3493)
* Add MongoDbMessageSource UPDATE option

* Extract `AbstractMongoDbMessageSource` with common options and methods
for both `MongoDbMessageSource` and `ReactiveMongoDbMessageSource`
* Add an `updateExpression` option into MongoDb source implementations
* Implement respective `update` logic after fetching the data from the collection
* Cover both reactive and blocking updates with tests
* Add `MongoDbMessageSourceSpec` into Java DSL for MongoDb channel adapters
* Expose an `update` XML attribute for the `<int-mongo:inbound-channel-adapter>`
* Upgrade MongoDb driver for latest Spring Data compatibility
* Document a new feature
* Upgrade `mongodb.adoc` for code block switch whenever it is appropriate

* * Add a Kotlin sample for `MongoDb.outboundGateway()` DSL

* Apply suggestions from code review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-16 16:46:45 -05:00
Artem Bilan
b4290dd2a5 GH-3492: Add ImapIdleChAdapterSpec.simpleContent
Fixes https://github.com/spring-projects/spring-integration/issues/3492

The `simpleContent` option is missed on the `ImapIdleChannelAdapterSpec`

* Add `ImapIdleChannelAdapterSpec.simpleContent(boolean)` option with
delegation to its internal `ImapMailReceiver`
* Convert all the `mail` tests to JUnit 5
2021-02-10 11:26:24 -05:00
Artem Bilan
43f90c4604 Fix some Sonar smells 2021-02-09 14:17:12 -05:00
Artem Bilan
52433b9aac Reinstate checksum for Gradle distribution
Looks like `distributionSha256Sum` property is removed from the
`gradle-wrapper.properties` when we run `wrapper` task.
We need this option for better performance on GH Actions when it builds
PRs.
Plus we won't see a respective warning over there
2021-02-08 16:48:06 -05:00
Artem Bilan
f940196c97 Gradle config clean up
* Move all the docs related configuration into a `docs.gradle`
* Move `publish-maven.gradle` into a `/gradle` dir for consistency
* Upgrade to Kotlin `1.4.30`
* Remove redundant `repository.apache.org` since we don't manage direct dependency for Kafka Client
* Upgrade to Gradle `6.8.2`
2021-02-08 16:18:56 -05:00
Gary Russell
138695be16 GH-3490: Initial Commit for block-switch Doc Plugin
Resolves https://github.com/spring-projects/spring-integration/issues/3490

- also fix double import of `jmx.adoc`
- also fix `redis.adoc`

* Remove redundant repo in `docs.gradle`
* Use `stylesheet.css` for `asciidoctorj` plugin config
* Fix `stylesheet.css` to import `spring.css` from the same dir
since they both are copied over there into the same dir
* Fix link for JMX chapter from the `metrics.adoc`
* Fix some inconsistency in the `amqp.adoc`
2021-02-08 14:48:35 -05:00
Artem Bilan
d55d8402d9 Fix race condition for FluxMessageChannelTests
The logic in the `FluxMessageChannelTests.testFluxMessageChannelCleanUp()`
relies on the `finishLatch.countDown()` which happens before we
release the lock for a group in the aggregator.
So, when we call a `destroy()` as the next statement, we interrupt
the lock instead of unlocking.
This cause a race condition in the Reactor's `Sink` to terminate properly

* Expose `CorrelationHandlerSpec.releaseLockBeforeSend()` and set it to `true`
in the test to properly unlock before we call `finishLatch.countDown()`
* Deprecate a couple introduced before options in favor of their variants
with `set` prefix
* Fix new Sonar smells
2021-02-08 10:33:01 -05:00
Gary Russell
338a26d34d GH-3488: Fix Persistent Filters with Recursion
Resolves https://github.com/spring-projects/spring-integration/issues/3488

Resolves two problems:

- When changes are made deep in the directory tree, they were not detected because
  the directory is in the metadata store and only passes the filter if a file
  immediately under it is changed, changing the directory's timestamp.

This is solved by subclassing `AbstractDirectoryAwareFileListFilter`, allowing its
`alwaysAcceptDirectories` property to be set.

- Only the filename was used as a metadata key; causing problems if a file with the
  same name appears multiple times in the tree.

This is solved with a new property on `AbstractDirectoryAwareFileListFilter` used by
the gateways to determine whether to filter the raw file names returned by the session
(previous behavior) or the full path relative to the root directory.

**cherry-pick to 5.4.x, 5.3.x**

* Some code style clean up
2021-02-05 16:49:31 -05:00
Artem Bilan
d070eb5783 GH-3464: Treat 0 as special for polling endpoint (#3487)
* GH-3464: Treat 0 as special for polling endpoint

Fixes https://github.com/spring-projects/spring-integration/issues/3464

The `maxMessagePerPoll <= 0` is considered as an unbound `receive()` call.
End-users would like to have a special treatment for `0` value -
skip the `receive()` call altogether for the current polling cycle.

* Change the logic for a scheduled poller to not call `pollForMessage()`
and just log an INFO when `maxMessagePerPoll == 0`
* Fix reactive poller to deal with `maxMessagePerPoll == 0` properly
* Expose `maxMessagesPerPoll` as a `@ManagedAttribute` to let it to be
modified via Control Bus and JMX
* Test and document a new behavior

* * Fix unused imports in the test class

* Fix language in docs accoridng PR review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-05 10:50:17 -05:00
Artem Bilan
88d2bebb9c Fix new Sonar smells 2021-02-03 10:54:30 -05:00
Artem Bilan
e847570b85 Fix new Sonar smells
* Fix typos in the `amqp.adoc`
2021-02-02 10:13:37 -05:00
Artem Bilan
77a343c3cf GH-3483: Fallback to RestTemplate.converters (#3485)
* GH-3483: Fallback to RestTemplate.converters

Fixes https://github.com/spring-projects/spring-integration/issues/3483

When there is no reasonable way to determine a `Content-Type`
from the request message, do not set an `application/x-java-serialized-object`
as a fallback and let the `RestTemplate` to determine the target type and
conversion through its `HttpMessageConverter` set

* Remove `application/x-java-serialized-object` fallback from the
`AbstractHttpRequestExecutingMessageHandler`
* Adjust its log messages according `LogAccessor`
* Un`@Disable` `WebFluxDslTests` since fix was done in Spring Security
* Add `HttpRequestExecutingMessageHandlerTests.testNoContentTypeAndSmartConverter()`
* Mention change in `What's New`

* Fix language in whats-new.adoc

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-01 16:53:30 -05:00
Gary Russell
d7740706bb GH-3482: (S)FTP: Fix Recursive LS (ARFOG)
Resolves https://github.com/spring-projects/spring-integration/issues/3482

`.` and `..` should be ignored when recursing.

**cherry-pick to 5.4.x, 5.3.x**

* Fix checkstyle.

* Fix test in `file` module - test was incorrect; it would have detected this problem.
2021-02-01 16:04:08 -05:00
Artem Bilan
98fbf62db2 Fix maxMessagesPerPoll for reactive poll endpoint
The current `Flux.take()` doesn't allow an arg `< 0` treating
it as an unbound request.

* Change `take()` to `limitRequest()` according strict `MessageSource.receive()`
producing expectations
* Treat `maxMessagesPerPoll < 0` as a `Long.MAX_VALUE` for unbound requests;
`0` is treated in the `limitRequest()` as "no more requests - cancel"
* Revise `AbstractPollingEndpoint` for `LogAccessor` usage
* Add `AbstractPollingEndpoint` class JavaDocs
* Fix tests according `AbstractPollingEndpoint` changes

**Cherry-pick to `5.4.x`**
2021-02-01 13:48:02 -05:00
Artem Bilan
e752c70fec Fix unused import in the JpaExecutor 2021-01-29 19:31:43 -05:00