Commit Graph

10791 Commits

Author SHA1 Message Date
Artem Bilan
4277cd713a Upgrade to Gradle 6.8.3 2021-02-23 10:05:26 -05:00
Artem Bilan
6c50419418 Increase Xmx for JVM in Gradle builds 2021-02-22 14:08:29 -05:00
Artem Bilan
64489e1aaa GH-3497: ReplyProducerCleaner: check only MPs
Fixes https://github.com/spring-projects/spring-integration/issues/3497

The `BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS` is filled up
only with the `MessageProducer` instances.
Therefore no reason to calculate a hash code for every single bean passed
to the `ReplyProducerCleaner.requiresDestruction()`

* Check for the `MessageProducer` instance before passing the bean to the
`BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS.contains()`
* Check for the `MessageProducer` in the `ReplyProducerCleaner.postProcessBeforeDestruction()`,
too.
It can be called independently of the `requiresDestruction()` and will calculate a hash code
from the bean again for nothing

**Cherry-pick to 5.4.x & 5.3.x**
2021-02-22 13:15:38 -05:00
Artem Bilan
6cad6edf8c Fix Checkstyle violation 2021-02-18 12:43:32 -05:00
Artem Bilan
a2081137f6 Fix new Sonar smells 2021-02-18 12:26:37 -05:00
Artem Bilan
e2e01db9a2 Add SFTP remote copy test-case 2021-02-18 11:31:08 -05:00
Spring Buildmaster
ee3ccc4493 [artifactory-release] Next development version 2021-02-17 23:35:20 +00:00
Spring Buildmaster
a395dd0ac5 [artifactory-release] Release version 5.5.0-M2 2021-02-17 23:35:16 +00:00
Artem Bilan
d1438e8a74 Upgrade dependencies; prepare for release
* Clean up code style in Gradle configs
* Fix JavaDoc errors
* Make JUnit 4 as `optional` dependency for `spring-integration-test-support`
* Remove redundant `javax.activation-api` from some modules
2021-02-17 16:49:52 -05:00
Artem Bilan
b97bedfc88 FileInbound DSL: Add recursive for convenience (#3495)
* FileInbound DSL: Add `recursive` for convenience

Related to https://stackoverflow.com/questions/66171881/how-to-read-nested-txt-file-from-spring-integration-file

The `FileInboundChannelAdapterSpec` can be configured with an external `DirectoryScanner`,
but it sometimes becomes burden for end-users to extract a scanner bean and configure it
with all the required file filters

* Expose a `recursive(boolean)` option for better end-user experience
* Rework `FileTests` for JUnit 5
* Mention a new option in the docs

* * Restore accidentally removed code
* Restore special symbols in the `FileTests`
* Fix language in the docs according review
2021-02-17 14:48:56 -05:00
Artem Bilan
281d8d5bf6 Deprecate Properties as bean for int global props
Having a bean as a `java.util.Properties` is not confusing
and may lead to some conflicts in the real application.
Plus it is not so easy to configure: need to know all
the possible integration properties - bad end-user experience

* Make an `IntegrationProperties` as a public POJO
for easy configuration of global properties
* Deprecate a presence of the `java.util.Properties`
* Leave framework-created `integrationGlobalProperties` as
a `Properties` instance for backward compatibility
* Fix tests to expose an `IntegrationProperties` bean instead
of deprecated `Properties`
* Fix docs according the change and current recommendations
2021-02-17 13:56:03 -05:00
Artem Bilan
aeb43f3069 JMS-in-gateway: destination to requestDestination (#3494)
* JMS-in-gateway: destination to requestDestination

Related to https://stackoverflow.com/questions/66174779/what-is-the-dsl-version-of-int-jmsinbound-gateway-request-destination

In Java DSL, the `JmsInboundGatewaySpec` expose a `destination()` option which is not correlated
what we have in XML with a `request-destination` and what we have with a `replyQueue` option

* Deprecate an existing `destination()` option in the `JmsInboundGatewaySpec`
in favor of newly introduced `requestDestination()`
* Clean up a bit `jms.adoc` and add some code block-switch sections to show
Java & Kotlin DSLs along side with existing XML configs

* Fix typo.

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-16 17:08:24 -05:00
Artem Bilan
b1cb9069fe Add MongoDbMessageSource UPDATE option (#3493)
* Add MongoDbMessageSource UPDATE option

* Extract `AbstractMongoDbMessageSource` with common options and methods
for both `MongoDbMessageSource` and `ReactiveMongoDbMessageSource`
* Add an `updateExpression` option into MongoDb source implementations
* Implement respective `update` logic after fetching the data from the collection
* Cover both reactive and blocking updates with tests
* Add `MongoDbMessageSourceSpec` into Java DSL for MongoDb channel adapters
* Expose an `update` XML attribute for the `<int-mongo:inbound-channel-adapter>`
* Upgrade MongoDb driver for latest Spring Data compatibility
* Document a new feature
* Upgrade `mongodb.adoc` for code block switch whenever it is appropriate

* * Add a Kotlin sample for `MongoDb.outboundGateway()` DSL

* Apply suggestions from code review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-16 16:46:45 -05:00
Artem Bilan
b4290dd2a5 GH-3492: Add ImapIdleChAdapterSpec.simpleContent
Fixes https://github.com/spring-projects/spring-integration/issues/3492

The `simpleContent` option is missed on the `ImapIdleChannelAdapterSpec`

* Add `ImapIdleChannelAdapterSpec.simpleContent(boolean)` option with
delegation to its internal `ImapMailReceiver`
* Convert all the `mail` tests to JUnit 5
2021-02-10 11:26:24 -05:00
Artem Bilan
43f90c4604 Fix some Sonar smells 2021-02-09 14:17:12 -05:00
Artem Bilan
52433b9aac Reinstate checksum for Gradle distribution
Looks like `distributionSha256Sum` property is removed from the
`gradle-wrapper.properties` when we run `wrapper` task.
We need this option for better performance on GH Actions when it builds
PRs.
Plus we won't see a respective warning over there
2021-02-08 16:48:06 -05:00
Artem Bilan
f940196c97 Gradle config clean up
* Move all the docs related configuration into a `docs.gradle`
* Move `publish-maven.gradle` into a `/gradle` dir for consistency
* Upgrade to Kotlin `1.4.30`
* Remove redundant `repository.apache.org` since we don't manage direct dependency for Kafka Client
* Upgrade to Gradle `6.8.2`
2021-02-08 16:18:56 -05:00
Gary Russell
138695be16 GH-3490: Initial Commit for block-switch Doc Plugin
Resolves https://github.com/spring-projects/spring-integration/issues/3490

- also fix double import of `jmx.adoc`
- also fix `redis.adoc`

* Remove redundant repo in `docs.gradle`
* Use `stylesheet.css` for `asciidoctorj` plugin config
* Fix `stylesheet.css` to import `spring.css` from the same dir
since they both are copied over there into the same dir
* Fix link for JMX chapter from the `metrics.adoc`
* Fix some inconsistency in the `amqp.adoc`
2021-02-08 14:48:35 -05:00
Artem Bilan
d55d8402d9 Fix race condition for FluxMessageChannelTests
The logic in the `FluxMessageChannelTests.testFluxMessageChannelCleanUp()`
relies on the `finishLatch.countDown()` which happens before we
release the lock for a group in the aggregator.
So, when we call a `destroy()` as the next statement, we interrupt
the lock instead of unlocking.
This cause a race condition in the Reactor's `Sink` to terminate properly

* Expose `CorrelationHandlerSpec.releaseLockBeforeSend()` and set it to `true`
in the test to properly unlock before we call `finishLatch.countDown()`
* Deprecate a couple introduced before options in favor of their variants
with `set` prefix
* Fix new Sonar smells
2021-02-08 10:33:01 -05:00
Gary Russell
338a26d34d GH-3488: Fix Persistent Filters with Recursion
Resolves https://github.com/spring-projects/spring-integration/issues/3488

Resolves two problems:

- When changes are made deep in the directory tree, they were not detected because
  the directory is in the metadata store and only passes the filter if a file
  immediately under it is changed, changing the directory's timestamp.

This is solved by subclassing `AbstractDirectoryAwareFileListFilter`, allowing its
`alwaysAcceptDirectories` property to be set.

- Only the filename was used as a metadata key; causing problems if a file with the
  same name appears multiple times in the tree.

This is solved with a new property on `AbstractDirectoryAwareFileListFilter` used by
the gateways to determine whether to filter the raw file names returned by the session
(previous behavior) or the full path relative to the root directory.

**cherry-pick to 5.4.x, 5.3.x**

* Some code style clean up
2021-02-05 16:49:31 -05:00
Artem Bilan
d070eb5783 GH-3464: Treat 0 as special for polling endpoint (#3487)
* GH-3464: Treat 0 as special for polling endpoint

Fixes https://github.com/spring-projects/spring-integration/issues/3464

The `maxMessagePerPoll <= 0` is considered as an unbound `receive()` call.
End-users would like to have a special treatment for `0` value -
skip the `receive()` call altogether for the current polling cycle.

* Change the logic for a scheduled poller to not call `pollForMessage()`
and just log an INFO when `maxMessagePerPoll == 0`
* Fix reactive poller to deal with `maxMessagePerPoll == 0` properly
* Expose `maxMessagesPerPoll` as a `@ManagedAttribute` to let it to be
modified via Control Bus and JMX
* Test and document a new behavior

* * Fix unused imports in the test class

* Fix language in docs accoridng PR review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-05 10:50:17 -05:00
Artem Bilan
88d2bebb9c Fix new Sonar smells 2021-02-03 10:54:30 -05:00
Artem Bilan
e847570b85 Fix new Sonar smells
* Fix typos in the `amqp.adoc`
2021-02-02 10:13:37 -05:00
Artem Bilan
77a343c3cf GH-3483: Fallback to RestTemplate.converters (#3485)
* GH-3483: Fallback to RestTemplate.converters

Fixes https://github.com/spring-projects/spring-integration/issues/3483

When there is no reasonable way to determine a `Content-Type`
from the request message, do not set an `application/x-java-serialized-object`
as a fallback and let the `RestTemplate` to determine the target type and
conversion through its `HttpMessageConverter` set

* Remove `application/x-java-serialized-object` fallback from the
`AbstractHttpRequestExecutingMessageHandler`
* Adjust its log messages according `LogAccessor`
* Un`@Disable` `WebFluxDslTests` since fix was done in Spring Security
* Add `HttpRequestExecutingMessageHandlerTests.testNoContentTypeAndSmartConverter()`
* Mention change in `What's New`

* Fix language in whats-new.adoc

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-02-01 16:53:30 -05:00
Gary Russell
d7740706bb GH-3482: (S)FTP: Fix Recursive LS (ARFOG)
Resolves https://github.com/spring-projects/spring-integration/issues/3482

`.` and `..` should be ignored when recursing.

**cherry-pick to 5.4.x, 5.3.x**

* Fix checkstyle.

* Fix test in `file` module - test was incorrect; it would have detected this problem.
2021-02-01 16:04:08 -05:00
Artem Bilan
98fbf62db2 Fix maxMessagesPerPoll for reactive poll endpoint
The current `Flux.take()` doesn't allow an arg `< 0` treating
it as an unbound request.

* Change `take()` to `limitRequest()` according strict `MessageSource.receive()`
producing expectations
* Treat `maxMessagesPerPoll < 0` as a `Long.MAX_VALUE` for unbound requests;
`0` is treated in the `limitRequest()` as "no more requests - cancel"
* Revise `AbstractPollingEndpoint` for `LogAccessor` usage
* Add `AbstractPollingEndpoint` class JavaDocs
* Fix tests according `AbstractPollingEndpoint` changes

**Cherry-pick to `5.4.x`**
2021-02-01 13:48:02 -05:00
Artem Bilan
e752c70fec Fix unused import in the JpaExecutor 2021-01-29 19:31:43 -05:00
Artem Bilan
4079cea8f5 Fix GatewayParserTests test against LogAccessor 2021-01-29 19:21:13 -05:00
Artem Bilan
ece0216bfe Fix new Sonar smells 2021-01-29 18:11:17 -05:00
Artem Bilan
8bff67425d Improve Sonar and Jacoco plugin configurations 2021-01-29 17:23:57 -05:00
Artem Bilan
73d9b8cccd Increase receive timeout in ZeroMqMHTests 2021-01-29 12:43:08 -05:00
Artem Bilan
1c6cf560d7 Expose error.ignoreFailures global property (#3481)
* Expose `error.ignoreFailures` global property

The global default `errorChannel` can have more then only one logging subscriber.
There is no guarantee in which order subscribers are called and one of them fails
the rest are not called.

* Expose `spring.integration.channels.error.ignoreFailures` global property for
default `errorChannel` and make it `true` by default for better end-user experience
when it is expected to have error being handled by all the subscribers

* * Fix language in the `whats-new.adoc`

* * Fix `JmsTests` for error channel to propagate an exception

* More language fixes for whats-new.adoc

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-01-28 17:12:36 -05:00
Artem Bilan
0b63128da6 GH-3473: Fix dead lock around lifecycleLock
Fixes https://github.com/spring-projects/spring-integration/issues/3473

When `AbstractEndpoint.start()` and `AbstractEndpoint.isRunning()`
are called from different thread on `synchronized` methods,
we may end up with a dead lock: one thread waits for monitor on
`synchronized` and another waits for the `lifecycleLock`

* Change `AbstractEndpoint.isRunning()` to a plain `return this.running;`
- there is no reason in a lock around returning this `volatile` property state

**Cherry-pick to 5.4.x & 5.3.x**
2021-01-27 11:42:22 -05:00
Artem Bilan
31863f06e3 GH-3474: LambdaMPP: rethrow runtime ex as is
Fixes https://github.com/spring-projects/spring-integration/issues/3474

The `AbstractTransformer` implementation just wraps a thrown exception into
a `MessageTransformationException`.
It is expected with any method transformer impl (POJO, Lambda direct etc.)
the exception is thrown as is without extra wrapping into an `IllegalStateException`
for consistency

* Fix `LambdaMessageProcessor` to rethrow `RuntimeException` as is
and only wrap into an `IllegalStateException` for all other exceptions
* Fix tests for missed extra stack trace
2021-01-26 16:06:29 -05:00
Artem Bilan
707b653d97 Copy request headers when reply is not a Message (#3478)
* Copy request headers when reply is not a Message

Related to https://stackoverflow.com/questions/65887787/spring-integration-unexpected-behavior-on-error-handling-in-splitter-aggregator

Typically `AbstractMessageProducingHandler` implementation with the `shouldCopyRequestHeaders = false`
returns a `Message`(or `MessageBuilder`) as a reply, but when some request handler `Advice` is involved,
we may produce a plain payload and the reply message does not have request headers

* Improve `AbstractMessageProducingHandler` logic to copy request headers into a reply message builder,
when the original reply is not a `Message` or `MessageBuilder`.
This way we improve end-user experience and set them free from copying those headers manually
* Justify a new behavior with the `TransformerTests.testFailedTransformWithRequestHeadersCopy()`

* * Fix Checkstyle violations
2021-01-26 16:02:19 -05:00
Gary Russell
f223343d0e GH-3477: Add DSL Docs for TCP Components
Resolves https://github.com/spring-projects/spring-integration/issues/3477

**cherry-pick to 5.4.x**
2021-01-26 15:52:18 -05:00
Artem Bilan
fa4957e545 Move Spring dependencies to SNAPSHOTs 2021-01-25 17:02:58 -05:00
Artem Bilan
335a30c0f0 Fix new Sonar smell 2021-01-25 11:10:59 -05:00
Gary Russell
524ad1cb0b GH-3461: Readme Polishing, eclipse and build
Resolves https://github.com/spring-projects/spring-integration/issues/3461

* Fix project URL
* Fix IDEA instructions
2021-01-22 12:24:47 -05:00
Gary Russell
f5b671c28c GH-3470: Resolve JDK8 Runtime Problem
Resolves https://github.com/spring-projects/spring-integration/issues/3470

Also fix `ByteBuffer.clear()`.
2021-01-22 11:53:58 -05:00
Artem Bilan
6ff5eac86c Increase timeouts for RedisLockLeaderInitTests
* `setBusyWaitMillis(10000)` before yielding to let another candidate to lock
* Add `Thread.sleep(100);` before yielding
2021-01-22 11:22:41 -05:00
Gary Russell
8daef115f7 GH-3470: Resolve JDK8 Runtime Problem
Resolves https://github.com/spring-projects/spring-integration/issues/3470

In order to catch issues like this at compile time, we need to set
the `bootClasspath` to ensure API compatibility but this would need some
consistent way to point the JDK 8 libs on all platforms (CI, GitHub Workflows,
dev machines, etc).
2021-01-22 10:54:04 -05:00
Artem Bilan
8763c95622 GH-3469 New Sonar smell & volatile busyWaitMillis
Fixes https://github.com/spring-projects/spring-integration/issues/3469

* Fix new Sonar smells
* Mark `LockRegistryLeaderInitiator.busyWaitMillis` as `volatile`
so runtime changes (e.g. `RedisLockRegistryLeaderInitiatorTests`)
will make an immediate effect
2021-01-21 14:55:14 -05:00
Artem Bilan
58ce578a10 Downgrade back to Sonar plugin 2.8
Looks the latest plugin is not compatible with latest Sonar server -
it doesn't trigger Coverage
2021-01-21 12:39:15 -05:00
Artem Bilan
88c3d0690e Upgrade to Sonar 3.1 2021-01-21 10:59:52 -05:00
Spring Buildmaster
aa4a186825 [artifactory-release] Next development version 2021-01-20 20:30:02 +00:00
Spring Buildmaster
b1d7ba19da [artifactory-release] Release version 5.5.0-M1 2021-01-20 20:29:57 +00:00
Gary Russell
14fe3093f2 Ignore Flaky Test 2021-01-20 15:22:18 -05:00
Gary Russell
bbc10fa6fc Revert spring-ws to 3.0.10; prepare for milestone 2021-01-20 14:57:02 -05:00
Artem Bilan
eabe891c15 Move dependencies to their milestones/latest GAs 2021-01-20 14:14:08 -05:00