Commit Graph

10680 Commits

Author SHA1 Message Date
Gary Russell
90a2f4bff1 GH-3509: Fix if test in previous commit
Caused interceptor to be skipped.

**cherry-pick to 5.4.x**

* Fix race in new tests

* Remove overloaded conn id in test interceptor.
2021-03-17 18:33:34 -04:00
Gary Russell
482673a159 GH-3509: Fix Memory Leak with Intercepted TCP Conn
Resolves https://github.com/spring-projects/spring-integration/issues/3509

When a connection was intercepted, the `TcpSender.add...` was called with the
interceptor, but the `removeDead...` was called with the actual connection,
causing a memory leak.

Always call the `TcpSender` with the outer-most interceptor.

# Conflicts:
#	spring-integration-ip/src/main/java/org/springframework/integration/ip/tcp/connection/TcpNetServerConnectionFactory.java
#	spring-integration-ip/src/test/java/org/springframework/integration/ip/tcp/TcpSendingMessageHandlerTests.java
2021-03-17 17:54:15 -04:00
Artem Bilan
321f03620f Revert "GH-3509 Register TcpSenders on wrapped connection"
This reverts commit 78a0ae8a04.
2021-03-17 17:45:28 -04:00
Artem Bilan
6eb0ff1f32 Upgrade dependencies; prepare for release 2021-03-17 17:44:58 -04:00
Artem Bilan
f07c44ba2d Add Publish Unit Test Results to GH Actions
* Upgrade dependencies
* Fix repos to not use superset to fail for
non-authorized artifacts

* * Revert Dokka plugin - too much JCenter deps
* Add `if: always()` for Test Results in GH Actions

* Add `https://repo.spring.io/release` for docs

* * Use `upload-artifact` action instead -
the `publish-unit-test-result-action` doesn't capture against PRs:
https://github.com/EnricoMi/publish-unit-test-result-action#support-fork-repositories
* `retention-days: 3`
2021-03-16 17:27:01 -04:00
Artem Bilan
4342586361 INT-4116: Introduce FileAggregator (#3511)
* INT-4116: Introduce FileAggregator

JIRA: https://jira.spring.io/browse/INT-4116

* Implement a `FileSplitter.FileMarker`-based aggregation strategies
and utilize them in a general `FileAggregator` component
* Make `HeaderAttributeCorrelationStrategy.attributeName` as `final`; add `Assert.notEmpty()`
* Fix `AggregatorFactoryBean` and `AggregatorSpec` to parse the provided processor
for possible `CorrelationStrategy` and/or `ReleaseStrategy`
* Introduce short-cut methods into Java & Kotlin DSL for an `aggregate()` configuration
* Introduce a `FileHeaders.LINE_COUNT` for header to be populated in the `FileSplitter`.
We need this info in the `FileAggregator` to avoid possible overhead with JSON deserialization
of the `FileSplitter.FileMarker` messages
* Test and document the feature
* Improve `FileSplitter` doc for code block switch (tabs)

* * Rework FileAggregator do not use Java Streams
2021-03-16 15:06:36 -04:00
Alex Vester
be2a698618 GH-3512: NULLS LAST for PostgreSQL priority index
Fixes https://github.com/spring-projects/spring-integration/issues/3512

See https://use-the-index-luke.com/sql/sorting-grouping/order-by-asc-desc-nulls-last for more info
2021-03-15 13:20:22 -04:00
Artem Bilan
bb608d8922 Rework more JMS tests to common CF resource
* Refactor STOMP and WebSocket tests into JUnit 5
* Close a server application context in those tests when Tomcat is destroyed
2021-03-15 12:03:27 -04:00
Mário Dias
78a0ae8a04 GH-3509 Register TcpSenders on wrapped connection
Fixes https://github.com/spring-projects/spring-integration/issues/3509

Dead connections are not being removed on `TcpSender` when using a `TcpConnectionInterceptor`.
May cause a memory leak

* Fix `TcpConnectionInterceptorSupport` to override `registerSenders()`
 and delegate to the `this.theConnection`
* Introduce vararg-based `setInterceptor()` into `TcpConnectionInterceptorFactoryChain`
* Remove redundant `//NOSONAR` and properly return an `Arrays.copyOf()` in the `getInterceptorFactories()`

**Cherry-pick to `5.4.x`**
2021-03-12 15:44:32 -05:00
Artem Bilan
42fa248753 Downgrade to Gradle 6.8.2
The CI plan has started to fails sporadically.
Currently with Gradle `6.8.3` we observer something like this:
in the Standard Output the stack trace is from another thread in another test (that timed out while this test was running).
It seems like Gradle is failing the test if it sees a stack trace in the log

* Trying with the previous Gradle version
2021-03-12 15:19:05 -05:00
Gary Russell
ebb8a65f73 CachingClientConnectionFactoryTests Fix Timeout
- second "Response" log causes a timeout, producing log noise in
later tests.
2021-03-12 13:49:32 -05:00
Gary Russell
2522c79cb3 Catch async MessageTimeoutException in IP test
- fix async exec thread names
2021-03-12 13:40:08 -05:00
Gary Russell
84ea4422a2 Name executors in IP tests
- aid in debugging failing tests
2021-03-12 13:20:21 -05:00
Gary Russell
3edfb962d2 Fix missing @DirtiesContext on TCP tests
- also add bean name (if available) to the gateway timeout exception message.
2021-03-12 11:43:17 -05:00
Artem Bilan
13f0bde989 Some clean up for jms.adoc 2021-03-11 14:38:48 -05:00
Trung Pham
f30ede47e2 GH-3507: Fix Tail producer for proper command
Fixes https://github.com/spring-projects/spring-integration/issues/3507

The `OSDelegatingFileTailingMessageProducer` passing command string to `Runtime.getRuntime().exec()`
may cause problems if spaces (and other special characters) are used in the filename.

* Use an array for command and its options to let the target `Runtime` to parse and
execute it properly

**Cherry-pick to 5.4.x, 5.3.x & 5.2.x**
2021-03-11 13:58:30 -05:00
Artem Bilan
010ce8b01f Fail JmsMDEndpoint if container has a listener
Right now the `JmsMessageDrivenEndpoint` only logs a warn about
already provided listener.
Turns out this is not enough for end-users and they may miss
the warn message in the logs somehow having some problems at
runtime trying to determine find out the root of unexpected behavior

* Change the warn message to the `Assert.isNull()` to fail fast.
Same is done in the `KafkaMessageDrivenChannelAdapter` & `AmqpInboundChannelAdapter`
2021-03-10 16:22:53 -05:00
Artem Bilan
a1a4341655 Add logging to CachingClientConnFactoryTests 2021-03-10 11:52:00 -05:00
Artem Bilan
207de60c0a Add logging diagnostics to UdpChannelAdapterTests 2021-03-10 11:16:18 -05:00
Artem Bilan
61153a96fc Resolve new Sonar smells 2021-03-10 10:42:43 -05:00
Artem Bilan
b2a4e67db7 INT-4377: aggregator groupTimeout as Date (#3505)
* INT-4377: aggregator groupTimeout as Date

JIRA: https://jira.spring.io/browse/INT-4377

Change the `groupTimeoutExpression` logic to let it to be evaluated to `Date`
instance for some fine-grained scheduling use-case, e.g. to determine a
scheduling moment from the group creation time (`timestamp`) instead of a
current message arrival

* Fix language in docs accoridng PR review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-03-08 16:24:29 -05:00
Artem Bilan
e9f234683e INT-4444: Introduce @Reactive & reactive() (#3503)
* INT-4444: Introduce `@Reactive` & `reactive()`

JIRA: https://jira.spring.io/browse/INT-4444

Right now the high-level API creates a `ReactiveStreamsConsumer`
only when the input channel is a `Publisher<?>` impl or target handler
is a `ReactiveMessageHandler`

* Add `@Reactive[] reactive()` attribute to messaging annotations
* Add `ConsumerEndpointSpec.reactive()`
Both options point to the same `ConsumerEndpointFactoryBean.setReactiveCustomizer()`
making the target endpoint always as a `ReactiveStreamsConsumer` independently of
the input channel and target handler
* Use the `Function` to customize a source `Flux` from the channel
* Test and document a new feature

* * Fix links in docs

* * Fix `ReactiveStreamsTests`

* * Rework `reactive()` attribute of messaging annotations ot a single `@Reactive` value
with default as `@Reactive(ValueConstants.DEFAULT_NONE)`
* Fix language in docs
* Fix `MessagingAnnotationUtils.resolveAttribute()` to use `requiredType.isInstance()`
instead of comparing classes since annotation instances are `Proxy` at runtime
2021-03-08 12:02:31 -05:00
Artem Bilan
f154088935 Rework most of JMS tests to JUnit 5
* Optimize some resources usage relying on a shared object
* Remove some redundant `ActiveMqTestUtils`
2021-03-05 16:51:49 -05:00
Artem Bilan
9761c7f0b7 Make custom converter compatible with latest S-D
The latest changes in the `MappingMongoConverter` resolves the target type from the document.
That type is a `GenericMessage`.
The `GenericConversionService` doesn't match such a target type for the
`MessageReadConverter implements Converter<Document, Message<?>>` and fail.

* Change the `MessageReadConverter` generic type into the `GenericMessage` for matching data in
the collection
2021-03-03 17:35:23 -05:00
Artem Bilan
9d8393eb3e Miscellaneous improvement for end-user experience (#3500)
* Miscellaneous improvement for end-user experience

Related to https://stackoverflow.com/questions/66428239/ineligiblemethodexception-found-more-than-one-parameter-type-candidate-spring
Related to https://stackoverflow.com/questions/66397350/spring-integration-how-do-i-call-the-spring-integration-from-spring-controller

The exception thrown from gateway method call for `null` arg is not clear that it cannot be `null`
* Improve `GatewayMethodInboundMessageMapper.mapArgumentsToMessage()` exception message to show args
and explain that it cannot be resolved to `null`

When more than one params are candidates for method invocation, the `MessagingMethodInvokerHelper`
throws an `IneligibleMethodException` which outcome is not clear what to do
* Add `@Payload` annotation recommendation into an exception message to make the reason more cleaner
* Some other code style clean up in the `MessagingMethodInvokerHelper`

* * Fix `PseudoTransactionalMessageSourceTests.testCommit()` for better coverage

* Apply suggestions from code review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-03-03 13:48:32 -05:00
Artem Bilan
cb74a9494c Remove unused imports from tests 2021-03-01 14:00:55 -05:00
Artem Bilan
f285e81d08 Refactor scripting module
Since Nashorn JavaScript Engine is deprecated in Java 11 (https://openjdk.java.net/jeps/335)
it is better do nothing with JavaScript in the project any more

* Replace JS script tests to some other languages
* Mentioned deprecation in the Docs
* Rework `scripting.adoc` for code snippet switches
2021-03-01 13:38:03 -05:00
Artem Bilan
df3fc4c9a6 Fix race condition in MixedDispatchConfigTests 2021-02-26 15:06:30 -05:00
Artem Bilan
be34fb6866 Some test improvement; fix new Sonar smell 2021-02-26 14:20:49 -05:00
Artem Bilan
40464ca4cf Optimize OrderedAwareCopyOnWriteArraySetTests
It looks like `OrderedAwareCopyOnWriteArraySetTests` uses
extra repetitions in its tests for nothing relevant to the
tested functionality.
Only the problem that it spawns too many threads
2021-02-25 14:30:38 -05:00
Artem Bilan
19d7fb7df4 Improve IdGenerator tests
When one of the IdGenerator tests fails, it affect the rest
because the ctx is not closed and therefore headers field
for IdGenerator is not reset
2021-02-25 13:56:56 -05:00
Artem Bilan
c023816c7e Various tests fixes
* Use a tmp file in the `FileTests.testFileReadingFlow()`
for writing content and then rename it to the target file.
It looks like we may already have a file just with a short content
and it is picked up by the watch service for processing in the flow
* Rework `AggregatorWithCustomReleaseStrategyTests` to JUnit 5 and remove
extra looping logic since it just does not add any extra coverage just
performance overhead via crating and destroying the same application ctx
* Rework `ChannelAdapterParserTests` to `@SpringJUnitConfig` managed test
2021-02-25 11:33:16 -05:00
Artem Bilan
51b6ba8de8 INT-3585: JsonPropertyAccessor: skip failed parse
JIRA: https://jira.spring.io/browse/INT-3585

Even if this fix doesn't provide an order support,
it does cover an original premise of an issue
With this fix the order of accessors doesn't matter.

* Catch an `AccessException` in the `JsonPropertyAccessor.canRead()` and return `false`
to let the rest accessors to deal with the property requested
2021-02-25 11:28:51 -05:00
Artem Bilan
eef31d4b34 Fix new Sonar smells; optimize some tests 2021-02-24 11:34:29 -05:00
Artem Bilan
a0093f016d Clean up some schedulers and contexts in tests
* Migrate affected test to JUnit 5

All the work is done with an attempt to figure out an OOM errors on CI
2021-02-23 16:59:16 -05:00
Artem Bilan
f200849eda Move deps back to SNAPSHOTs
* Increase a `remoteTimeout` for `IpIntegrationTests.testCloseStream()`
2021-02-23 14:06:57 -05:00
Artem Bilan
6682c7db57 Increase JVM heap for test and HeapDumpOnOOMError 2021-02-23 12:30:03 -05:00
Artem Bilan
9c686ddcb7 More JVM memory for Gradle
* Fix race condition in the `KafkaOutboundAdapterParserTests`
2021-02-23 11:28:04 -05:00
Artem Bilan
ba3c9bf00f Use correct checksum for Gradle wrapper 2021-02-23 10:12:46 -05:00
Artem Bilan
4277cd713a Upgrade to Gradle 6.8.3 2021-02-23 10:05:26 -05:00
Artem Bilan
6c50419418 Increase Xmx for JVM in Gradle builds 2021-02-22 14:08:29 -05:00
Artem Bilan
64489e1aaa GH-3497: ReplyProducerCleaner: check only MPs
Fixes https://github.com/spring-projects/spring-integration/issues/3497

The `BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS` is filled up
only with the `MessageProducer` instances.
Therefore no reason to calculate a hash code for every single bean passed
to the `ReplyProducerCleaner.requiresDestruction()`

* Check for the `MessageProducer` instance before passing the bean to the
`BaseIntegrationFlowDefinition.REFERENCED_REPLY_PRODUCERS.contains()`
* Check for the `MessageProducer` in the `ReplyProducerCleaner.postProcessBeforeDestruction()`,
too.
It can be called independently of the `requiresDestruction()` and will calculate a hash code
from the bean again for nothing

**Cherry-pick to 5.4.x & 5.3.x**
2021-02-22 13:15:38 -05:00
Artem Bilan
6cad6edf8c Fix Checkstyle violation 2021-02-18 12:43:32 -05:00
Artem Bilan
a2081137f6 Fix new Sonar smells 2021-02-18 12:26:37 -05:00
Artem Bilan
e2e01db9a2 Add SFTP remote copy test-case 2021-02-18 11:31:08 -05:00
Spring Buildmaster
ee3ccc4493 [artifactory-release] Next development version 2021-02-17 23:35:20 +00:00
Spring Buildmaster
a395dd0ac5 [artifactory-release] Release version 5.5.0-M2 2021-02-17 23:35:16 +00:00
Artem Bilan
d1438e8a74 Upgrade dependencies; prepare for release
* Clean up code style in Gradle configs
* Fix JavaDoc errors
* Make JUnit 4 as `optional` dependency for `spring-integration-test-support`
* Remove redundant `javax.activation-api` from some modules
2021-02-17 16:49:52 -05:00
Artem Bilan
b97bedfc88 FileInbound DSL: Add recursive for convenience (#3495)
* FileInbound DSL: Add `recursive` for convenience

Related to https://stackoverflow.com/questions/66171881/how-to-read-nested-txt-file-from-spring-integration-file

The `FileInboundChannelAdapterSpec` can be configured with an external `DirectoryScanner`,
but it sometimes becomes burden for end-users to extract a scanner bean and configure it
with all the required file filters

* Expose a `recursive(boolean)` option for better end-user experience
* Rework `FileTests` for JUnit 5
* Mention a new option in the docs

* * Restore accidentally removed code
* Restore special symbols in the `FileTests`
* Fix language in the docs according review
2021-02-17 14:48:56 -05:00
Artem Bilan
281d8d5bf6 Deprecate Properties as bean for int global props
Having a bean as a `java.util.Properties` is not confusing
and may lead to some conflicts in the real application.
Plus it is not so easy to configure: need to know all
the possible integration properties - bad end-user experience

* Make an `IntegrationProperties` as a public POJO
for easy configuration of global properties
* Deprecate a presence of the `java.util.Properties`
* Leave framework-created `integrationGlobalProperties` as
a `Properties` instance for backward compatibility
* Fix tests to expose an `IntegrationProperties` bean instead
of deprecated `Properties`
* Fix docs according the change and current recommendations
2021-02-17 13:56:03 -05:00