Commit Graph

660 Commits

Author SHA1 Message Date
Artem Bilan
2d9a5f60f4 Introduce a ReceiveMessageAdvice (#3265)
* Introduce a `ReceiveMessageAdvice`

* Deprecate an `AbstractMessageSourceAdvice` in favor of
`default` method in the `MessageSourceMutator`
* Move a `applyReceiveOnlyAdviceChain()` logic into the `AbstractPollingEndpoint`:
now both `PollingConsumer` and `SourcePollingChannelAdapter` can use
`ReceiveMessageAdvice`
* Introduce a `SimpleActiveIdleReceiveMessageAdvice` based already
on the `ReceiveMessageAdvice` and deprecate a `SimpleActiveIdleMessageSourceAdvice`
which is fully replaceable with newly introduced `SimpleActiveIdleReceiveMessageAdvice`
* Add `@SuppressWarnings("deprecation")` for those out-of-the-box `ReceiveMessageAdvice`
implementation which still use an `AbstractMessageSourceAdvice` for
backward compatibility
* Document a new feature and give the `MessageSourceMutator` a new meaning

* * Fix language in the `polling-consumer.adoc`
2020-04-28 13:03:30 -04:00
Artem Bilan
cfd03f89a0 Use a SimpleJsonSerializer in the FileSplitter (#3262)
* Use a SimpleJsonSerializer in the FileSplitter

* To avoid extra dependency for Jackson when we
serialize `FileSplitter.FileMarker` to JSON, use
a `SimpleJsonSerializer` instead.
* Fix `SimpleJsonSerializer` to escape a `\` symbol from property values
since it is used for quoting string values in the final JSON

* * Document the change
2020-04-27 15:42:45 -04:00
Artem Bilan
b06322787e Fix RemoteFileOutboundGWTests for the proper mock 2020-04-18 21:21:08 -04:00
Artem Bilan
5ba1b9321f GH-3249: Fix RemoteFileTemplate dead lock in send
Fixes: https://github.com/spring-projects/spring-integration/issues/3249

When the `CachingSessionFactory` is configured with small enough pool
and it is very likely that dead lock may happen when `RemoteFileTemplate.send()`
is used.
The problem happens when we reach the `RemoteFileTemplate.exists()` call
which is done from the internal method called from already pulled from cache
`Session`

* Fix `RemoteFileTemplate` to use a `session.exists()` instead on the provided
into the method `Session`
* Demonstrate the problem in the `SftpRemoteFileTemplateTests.testNoDeadLockOnSend()`

**Cherry-pick to 5.2.x, 5.1.x & 4.3.x**
2020-04-18 21:06:15 -04:00
Artem Bilan
d24c2c8431 Fix Checkstyle violations & RMI tests
* Increase latch timeouts for `RedisLockRegistryLeaderInitiatorTests`
* Some RMI tests fail with different outcome because an RMI registry might be
available on default `1099` port.
Fix them to not rely on that port
2020-04-07 13:21:46 -04:00
Gary Russell
7d7c273515 GH-3241: MetadataStoreSelector - compare old/new (#3242)
Resolves: https://github.com/spring-projects/spring-integration/issues/3241

 * Docs and XML namespace support.
2020-04-06 18:12:36 -04:00
Artem Bilan
5b03db3f0f Upgrade dependencies; prepare for release
* Fix new Sonar smell
2020-04-01 13:42:14 -04:00
Artem Bilan
6d5690fd58 GH-2996: Add Resource for remote file transfer (#3231)
* GH-2996: Add Resource for remote file transfer

Fixes https://github.com/spring-projects/spring-integration/issues/2996

* Fix `RemoteFileTemplate.send()` to support a `Resource` payload
for remote file transferring content
* Code style clean up for `RemoteFileTemplate`
* Remove `volatile` for configuration properties for better performance
* Change a `charset` to the `Charset` for only once conversion from string
during configuration phase
* Fix (S)FTP tests for new functionality
* Change affected tests to JUnit 5
* Document a new feature; mention all the supported types and `FileExistsMode` constants

* * Fix language in `whats-new.adoc`
2020-03-31 16:09:07 -04:00
Tim Feuerbach
c6cf261e25 GH-3202: Fix tests failing with non-English locales
Fixes https://github.com/spring-projects/spring-integration/issues/3202

Set the locale to en-US for tests that assert error messages
2020-03-02 14:03:27 -05:00
Gary Russell
1ff69d4d24 GH-3175: Add .scanner() to inbound file sync spec
Resolves https://github.com/spring-projects/spring-integration/issues/3175

* Fix `@since` for backport

**Cherry-pick to `5.2.x`**
2020-02-11 12:57:26 -05:00
Artem Bilan
867a8cf108 GH-3155: Add support for Java DSL extensions (#3167)
* GH-3155: Add support for Java DSL extensions

Fixes https://github.com/spring-projects/spring-integration/issues/3155

Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.

* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature

* * Fix language and typos in docs

* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods

* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
2020-02-07 13:40:39 -05:00
Artem Bilan
00b771d8a8 Fix NPE in the DefaultSessionFactoryLocator
**Cherry-pick to 5.2.x**
2020-02-07 10:51:22 -05:00
Andrey Kezhevatov
edf84a393d GH-3169: DSFL: addSessionFactory based on Object
Fixes https://github.com/spring-projects/spring-integration/issues/3169

All other `DefaultSessionFactoryLocator` contracts are based on the `Object`,
so this `addSessionFactor`y has to be on `Object` as well.

* Add `DefaultSessionFactoryLocator.addSessionFactory(Object key, SessionFactory<F> factory)`
* Deprecate existing one based on `String`
* Fix tests do no use a deprecated API
* Some other code style clean up in the affected classes

**Cherry-pick to 5.2.x**
2020-02-07 10:38:07 -05:00
Artem Bilan
370e943428 Remove deprecations from previous versions
* Remove Boon dependency and its usage
* Remove overloaded methods from the `IntegrationFlowDefinition`
- we can simply rely now on the super class
* Remove (or rework) deprecated entities in the docs
* Fix tests for removed deprecated APIs
* Rework affected tests to JUnit 5
2020-01-09 15:54:04 -05:00
Artem Bilan
5ac262f866 GH-3132: Remove usage of super();
Fixes https://github.com/spring-projects/spring-integration/issues/3132

It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation

* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions

* Fix new Sonar smells
2019-12-27 15:13:00 -05:00
Artem Bilan
c0a32622ea Introduce IntegrationPattern abstraction
* Add `IntegrationPattern` contract to implement on the target components
which represent particular EIP
* Add `IntegrationPatternType` with an internal `IntegrationPatternCategory`
to return from the component implementing `IntegrationPattern`
* Parse `IntegrationPatternType` in the `IntegrationNode` for potential
use on the UI for drawing a particular icon

* More pattern representations

* Clean up Checkstyle
* Fix JavaDocs
* Add `integrationPatternCategory` assertion into the `IntegrationGraphServerTests`

* Add more IntegrationPattern implementations
* Provide some delegation and overriding logic whenever we have
components wrapping

* Fix unused imports

* Add `inbound_gateway` pattern indicator
* Add conditional on `expectReply` to indicate a component as an
`IntegrationPatternType.outbound_channel_adapter` or
`IntegrationPatternType.outbound_gateway`
* Make some code clean up in affected classes

* Add a `gateway` type for `@MessagingGateway`

* Comment the reason for `outbound_gateway` type in the `AbstractReplyProducingMessageHandler`
* Bump `IntegrationGraphServer.GRAPH_VERSION`
* Add new attributes into graph sample in the `graph.adoc`
* Document an `IntegrationPattern`

* Apply changes for version 5.3

* Rebased into `5.3-WIP`
* Add a `whats-new.adoc` note about an `IntegrationPattern`
2019-12-26 15:41:19 -05:00
Artem Bilan
9dc3519f20 Start version 5.3
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
 1. Align with SF
 2. The version for XSD doesn't matter for the current jar version,
 since only the version is available in classpath is from the current
 jar version
 3. Modify `spring.schemas` to map all the possible versions for XSD
 into the current one in a jar.
 This way target applications can upgrade without changing the version
 for XSD location and we don't need to require `versionless` variant
 any more
 4. The jar in classpath can handle only its own XSD, so independently
 of version or no-version variants we still may fail because the current
 jar may not support end-user code any more - need some fix over there
 anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
2019-12-26 15:41:19 -05:00
Gary Russell
34c4554adf Fix UUID import in test 2019-12-12 10:30:30 -05:00
Artem Bilan
8e7c1067f3 Fix pub/sub race conditions in Reactive tests 2019-12-02 14:03:12 -05:00
Artem Bilan
0ec9859c0f Refactor some tests into JUnit 5
* Improve their interaction with Reactor Test
2019-11-19 15:28:55 -05:00
Artem Bilan
0f5bd4a40a INT-3494: Resolve dir for writing as a Resource (#3109)
* INT-3494: Resolve dir for writing as a Resource

JIRA: https://jira.spring.io/browse/INT-3494

The expression for local directory can be resolved into a `Resource`
or resource location.
* Fix `ExpressionUtils.expressionToFile()` to support `Resource` and
also use `ResourceUtils.getFile(path)` when expression result is a string
* Modify tests to ensure that resource is resolved properly
* Upgrade affected tests to JUnit 5
* Mention an new functionality in docs

* * Improve Java doc for `ExpressionUtils.expressionToFile()`
* Finish the sentence in the `file.adoc`
2019-11-14 16:27:59 -05:00
lsgeme
1820c05c26 GH-3105: Fix (S)FTP streaming single filter logic
Fixes https://github.com/spring-projects/spring-integration/issues/3105

The loop in the `AbstractRemoteFileStreamingMessageSource` doesn't check
the next polled file for filtering.

* Add `continue;` when we filter the current file and poll the next one.
So, we go over into the `while()` beginning
* Clean up the code style and cover more code path in test
2019-11-08 11:44:42 -05:00
Joaquin Santana
7aea76c6c4 GH-3026: Fix chmod support for DSL
Fixes https://github.com/spring-projects/spring-integration/issues/3026

**Cherry-pick to `5.1.x`**

* Populate proper `FileTransferringMessageHandler` impl from DSL spec
implementations.
This way we are able to use a provided `chmod` from Java DSL
* Added `FileTransferringMessageHandlerSpec` ctor TODO
* Update SftpTests
* Code cleanup; `@Ignore` `SftpTests.testSftpOutboundFlowWithChmod()`
since it doesn't work properly on Windows
2019-10-29 15:20:08 -04:00
Emmanuel Roux
c420c414d3 GH-3076: NioFileLocker: add file existence check
Fixes spring-projects/spring-integration#3076

* Return `null` instead of throwing an exception

* Fix license header

* Add missing `@since`

* Avoid over-engineered test logic

* Fix code style

* Test non-existent file does not get created

* Fix: remove unused import
2019-10-17 14:52:46 -04:00
Gary Russell
9994997917 GH-3067: Use default NullChannel instance if poss.
Resolves https://github.com/spring-projects/spring-integration/issues/3067

Also add missing receive counter.

* Late binding of null discard channel; checkstyle

* Fix test; fall back to new NullChannel(); always evaluate tx expressions
2019-09-27 17:14:51 -04:00
Artem Bilan
075086775d GH-3038: Delegate more RemoteFileTempalte options
Fixes https://github.com/spring-projects/spring-integration/issues/3038

* Add more delegating setters into the `AbstractRemoteFileOutboundGateway`
for its `RemoteFileTemplate` property
* Restrict externally provided `RemoteFileTemplate` from modifications
in the `AbstractRemoteFileOutboundGateway`: those options must be
configured on that external `RemoteFileTemplate`
* Expose new options in the Java DSL specs
2019-08-29 11:20:52 -04:00
Artem Bilan
a756e6334d GH-3043: Add FileHeaders.REMOTE_HOST header (#3044)
* GH-3043: Add FileHeaders.REMOTE_HOST header

Fixes https://github.com/spring-projects/spring-integration/issues/3043

* Populate a `FileHeaders.REMOTE_HOST` from the
`AbstractRemoteFileStreamingMessageSource` and "get"-based commands
in the `AbstractRemoteFileOutboundGateway`
* Extract the value from the a `Session.getHost()` contract
* The `AbstractInboundFileSynchronizingMessageSource` cannot be
addressed with this because the real message is already based on the
locally stored file
* Adjust some affected tests according our code style requirements

* * Add remote file info support into `AbstractInboundFileSynchronizingMessageSource`
* Introduce a `MetadataStore` functionality into the `AbstractInboundFileSynchronizer`
to gather a remote file info an save it in the URI style against local file
* Retrieve such an info in the `AbstractInboundFileSynchronizingMessageSource`
during local file polling
* Introduce `protocol()` contract for the `AbstractInboundFileSynchronizer`
to build a proper URI in the metadata for external readers to distinguish
remote files properly
* Document the feature

* * Fix some typos in Docs

* * Rename property and header constant to the `HOST_PORT` pair
* Fix typos in Docs
* Add  `remote-file-metadata-store` and `metadata-store-prefix` into XSD
of (S)FTP Inbound Channel Adapters
* Add `remoteFileMetadataStore` and `metadataStorePrefix` options
into `RemoteFileInboundChannelAdapterSpec` for Java DSL
2019-08-28 08:58:13 -04:00
Artem Bilan
a835a63aad Apply changes from 5.1 for StandardRotationPolicy
Related to https://github.com/spring-projects/spring-integration/issues/3027

* Remove `Assert.isTrue()` from the `onRotation()` to make the behavior
as void in case of non-standard `MessageSource` provided
* Add JavaDocs into `onRotation()` to describe the behavior and
possibility to override
2019-08-19 13:34:12 -04:00
Artem Bilan
13a8623b59 GH-3027: Decouple RotationPolicy
Fixes https://github.com/spring-projects/spring-integration/issues/3027

This is a polishing for PR https://github.com/spring-projects/spring-integration/pull/3029

* Move `KeyDirectory` in to the `RotationPolicy` since it looks like fully
coupled concept with that abstraction and the class name is so bad for
the public API
* Remove `AbstractStandardRotationPolicy` in favor of a `StandardRotationPolicy`
extendability
* Change `ftp.adoc` and `sftp.adoc` to reflect a new API reality
* Mantion these changes in the `whats-new.adoc`
2019-08-15 11:32:25 -04:00
Gary Russell
a66b5ed6e2 GH-3027: Fix @since 2019-08-14 14:13:28 -04:00
David Turanski
ba017686fb GH-3027: Enhancements to RotatingServerAdvice
Resolves https://github.com/spring-projects/spring-integration/issues/3027

Fix checkstyle errors

Polishing - wrap code at 120, javadocs at 90; don't use this. on method calls
2019-08-14 13:45:11 -04:00
Gary Russell
0edce21cfe Remove remaining redundant superinterfaces 2019-07-31 11:52:22 -04:00
Gary Russell
cd0f56bc87 Add Apache MINA SftpEventListener
- republish certain events as `ApplicationEvent`s.

* * Add ApacheMinaFtplet to provide the same functionality with FTP
* Fix typo

* * Polishing javadocs and event toString() methods
2019-07-30 10:31:23 -04:00
Artem Bilan
80d679a9b0 GH-2748: More bean definitions into exceptions
Fixes https://github.com/spring-projects/spring-integration/issues/2748

* Refactor more `MessageHandlingException`s to include `this` into an
exception message
* Revert using `MessagingException` in some places which really are not
about messaging.
This helps to wrap them into `MessageHandlingException` later in the
`MessageHandler` for the `BeanDefinition` reference
* Remove `volatile` from configuration properties in the affected
classes
* Remove already deprecated `JmsOutboundGateway.setPriority()`
* Add `resource` and `source` for `BeanDefinition` in the
`AbstractChannelAdapterParser` & `AbstractInboundGatewayParser`
* Document the feature
2019-07-23 15:19:56 -04:00
Gary Russell
ce85b64aa8 Improve RemoteFileTemplate invalid payload except.
- include the invalid payload type
2019-07-23 14:11:07 -04:00
Gary Russell
0259d0820c RemoteFileTemplate: Improve test coverage 2019-07-23 13:53:29 -04:00
Gary Russell
d2eee913d7 INT-3746: Polishing
- move `InputStream` test later
- add `name` for error messages
- add test
2019-07-23 13:01:08 -04:00
Gary Russell
8bb0e194c0 Clear lock cache in NioFileLocker (#2998)
Fixes https://github.com/spring-projects/spring-integration/issues/2980
2019-07-22 15:10:18 -04:00
Artem Bilan
c109e1df34 GH-2987: Add HTTPS entries into spring.schemas
Fixes https://github.com/spring-projects/spring-integration/issues/2987

To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file

**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
2019-07-18 14:05:18 -04:00
Artem Bilan
5a1846cfe5 INT-2480: Add aggregate headers strategy
JIRA: https://jira.spring.io/browse/INT-2480

* Introduce `headers-function` option into the `aggregator` for merging
and computing headers for the output message based on the completed
group
* Implement a `DefaultAggregateHeadersFunction` and use it in the
`AbstractAggregatingMessageGroupProcessor` for default behavior with
possible injection for any other implementation
* Add `DelegatingMessageGroupProcessor` to wrap any other
`MessageGroupProcessor` implementations with possible usage of the
`headersFunction` if result is not a `Message` or `MessageBuilder`
* Make `AbstractCorrelatingMessageHandler.getOutputProcessor()` as
`public` rto give access to this option from the `AggregatorSpec` to
be able to inject a `headersFunction` in Java DSL configuration
* Add `AbstractIntegrationMessageBuilder.getHeader()` to get access to
some underlying header avoiding extra `Map` in case of `getHeaders()`
* Change a logic in the `AbstractMessageProducingHandler.produceOutput()`
to consult a `reply` for the `replyChannel` as well `routingSlip` header
if the `reply` is a `Message` or `MessageBuilder`
* Introduce a `AbstractMessageProducingHandler.messageBuilderForReply()`
and use it in `AbstractMessageSplitter` to avoid duplication
* Validate a new functionality in tests
* Fix `FileOutboundGatewayParserTests` to rely on the `TemporaryFolder`
to clean up test files after using

* JavaDocs for `DefaultAggregateHeadersFunction`
* Some `router.adoc` polishing

* Fix link to Reactor in the `router.adoc`

* Add docs for new `Function<MessageGroup, Map<String, Object>>` strategy

* Doc polishing.
2019-07-03 16:32:23 -04:00
Artem Bilan
09c4f03d7c Upgrade to JUnit 5.5, Mockito 2.28.2
* Fix `LongRunningTest` to rely on the `@EnabledIf` from Spring Test
Framework instead of deprecated now in JUnit 5
* Remove unnecessary stubbing in the `FileReadingMessageSourceTest`
2019-07-01 17:04:00 -04:00
Artem Bilan
315f0e711f Fix Sonar vulnerabilities for varargs
* Fix smell for static `AmqpInboundGateway.attributesHolder`

* Fix readOnlyHeaders in the `MessageBuilder`
2019-05-30 15:41:53 -04:00
Gary Russell
86c7e36667 Sonar issues - complexity
* Fix checkstyle issue.
2019-05-03 20:05:16 -04:00
Gary Russell
0d6faa2e34 Sonar - protected fields 2019-05-01 17:28:42 -04:00
Gary Russell
19b9944dd8 Sonar fixes
* use equals
* inner assignment
* local before return
* unnecessary locals
* catch and throw
* uninstantiable with no statics
* implement `Serializable` in `Comparator` to enable `TreeMap` serialization
* exceptions as flow control
2019-04-30 18:47:43 -04:00
Gary Russell
130e1bba31 Synchronize RemoteFileUtils.makeDirectories()
https://stackoverflow.com/questions/55918648

Avoid the race with concurrent threads.

**cherry-pick to 5.x.x, 4.3.x**
2019-04-30 14:09:45 -04:00
Gary Russell
0bb901b286 Sonar fixes
- printStackTrace in test mail server
- Illegal throws
- Large anon. classes

* - indexOf char
- stored external object

* - ignored exceptional return values

* - checkstyle
2019-04-30 13:57:12 -04:00
Gary Russell
36c33bb5ab Sonar fixes
- a few complexities
- final method calls from ctor
- raw exception throwing
- useless overrides
- loss of stack trace
2019-04-29 18:20:40 -04:00
Artem Bilan
818be4cbe8 GH-2765: Add discardChannel for splitter (#2883)
* GH-2765: Add discardChannel for splitter

Fixes https://github.com/spring-projects/spring-integration/issues/2765

When encountering empty collections, splitter should be able to send
the result to a discard channel.
Currently, when encountering an empty collection,
the splitter ends the flow.
Some use-cases may rely on a custom split function which may returns
empty collections.
These use-cases should be able to define a discard channel
so they can proceed with a possible compensation flow.

* Add `discardChannel` option to the `AbstractMessageSplitter`
* Delegate `discardChannel` population from everywhere it is possible:
DSL, XML, `AbstractMessageSplitter` extension like `FileSplitter` etc.
* Fix `FileSplitterTests` for broken charset
* Document new feature; fix some typos and out-dated code sample

* * Fix `SplitterFactoryBean` for NPE on the `discardChannelName`
propagation

* * Check `this.discardChannel` first
* `Assert.state()` in `doInit()` for mutual exclusiveness
2019-04-08 16:42:57 -04:00
Artem Bilan
872302de6e Add -Dfile.encoding=UTF-8 into gradle.properties
To avoid wrong encoding during reading and writing files by the
`updateCopyrights` Gradle task, it is better to have an explicit
`-Dfile.encoding=UTF-8` before starting JVM.

* Fix `AbstractFilePayloadTransformerTests` for non-standard symbols in
the test data
* fix broken test
`FileToStringTransformerTests.transform_withWrongEncoding_notMatching()`
2019-03-22 14:13:20 -04:00