Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.
* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
* Some file module Sonar fixes
* * Fixes after testing
* * Some further Sonar fixes for file module
* * Further Sonar fixes for file module
* * More Sonar fixes for file module
* * Fix `RemoteFileTemplate` issues
* * Fix new sonar smells after previous one
* * Review feedback
https://build.spring.io/browse/INT-FATS5IC-685/
Even if we stop `SourcePollingChannelAdapter`, the task on the fly is
still going to be executed.
This way we may have extra calls to our logic.
* Use `CopyOnWriteArrayList` for the `sessionSources` to avoid
`ConcurrentModificationException` during this collection read in the
tests
**Cherry-pick to 5.0.x**
* Add Reactive mode for AbstractPollingEndpoint
* When `SourcePollingChannelAdapter.outputChannel` is a
`ReactiveStreamsSubscribableChannel`, use `Flux.generate()` for polling
* Refactor `AbstractPollingEndpoint` to remove redundant `Poller` class
in favor of lambda
* Extract `pollForMessage()` method to handle TX states instead of
`Poller` class previously
* * Rebase and fix conflicts
* Polishing for GatewayProxyFactoryBeanTests
https://build.spring.io/browse/INT-MASTERSPRING40-458/
We need to stop Inbound Channel Adapter first and only then wait for
the latch.
Otherwise there is a chance that new item is added to the collection
in between `latch.await()` and `stop()`
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4513
The `IntegrationManagementConfigurer` keeps metrics beans in its local
stores.
When we add beans at runtime, they are not removed from those caches
after their removal
* Implement `DestructionAwareBeanPostProcessor` in the `IntegrationManagementConfigurer`
and clean up caches according provided bean type and its name
* Improve `ManualFlowTests` and `FtpTests` to be sure that
`IntegrationManagementConfigurer` caches are cleared after destroying
`IntegrationFlowRegistration`
**Cherry-pick to 5.0.x**
* INT-4381: MessageSources refactoring
JIRA: https://jira.spring.io/browse/INT-4381
* Make all the out-of-the-box `MessageSource` implementations based
on the `AbstractMessageSource`
* Fix `JdbcPollingChannelAdapterIntegrationTests` for sporadic failure
because of `fixed-rate` for the poller
* Fix HeaderEnricherTests race condition
The `errorChannel()` expect an error in the `testErrorChannel` after
`1000` ms, but at the same time the `poller` in configured for the
`3000` ms.
* Increase all the timeouts for replies
* Decrease a `fixed-delay` on the `poller`
* Some other code style polishing for the `HeaderEnricherTests`
Windows is too scrupulous for non closed file resources.
Our `RemoteFileTestSupport` recreates a test directory with files
for each test.
When we don't close some file handler, we are not able to delete the
directory and we fail with subsequent tests.
* Close `InputStream` for each polled file in new tests in the
`SftpStreamingMessageSourceTests`
* Close `CLOSEABLE_RESOURCE` in new tests in the
`FtpStreamingMessageSourceTests`
* Rework `FtpStreamingMessageSourceTests.testAllContents()` do not
poll all the files on each polling cycle - this causes a race condition
when we don't close `CLOSEABLE_RESOURCE` yet in the `StreamTransformer`,
but try to proceed with recreation a test directory structure
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4495
Previously, `maxFetch` was applied before directories were removed from
the fetch list.
Remove the directories before filtering and applying `maxFetch`.
* Polishing - PR Comments
* More polishing
* More polishing.
* Check for empty array.
* Remove test main method.
On Windows `File.delete()` causes to file to be modified first of all.
with the high polling interval (`fixedDelay(1)`) there is a chance
to pick the same file from the local directory again, since the
`AbstractPersistentAcceptOnceFileListFilter` is based on the `lastModified`
* Do not delete files manually at all - just rely on the `clean()`
with the `@Before`
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4390
Phase I: flow context/registration cycle
- extract interfaces
- rename implementations to `Standard...`
Phase II - dsl<->dsl.channel tangles
Move the channel specs to dsl.
Fix missing refactorings
(Not related to DSL) - rename core `event` package to `events`
- avoid collision with event module package
- fix tangle caused by `MessageGroupExpiredEvent`.
Phase III - dsl<->config.dsl
- move classes from config.dsl to dsl
(Not related to DSL) - fix many tangles caused by graph being a sub-package of support.management
- make `graph` a top-level package
(Not related to DSL) - move `IntegrationManagementConfigurer` from
`management` to `config` - tangle between core and management
* Polishing Copyrights, diamonds, some JavaDocs and What's New
JIRA: https://jira.spring.io/browse/INT-4407
To make a `Log4j2LevelAdjuster` more builder friendly, add a `merge`
option to the overloaded `classes()` and `categories()` methods.
This way the existing (or previously configured) `categories` and
`classes` are merged with the provided values and default
`org.springframework.integration` category can be reuse.
Note: this behavior can be reconsidered as default one in the `5.1`
* INT-4376: Upgage to Log4J 2
JIRA: https://jira.spring.io/browse/INT-4376
* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2
* Polishing after testing
JIRA: https://jira.spring.io/browse/INT-4344
The `FtpOutboundGateway` provides ctors without expression for remote path.
In this case it is treated as a `working directory` but only for the
`LS`, `NLST`, `PUT` and `MPUT` commands.
* Add assertion in the `AbstractRemoteFileOutboundGateway` to discard
configuration for all other commands when `expression` is `null`
**Cherry-pick to 4.3.x**
Fixesspring-projects/spring-integration#2253
For backward compatibility and use-cases when processed files are stored
in sub-directories in the `localDirectory` revert the
`AbstractInboundFileSynchronizingMessageSource` logic back to the
`DefaultDirectoryScanner`.
The `RecursiveDirectoryScanner` can be injected via `scanner` option
JIRA: https://jira.springsource.org/browse/INT-4334
* Add a `DirectoryScanner` setter named `setScanner` in
`AbstractInboundFileSynchronizingMessageSource` to set the
`FileReadingMessageSource`'s scanner.
* Add tests to `AbstractRemoteFileSynchronizerTests`.
Polish.
Updates xsd, tests and documentation.
Fix xsd formatting.
Addresses review comments.
* Some code style polishing
JIRA: https://jira.spring.io/browse/INT-4305
The `File.renameTo()` operation may fail, therefore the content of the
local file isn't changed, but since we change `setLastModified()` anyway,
this file might be eligible for local polling.
So, we end up with the same content from local file in a new message,
meanwhile we expect a new content from the remote file
* Check the `File.renameTo()` result and attempt to `delete()`
for existing local file
* When file isn't renames remove the remote file from the `filter` to let
it be transferred one more time on the next poll.
The local file might be opened for processing, so this way we postpone a fresh
remote file for the future poll rounds
* Modify `copyFileToLocalDirectory()` to return `boolean` to reflect the fact of copy.
This way we check the real number of transferred files
**Cherry-pick to 4.3.x**
Do not transfer remote file if we can't remove local one
Polishing Log Messages
JIRA: https://jira.spring.io/browse/INT-4303
Don't emit ignored files in the output message payload.
Also, always preserve the timestamp when using `FileExistsMode.REPLACE_IF_MODIFIED`.
Some code style polishing
JIRA: https://jira.spring.io/browse/INT-4283
Filter that only passes files where a corresponding "marker" file is also present.
Implementations for file and(S)FTP.
* Polishing - PR Comments
* Missed one
JIRA: https://jira.spring.io/browse/INT-4060
* Add `NLST` command to the `AbstractRemoteFileOutboundGateway` to perform
`listNames` on the target session.
Useful in case of server doesn't allow to perform `LS` or the names set is
sufficient for application requirements
* Add `workingDirExpression` to the `FtpOutboundGateway` to allow to perform
`FtpClient.changeWorkingDirectory()` based on the current request message
* Change `slf4j-log4j12` to the `testCompile` -
the FTP tests fail in the IDE with `ClassNotFoundException`
Address PR comments
* Add `nlst` to XSD config
* Reinstate `ls -1` test-case for the `FtpServerOutboundTests`
* wrap more commands to the `doInWorkingDirectory()`
Fix `MV` command in the `AbstractRemoteFileOutboundGateway`
* Implement `RemoteFileOperations#invoke(OperationsCallback<F, T>)`
for thread-bound `session`s
* Use a new `invoke()` for `put()` and `mPut()` commands in the `AbstractRemoteFileOutboundGateway`
* Add delegation for the `put()` and `mPut()` commands in the `FtpOutboundGateway`
* Add DSL support for the `workingDirExpression` and add `whats-new.adoc` note
Document changes
Address some PR comments:
* Add `invokeScope` variable to `execute` to track `ThreadLocal` session or not
* Check for `null` in the `getSession()` and fallback to regular
`sessionFactory.getSession()`.
Most likely the `invoke()` is called from other thread
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4086
The remote files may be carried with sub-path, not just file name.
The same sub-dirs logic can be achieve with the `localFilenameGeneratorExpression`
* Add the logic into `AbstractInboundFileSynchronizer` to build sub-dirs for local files
* Switch on `WatchService` for the internal `FileReadingMessageSource` in the `AbstractInboundFileSynchronizingMessageSource` to react for all the changes in the `localDirectory`
Introduce `RecursiveDirectoryScanner` and use it for `AbstractInboundFileSynchronizingMessageSource` by default
* Add `maxDepth` and `fileVisitOptions` options to the `RecursiveDirectoryScanner`
* Polishing JavaDocs, tests and docs
JIRA: https://jira.spring.io/browse/INT-4115
Apply `(S)FtpPersistentAcceptOnceFileListFilter` for the `(S)FtpInboundFileSynchronizer` by default to avoid cases to sync the same remote files to the local directory again.
Especially when `localFileName` strategy is applied and we end up with new local files, but with the same remote content
Accept `(S)FtpPersistentAcceptOnceFileListFilter` for streaming adapters
Make `doSetFilter()` as `protected final`
Fix tests after rebase
Fix "What's New" after rebase
Compose `PersistentAcceptOnceFileListFilter` together with the regex or pattern filters
Document such a behavior
Address PR comments for formatting and typo
JIRA: https://jira.spring.io/browse/INT-4018
* Resolve some TODOs as expected
* Remove some as logically or physically with useless effort
* Leave others for future considerations
JIRA: https://jira.spring.io/browse/INT-4219
- Use `AdditionalAnswers.returnFirstArgument` instead of lambda to return first arg
- Enable CheckStyle `AvoidStaticImport` for tests
- Fix static import violations in tests
JIRA: https://jira.spring.io/browse/INT-4232
Currently to implement the synchronization logic when remote file is modified there is only one way: remove local file and provide a `localFilter` which can be clear from the removed file as well.
The Reference Manual claims that `FileSystemPersistentAcceptOnceFileListFilter` can let us pick up a fresh version of the remote file if we configure `preserveTimestamp = true`, but actually `AbstractInboundFileSynchronizer` just bypass that logic because it check a local file for existence.
* Modify `AbstractInboundFileSynchronizer` to compare `lastModified` of the remote file with local version if `preserveTimestamp == true`
* Also replace `AcceptOnceFileListFilter` to the `FileSystemPersistentAcceptOnceFileListFilter` since the first one doesn't care about `lastModified`.
If `preserveTimestamp` isn't in use the `FileSystemPersistentAcceptOnceFileListFilter` works fully similar to the `AcceptOnceFileListFilter`
**Cherry-pick to 4.3.x except test-case**
Make `SftpStreamingMessageSourceTests` compatible with Windows
Looks like SFTP embedded server provides different permissions for files on different OSs
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4233
INT-4233: Add (S)FTP FileInfo Header (Streaming)
JIRA: https://jira.spring.io/browse/INT-4233
Add the complete file info as JSON (when Jackson or Boon available) to the message headers
when streaming inbound.
Provide a mechanism to configure Boon to provide similar output to Jackson.
Also allow subclasses to provide their own object mapper.
Also clean up after the AMQP DSL tests, and don't use a queue `foo`. I often have such a queue
with content; this caused tests to fail.
Use SimpleJsonSerializer
Doc Polishing
Polishing - PR Comments
More Polishing
* Polishing according latest PR comments
