Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4351: Add DiscardAwareFileListFilter

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4351

The `WatchService` reacts to the events in the file system and keep
track ove the events until we poll them, e.g. via `listEligibleFiles()`
in the `WatchServiceDirectoryScanner`.
On the other hand the `SourcePollingChannelAdapter` calls the mentioned
`listEligibleFiles()` according its polling period.
At this moment the `FileListFilter` is applied to the polled files.
It may happen that `LastModifiedFileListFilter` can't accept too young
files yet and they are lost for the future consideration.

* To allow, for example, to retain young files by the
`LastModifiedFileListFilter` judgment for the future cycles add
`DiscardAwareFileListFilter` with the `DiscardCallback` support.
The `WatchServiceDirectoryScanner` now registers such a callback into
the filter and stores discarded files into the `filesToPoll` queue
for the future poll cycle.

**Cherry-pick to 5.0**

Fix compilation warnings

* Replace `DiscardCallback` with the plain `Consumer`
* Ensure uniqueness in the `WatchServiceDirectoryScanner` internal
queue via a `Set` implementation, since discard callback may be called
several times for the same file from the `CompositeFileListFilter`
according to its nature
* Add JavaDocs and Docs

Change `@since` to `5.0.5`
This commit is contained in:
Artem Bilan
2018-03-29 17:37:24 -04:00
committed by Gary Russell
parent 10805e5b8a
commit a48d54bed9
6 changed files with 135 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
src/reference/asciidoc/file.adoc
View File

@@ -130,6 +130,17 @@ For this purpose all the XML components for file handling (local and remote), al
auto-startup="false"/>
----
Starting with _version 5.0.5_, a `DiscardAwareFileListFilter` is provided for implementations when there is an interest in the event of the rejected files.
For this purpose such a filter implementation should be supplied with a callback via `addDiscardCallback(Consumer<File>)`.
In the Framework this functionality is used from the `FileReadingMessageSource.WatchServiceDirectoryScanner` in combination with `LastModifiedFileListFilter`.
Unlike the regular `DirectoryScanner`, the `WatchService` provides files for processing according the events on the target file system.
At the moment of polling an internal queue with those files, the `LastModifiedFileListFilter` may discard them because they are too young in regards to its configured `age`.
Therefore we lose the file for the future possible considerations.
The discard callback hook allows us to retain the file in the internal queue, so it is available to be checked against the `age` in the subsequent polls.
The `CompositeFileListFilter` also implements a `DiscardAwareFileListFilter` and populates provided discard callback to all its `DiscardAwareFileListFilter` delegates.
NOTE: Since `CompositeFileListFilter` matches the files against all delegates, the `discardCallback` may be called several times for the same file.
*Message Headers*
Starting with _version 5.0_ the `FileReadingMessageSource`, in addition to the `payload` as a polled `File`, populates these headers to the outbound `Message`: