INT-3989: WatchServiceDirectoryScanner Improvement

JIRA: https://jira.spring.io/browse/INT-3989,
https://jira.spring.io/browse/INT-3990,
https://jira.spring.io/browse/INT-3988

INT-3989: Add `FileReadingMessageSource.WatchServiceDirectoryScanner`

* Deprecate top-level `WatchServiceDirectoryScanner` because of inconsistency around `Lifecycle` and shared `directory` property
* Copy/paste its logic into the `FileReadingMessageSource.WatchServiceDirectoryScanner` to hide that inconsistency, but still get a gain from the `WatchService` benefits
* Add support for the `StandardWatchEventKinds.ENTRY_MODIFY` and `StandardWatchEventKinds.ENTRY_DELETE` events in the `FileReadingMessageSource.WatchServiceDirectoryScanner`
* Introduce `useWatchService` option to switch to the internal `FileReadingMessageSource.WatchServiceDirectoryScanner`
* Make `CompositeFileListFilter` also as `ResettableFileListFilter`
* Deprecate weird `FileReadingMessageSource.onSend()` method and remove its usage from tests
* Modify `WatchServiceDirectoryScannerTests` for the new logic
* Document changes

Add `MODIFY` and `DELETE` test coverage

Optimize the `filesFromEvents()` logic replacing item with the fresh event sources.
Remove the item from the result set in case of `DELETE` event, because file removal generates both `MODIFY` and `DELETE` events.

* Add `FileReadingMessageSource.setWatchEvents` to allow to listen to the specific events,
not only `CREATE` or all of them.
* Modify tests and docs to reflect the new API

Address PR comments

* Improve `FileReadingMessageSource.setWatchEvents()`
* Add `file.exists()` before adding file from event.

Add more DEBUG logs into the `WatchServiceDirectoryScanner`

With the fact of those logs provide more optimizations:
* Don't register the same directory for watching:
  - use the `ConcurrentMap<Path, WatchKey> pathKeys` to track registrations
  - any modification within the directory causes the `ENTRY_MODIFY` for the directory as well.
    So, skip such an event exactly for the directory during `walkDirectory`
* Add debug logs in case of `ENTRY_DELETE`

src/reference/asciidoc/file.adoc

@@ -231,11 +231,12 @@ on that `scanner` not on the `FileReadingMessageSource`.
 NOTE: The `DefaultDirectoryScanner` uses a `IgnoreHiddenFileListFilter` and `AcceptOnceFileListFilter` by default.
 To prevent their use, you should configure your own filter (e.g. `AcceptAllFileListFilter`) or even set it to `null`.
 
-
+[[watch-service-directory-scanner]]
 ==== WatchServiceDirectoryScanner
 
 This scanner was added in _version 4.2_. It replaces the existing `RecursiveLeafOnlyDirectoryScanner` which is
-inefficient for large directory trees. The `WatchServiceDirectoryScanner` requires Java 7 or above.
+inefficient for large directory trees.
+The `FileReadingMessageSource.WatchServiceDirectoryScanner` requires Java 7 or above.
 
 This scanner relies on file system events when new files are added to the directory.
 During initialization, the directory is registered to generate events; the initial file list is also built.
@@ -253,19 +254,38 @@ In this case, the root directory is re-scanned completely.
 To avoid duplicates consider using an appropriate `FileListFilter` such as the `AcceptOnceFileListFilter` and/or
 remove files when processing is completed.
 
-[source, xml]
-----
-<bean id="wsScanner" class="org.springframework.integration.file.WatchServiceDirectoryScanner">
-    <constructor-arg value="/tmp/myDir" />
-</bean>
-----
+Since _version 4.3_, the top level `WatchServiceDirectoryScanner` has been deprecated in favor of
+`FileReadingMessageSource` internal logic for the `WatchService`.
+Now this can be enable via `use-watch-service` option, which is mutually exclusive with the `scanner` option.
+An internal `FileReadingMessageSource.WatchServiceDirectoryScanner` instance is populated for the provided `directory`.
 
-[source, java]
+In addition, now the `WatchService` polling logic can track the `StandardWatchEventKinds.ENTRY_MODIFY` and
+`StandardWatchEventKinds.ENTRY_DELETE`, too.
+
+The `ENTRY_MODIFY` events logic should be implemented properly in the `FileListFilter` to track not only new files but
+also the modification, if that is requirement.
+Otherwise the files from those events are treated the same way.
+
+The `ENTRY_DELETE` events have effect for the `ResettableFileListFilter` implementations and, therefore, their files
+ are provided for the `remove()` operation.
+
+For this purpose the `watch-events`
+(`FileReadingMessageSource.setWatchEvents(FileReadingMessageSource.WatchEventType... watchEvents)`) has been introduced.
+With such an option we can implement some scenarios, when we would like to do one downstream flow logic for new files,
+and other for modified.
+We can achieve that with different `<int-file:inbound-channel-adapter>` definitions, but for the same directory:
+
+[source,xml]
 ----
-@Bean
-public DirectoryScanner scanner() {
-    return new WatchServiceDirectoryScanner("/tmp/myDir");
-}
+<int-file:inbound-channel-adapter id="newFiles"
+     directory="${input.directory}"
+     use-watch-service="true"/>
+
+<int-file:inbound-channel-adapter id="modifiedFiles"
+     directory="${input.directory}"
+     use-watch-service="true"
+     filter="acceptAllFilter"
+     watch-events="MODIFY"/> <!-- CREATE by default -->
 ----
 
 ==== Limiting Memory Consumption

src/reference/asciidoc/whats-new.adoc

@@ -94,6 +94,11 @@ The generated file name for the `FileWritingMessageHandler` can represent _sub-p
 structure for file in the target directory.
 See <<file-writing-file-names>> for more information.
 
+The `FileReadingMessageSource` now hides the `WatchService` directory scanning logic in the inner class.
+The `use-watch-service` and `watch-events` options are provided to enable such a behaviour.
+The top level `WatchServiceDirectoryScanner` has been deprecated because of inconsistency around API.
+See <<watch-service-directory-scanner>> for more information.
+
 ===== Buffer Size
 
 When writing files, you can now specify the buffer size to use.