INT-4149: Improve (S)FTP Recursive MGET

JIRA: https://jira.spring.io/browse/INT-4149

Add an option to always pass directories while recursing.

* Fix typos in docs
* Revert `SyslogReceivingChannelAdapterTests` `Thread.sleep()` fixes

src/reference/asciidoc/ftp.adoc

@@ -790,6 +790,8 @@ By default, the entire remote tree is retrieved.
 However, files in the tree can be filtered, by providing a`FileListFilter`; directories in the tree can also be filtered this way.
 A `FileListFilter` can be provided by reference or by `filename-pattern` or `filename-regex` attributes.
 For example, `filename-regex="(subDir|.*1.txt)"` will retrieve all files ending with `1.txt` in the remote directory and the subdirectory `subDir`.
+However, see below for an alternative available in _version 5.0_.
+
 If a subdirectory is filtered, no additional traversal of that subdirectory is performed.
 
 The `-dirs` option is not allowed (the recursive mget uses the recursive `ls` to obtain the directory tree and the directories themselves cannot be included in the list).
@@ -797,6 +799,25 @@ The `-dirs` option is not allowed (the recursive mget uses the recursive `ls` to
 Typically, you would use the `#remoteDirectory` variable in the `local-directory-expression` so that the remote directory structure is retained locally.
 =====
 
+Starting with _version 5.0_, the `FtpSimplePatternFileListFilter` and `FtpRegexPatternFileListFilter` can be configured to always pass directories by setting the `alwaysAcceptDirectorties` to `true`.
+This allows recursion for a simple pattern; examples follow:
+
+[code, xml]
+----
+<bean id="startDotTxtFilter" class="org.springframework.integration.ftp.filters.FtpSimplePatternFileListFilter">
+    <constructor-arg value="*.txt" />
+    <property name="alwaysAcceptDirectories" value="true" />
+</bean>
+
+<bean id="dotStarDotTxtFilter"
+            class="org.springframework.integration.ftp.filters.FtpRegexPatternFileListFilter">
+    <constructor-arg value="^.*\.txt$" />
+    <property name="alwaysAcceptDirectories" value="true" />
+</bean>
+----
+
+and provide one of these filters using `filter` property on the gateway.
+
 See also <<ftp-partial>>.
 
 *put*

src/reference/asciidoc/sftp.adoc

@@ -879,6 +879,8 @@ By default, the entire remote tree is retrieved.
 However, files in the tree can be filtered, by providing a`FileListFilter`; directories in the tree can also be filtered this way.
 A `FileListFilter` can be provided by reference or by `filename-pattern` or `filename-regex` attributes.
 For example, `filename-regex="(subDir|.*1.txt)"` will retrieve all files ending with `1.txt` in the remote directory and the subdirectory `subDir`.
+However, see below for an alternative available in _version 5.0_.
+
 If a subdirectory is filtered, no additional traversal of that subdirectory is performed.
 
 The `-dirs` option is not allowed (the recursive mget uses the recursive `ls` to obtain the directory tree and the directories themselves cannot be included in the list).
@@ -886,6 +888,26 @@ The `-dirs` option is not allowed (the recursive mget uses the recursive `ls` to
 Typically, you would use the `#remoteDirectory` variable in the `local-directory-expression` so that the remote directory structure is retained locally.
 =====
 
+Starting with _version 5.0_, the `SftpSimplePatternFileListFilter` and `SftpRegexPatternFileListFilter` can be configured to always pass directories by setting the `alwaysAcceptDirectorties` to `true`.
+This allows recursion for a simple pattern; examples follow:
+
+[code, xml]
+----
+<bean id="startDotTxtFilter" class="org.springframework.integration.sftp.filters.SftpSimplePatternFileListFilter">
+    <constructor-arg value="*.txt" />
+    <property name="alwaysAcceptDirectories" value="true" />
+</bean>
+
+<bean id="dotStarDotTxtFilter"
+            class="org.springframework.integration.sftp.filters.SftpRegexPatternFileListFilter">
+    <constructor-arg value="^.*\.txt$" />
+    <property name="alwaysAcceptDirectories" value="true" />
+</bean>
+----
+
+and provide one of these filters using `filter` property on the gateway.
+
+
 See also <<sftp-partial>>
 
 *put*

src/reference/asciidoc/whats-new.adoc

@@ -46,6 +46,10 @@ See <<file-reading>> for more information.
 
 The inbound channel adapters now have a property `max-fetch-size` which is used to limit the number of files fetched during a poll when there are no files currently in the local directory.
 
+The regex and pattern filters can now be configured to always pass directories.
+This can be useful when using recursion in the outbound gateways.
+See <<ftp-outbound-gateway>> and <<sftp-outbound-gateway>> for more information.
+
 ==== Integration Properties
 
 Since _version 4.3.2_ a new `spring.integration.readOnly.headers` global property has been added to customize the list of headers which should not be copied to a newly created `Message` by the `MessageBuilder`.