Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

GH-2253: Revert (S)FTP Source to DefaultDirScann

Browse Source 
Fixes spring-projects/spring-integration#2253

For backward compatibility and use-cases when processed files are stored
in sub-directories in the `localDirectory` revert the
`AbstractInboundFileSynchronizingMessageSource` logic back to the
`DefaultDirectoryScanner`.
The `RecursiveDirectoryScanner` can be injected via `scanner` option
This commit is contained in:
Artem Bilan
2017-10-03 17:08:17 -04:00
committed by Gary Russell
parent 1fa73a9728
commit bf63ef5491
5 changed files with 12 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
spring-integration-file/src/main/java/org/springframework/integration/file/remote/synchronizer/AbstractInboundFileSynchronizingMessageSource.java
View File

@@ -26,9 +26,9 @@ import java.util.regex.Pattern;
import org.springframework.beans.factory.BeanInitializationException;
import org.springframework.context.Lifecycle;
import org.springframework.integration.endpoint.AbstractFetchLimitingMessageSource;
import org.springframework.integration.file.DefaultDirectoryScanner;
import org.springframework.integration.file.DirectoryScanner;
import org.springframework.integration.file.FileReadingMessageSource;
import org.springframework.integration.file.RecursiveDirectoryScanner;
import org.springframework.integration.file.filters.AcceptOnceFileListFilter;
import org.springframework.integration.file.filters.CompositeFileListFilter;
import org.springframework.integration.file.filters.FileListFilter;
@@ -190,7 +190,7 @@ public abstract class AbstractInboundFileSynchronizingMessageSource<F>
this.fileSource.getScanner().setFilter(filter);
}
else if (!this.fileSource.isUseWatchService()) {
RecursiveDirectoryScanner directoryScanner = new RecursiveDirectoryScanner();
DirectoryScanner directoryScanner = new DefaultDirectoryScanner();
directoryScanner.setFilter(filter);
this.fileSource.setScanner(directoryScanner);
}

4
spring-integration-ftp/src/test/java/org/springframework/integration/ftp/inbound/FtpInboundRemoteFileSystemSynchronizerTests.java
View File

@@ -49,6 +49,7 @@ import org.springframework.expression.ExpressionParser;
import org.springframework.expression.spel.SpelParserConfiguration;
import org.springframework.expression.spel.standard.SpelExpressionParser;
import org.springframework.integration.file.FileHeaders;
import org.springframework.integration.file.RecursiveDirectoryScanner;
import org.springframework.integration.file.filters.AcceptOnceFileListFilter;
import org.springframework.integration.file.filters.CompositeFileListFilter;
import org.springframework.integration.file.filters.FileListFilter;
@@ -111,13 +112,14 @@ public class FtpInboundRemoteFileSystemSynchronizerTests {
FtpInboundFileSynchronizingMessageSource ms = new FtpInboundFileSynchronizingMessageSource(synchronizer);
ms.setAutoCreateLocalDirectory(true);
ms.setLocalDirectory(localDirectory);
ms.setBeanFactory(mock(BeanFactory.class));
CompositeFileListFilter<File> localFileListFilter = new CompositeFileListFilter<File>();
localFileListFilter.addFilter(new RegexPatternFileListFilter(".*\\.TEST\\.a$"));
AcceptOnceFileListFilter<File> localAcceptOnceFilter = new AcceptOnceFileListFilter<File>();
localFileListFilter.addFilter(localAcceptOnceFilter);
RecursiveDirectoryScanner scanner = new RecursiveDirectoryScanner();
ms.setScanner(scanner);
ms.setLocalFilter(localFileListFilter);
ms.afterPropertiesSet();
ms.start();

3
src/reference/asciidoc/ftp.adoc
View File

@@ -410,7 +410,8 @@ This will work for any `ResettableFileListFilter`.
Starting with _version 5.0_, the Inbound Channel Adapter can build sub-directories locally according the generated local file name.
That can be a remote sub-path as well.
To be able to read local directory recursively for modification according the hierarchy support, an internal `FileReadingMessageSource` now have been switched to use a new `RecursiveDirectoryScanner` based on the `Files.walk()` algorithm.
To be able to read local directory recursively for modification according the hierarchy support, an internal `FileReadingMessageSource` now can be supplied with a new `RecursiveDirectoryScanner` based on the `Files.walk()` algorithm.
See `AbstractInboundFileSynchronizingMessageSource.setScanner()` for more information.
Also the `AbstractInboundFileSynchronizingMessageSource` can now be switched to the `WatchService` -based `DirectoryScanner` via `setUseWatchService()` option.
It is also configured for all the `WatchEventType` s to react for any modifications in local directory.
The reprocessing sample above is based on the build-in functionality of the `FileReadingMessageSource.WatchServiceDirectoryScanner` to perform `ResettableFileListFilter.remove()` when the file is deleted (`StandardWatchEventKinds.ENTRY_DELETE`) from the local directory.

3
src/reference/asciidoc/sftp.adoc
View File

@@ -455,7 +455,8 @@ This will work for any `ResettableFileListFilter`.
Starting with _version 5.0_, the Inbound Channel Adapter can build sub-directories locally according the generated local file name.
That can be a remote sub-path as well.
To be able to read local directory recursively for modification according the hierarchy support, an internal `FileReadingMessageSource` now have been switched to use a new `RecursiveDirectoryScanner` based on the `Files.walk()` algorithm.
To be able to read local directory recursively for modification according the hierarchy support, an internal `FileReadingMessageSource` now can bve supplied with a new `RecursiveDirectoryScanner` based on the `Files.walk()` algorithm.
See `AbstractInboundFileSynchronizingMessageSource.setScanner()` for more information.
Also the `AbstractInboundFileSynchronizingMessageSource` can now be switched to the `WatchService` -based `DirectoryScanner` via `setUseWatchService()` option.
It is also configured for all the `WatchEventType` s to react for any modifications in local directory.
The reprocessing sample above is based on the build-in functionality of the `FileReadingMessageSource.WatchServiceDirectoryScanner` to perform `ResettableFileListFilter.remove()` when the file is deleted (`StandardWatchEventKinds.ENTRY_DELETE`) from the local directory.

4
src/reference/asciidoc/whats-new.adoc
View File

@@ -174,6 +174,7 @@ See <<files>> for more information.
The Inbound Channel Adapters now have a property `max-fetch-size` which is used to limit the number of files fetched during a poll when there are no files currently in the local directory.
They also are configured with a `FileSystemPersistentAcceptOnceFileListFilter` in the `local-filter` by default.
You can also provide a custom `DirectoryScanner` implementation to Inbound Channel Adapters via the newly introduced `scanner` attribute.
The regex and pattern filters can now be configured to always pass directories.
@@ -187,7 +188,8 @@ The (S)FTP streaming inbound channel adapters now add remote file information in
The FTP and SFTP outbound channel adapters, as well as `PUT` command of the outbound gateways, now support `InputStream` as `payload`, too.
The inbound channel adapters now can build file tree locally and use a new `RecursiveDirectoryScanner` by default for local directory.
The inbound channel adapters now can build file tree locally using a newly introduced `RecursiveDirectoryScanner`.
See `scanner` option for injection.
Also these adapters can now be switched to the `WatchService` instead.
The `NLST` command has been added to the `AbstractRemoteFileOutboundGateway` to perform only list files names remote command.