Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4140: File Streaming Adapter: Add maxFetchSize

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4140
This commit is contained in:
Gary Russell
2016-10-18 10:32:23 -04:00
parent 3c284871c3
commit 5a44216201
11 changed files with 92 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/reference/asciidoc/ftp.adoc
View File

@@ -446,6 +446,7 @@ See <<file-splitter>> and <<stream-transformer>> for more information about thes
filter="filter"
remote-file-separator="/"
comparator="comparator"
max-fetch-size="1"
remote-directory-expression="'foo/bar'">
<int:poller fixed-rate="1000" />
</int-ftp:inbound-streaming-channel-adapter>
@@ -459,6 +460,8 @@ file being processed again, you can configure an `FtpPersistentFileListFilter` i
If you don't actually want to persist the state, an in-memory `SimpleMetadataStore` can be used with the filter.
If you wish to use a filename pattern (or regex) as well, use a `CompositeFileListFilter`.
Use the `max-fetch-size` attribute to limit the number of files fetched on each poll when a fetch is necessary; set to 1 and use a persistent filter when running in a clustered environment;
==== Configuring with Java Configuration
The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration:
@@ -480,6 +483,7 @@ public class FtpJavaApplication {
messageSource.setRemoteDirectory("ftpSource/");
messageSource.setFilter(new FtpPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(),
"streaming"));
messageSource.setMaxFetchSize(1);
return messageSource;
}

4
src/reference/asciidoc/sftp.adoc
View File

@@ -555,6 +555,7 @@ See <<file-splitter>> and <<stream-transformer>> for more information about thes
filter="filter"
remote-file-separator="/"
comparator="comparator"
max-fetch-size="1"
remote-directory-expression="'foo/bar'">
<int:poller fixed-rate="1000" />
</int-sftp:inbound-streaming-channel-adapter>
@@ -568,6 +569,8 @@ file being processed again, you can configure an `SftpPersistentFileListFilter`
If you don't actually want to persist the state, an in-memory `SimpleMetadataStore` can be used with the filter.
If you wish to use a filename pattern (or regex) as well, use a `CompositeFileListFilter`.
Use the `max-fetch-size` attribute to limit the number of files fetched on each poll when a fetch is necessary; set to 1 and use a persistent filter when running in a clustered environment;
==== Configuring with Java Configuration
The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration:
@@ -589,6 +592,7 @@ public class SftpJavaApplication {
messageSource.setRemoteDirectory("sftpSource/");
messageSource.setFilter(new SftpPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(),
"streaming"));
messageSource.setMaxFetchSize(1);
return messageSource;
}