Files
Chris Bono c4b411e6ee Update main to next version (4.0.0) (#314)
* Next version for 4.0 (core)

* Next version for 4.0 (apps)

* Next version for 4.0 (release train)
2022-07-15 18:59:31 -05:00
..

//tag::ref-doc[]
= File Source

This application polls a directory and sends new files or their contents to the output channel.
The file source provides the contents of a File as a byte array by default.
However, this can be customized using the --file.supplier.mode option:

* ref Provides a java.io.File reference

* lines Will split files line-by-line and emit a new message for each line

* contents The default. Provides the contents of a file as a byte array

When using `--file.supplier.mode=lines`, you can also provide the additional option `--file.supplier.withMarkers=true`.
If set to true, the underlying FileSplitter will emit additional start-of-file and end-of-file marker messages before and after the actual data.
The payload of these 2 additional marker messages is of type `FileSplitter.FileMarker`. The option withMarkers defaults to false if not explicitly set.

== Options

The **$$file$$** $$source$$ has the following options:

//tag::configuration-properties[]
Properties grouped by prefix:


=== file.consumer

$$markers-json$$:: $$When 'fileMarkers == true', specify if they should be produced as FileSplitter.FileMarker objects or JSON.$$ *($$Boolean$$, default: `$$true$$`)*
$$mode$$:: $$The FileReadingMode to use for file reading sources. Values are 'ref' - The File object, 'lines' - a message per line, or 'contents' - the contents as bytes.$$ *($$FileReadingMode$$, default: `$$<none>$$`, possible values: `ref`,`lines`,`contents`)*
$$with-markers$$:: $$Set to true to emit start of file/end of file marker messages before/after the data. Only valid with FileReadingMode 'lines'.$$ *($$Boolean$$, default: `$$<none>$$`)*

=== file.supplier

$$delay-when-empty$$:: $$Duration of delay when no new files are detected.$$ *($$Duration$$, default: `$$1s$$`)*
$$directory$$:: $$The directory to poll for new files.$$ *($$File$$, default: `$$<none>$$`)*
$$filename-pattern$$:: $$A simple ant pattern to match files.$$ *($$String$$, default: `$$<none>$$`)*
$$filename-regex$$:: $$A regex pattern to match files.$$ *($$Pattern$$, default: `$$<none>$$`)*
$$prevent-duplicates$$:: $$Set to true to include an AcceptOnceFileListFilter which prevents duplicates.$$ *($$Boolean$$, default: `$$true$$`)*

=== metadata.store.dynamo-db

$$create-delay$$:: $$Delay between create table retries.$$ *($$Integer$$, default: `$$1$$`)*
$$create-retries$$:: $$Retry number for create table request.$$ *($$Integer$$, default: `$$25$$`)*
$$read-capacity$$:: $$Read capacity on the table.$$ *($$Long$$, default: `$$1$$`)*
$$table$$:: $$Table name for metadata.$$ *($$String$$, default: `$$<none>$$`)*
$$time-to-live$$:: $$TTL for table entries.$$ *($$Integer$$, default: `$$<none>$$`)*
$$write-capacity$$:: $$Write capacity on the table.$$ *($$Long$$, default: `$$1$$`)*

=== metadata.store.gemfire

$$region$$:: $$Gemfire region name for metadata.$$ *($$String$$, default: `$$<none>$$`)*

=== metadata.store.jdbc

$$region$$:: $$Unique grouping identifier for messages persisted with this store.$$ *($$String$$, default: `$$DEFAULT$$`)*
$$table-prefix$$:: $$Prefix for the custom table name.$$ *($$String$$, default: `$$<none>$$`)*

=== metadata.store.mongo-db

$$collection$$:: $$MongoDB collection name for metadata.$$ *($$String$$, default: `$$metadataStore$$`)*

=== metadata.store.redis

$$key$$:: $$Redis key for metadata.$$ *($$String$$, default: `$$<none>$$`)*

=== metadata.store

$$type$$:: $$Indicates the type of metadata store to configure (default is 'memory'). You must include the corresponding Spring Integration dependency to use a persistent store.$$ *($$StoreType$$, default: `$$<none>$$`, possible values: `mongodb`,`gemfire`,`redis`,`dynamodb`,`jdbc`,`zookeeper`,`hazelcast`,`memory`)*

=== metadata.store.zookeeper

$$connect-string$$:: $$Zookeeper connect string in form HOST:PORT.$$ *($$String$$, default: `$$127.0.0.1:2181$$`)*
$$encoding$$:: $$Encoding to use when storing data in Zookeeper.$$ *($$Charset$$, default: `$$UTF-8$$`)*
$$retry-interval$$:: $$Retry interval for Zookeeper operations in milliseconds.$$ *($$Integer$$, default: `$$1000$$`)*
$$root$$:: $$Root node - store entries are children of this node.$$ *($$String$$, default: `$$/SpringIntegration-MetadataStore$$`)*
//end::configuration-properties[]

//end::ref-doc[]