Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4049: FileSplitter: JSON File Markers

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4049

INT-4049: Json FileMarker Namespace Support

* Address PR comments
This commit is contained in:
Gary Russell
2016-06-08 13:03:09 -04:00
committed by Artem Bilan
parent c620438913
commit d521031a7d
11 changed files with 172 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
src/reference/asciidoc/file.adoc
View File

@@ -799,15 +799,16 @@ Other payload types will be emitted unchanged.
<int-file:splitter id="splitter" <1>
iterator="" <2>
markers="" <3>
apply-sequence="" <4>
requires-reply="" <5>
charset="" <6>
input-channel="" <7>
output-channel="" <8>
send-timeout="" <9>
auto-startup="" <10>
order="" <11>
phase="" /> <12>
markers-json="" <4>
apply-sequence="" <5>
requires-reply="" <6>
charset="" <7>
input-channel="" <8>
output-channel="" <9>
send-timeout="" <10>
auto-startup="" <11>
order="" <12>
phase="" /> <13>
----
<1> The bean name of the splitter.
@@ -818,33 +819,38 @@ Other payload types will be emitted unchanged.
Markers are messages with `FileSplitter.FileMarker` payloads (with `START` and `END` values in the `mark` property).
Markers might be used when sequentially processing files in a downstream flow where some lines are filtered.
They enable the downstream processing to know when a file has been completely processed.
In addition, a header `file_marker` containing `START` or `END` are added to these messages.
The 'END' marker includes a line count.
Default: `false`.
When `true`, `apply-sequence` is `false` by default.
Also see `markers-json`.
<4> Set to `false` to disable the inclusion of `sequenceSize` and `sequenceNumber` headers in messages.
<4> When `markers` is true, set this to `true` and the `FileMarker` objects will be converted to a JSON String.
Requires a supported JSON processor library on the classpath (Jackson, Boon).
<5> Set to `false` to disable the inclusion of `sequenceSize` and `sequenceNumber` headers in messages.
Default: `true`, unless `markers` is `true`.
When `true` and `markers` is `true`, the markers are included in the sequencing.
When `true` and `iterator` is `true`, the `sequenceSize` header is set to `0` because the size is unknown.
<5> Set to `true` to cause a `RequiresReplyException` to be thrown if there are no lines in the file.
<6> Set to `true` to cause a `RequiresReplyException` to be thrown if there are no lines in the file.
Default: `false`.
<6> Set the charset name to be used when reading the text data into `String` payloads.
<7> Set the charset name to be used when reading the text data into `String` payloads.
Default: platform charset.
<7> Set the input channel used to send messages to the splitter.
<8> Set the input channel used to send messages to the splitter.
<8> Set the output channel to which messages will be sent.
<9> Set the output channel to which messages will be sent.
<9> Set the send timeout - only applies if the `output-channel` can block - such as a full `QueueChannel`.
<10> Set the send timeout - only applies if the `output-channel` can block - such as a full `QueueChannel`.
<10> Set to `false` to disable automatically starting the splitter when the context is refreshed.
<11> Set to `false` to disable automatically starting the splitter when the context is refreshed.
Default: `true`.
<11> Set the order of this endpoint if the `input-channel` is a `<publish-subscribe-channel/>`.
<12> Set the order of this endpoint if the `input-channel` is a `<publish-subscribe-channel/>`.
<12> Set the startup phase for the splitter (used when `auto-startup` is `true`).
<13> Set the startup phase for the splitter (used when `auto-startup` is `true`).
*Java Configuration*
@@ -866,3 +872,12 @@ using the `stream` option to retrieve a file, starting with _version 4.3_, the s
the session supporting the stream, when the file is completely consumed.
See <<ftp-streaming>> and <<sftp-streaming>> as well as <<ftp-outbound-gateway>> and <<sftp-outbound-gateway>> for more
information about these facilities.
When using Java configuration, an additional constructor is available:
[source, java]
----
public FileSplitter(boolean iterator, boolean markers, boolean markersJson)
----
When `markersJson` is true, the markers will be represented as a JSON string, as long as a suitable JSON processor library, such as Jackson or Boon, is on the classpath.

6
src/reference/asciidoc/whats-new.adoc
View File

@@ -149,14 +149,14 @@ See <<file-flushing>> for more information.
===== Preserving Timestamps
The outbound channel adapter can now be configured to set the destination file's lastmodified timestamp.
The outbound channel adapter can now be configured to set the destination file's `lastmodified` timestamp.
See <<file-timestamps>> for more information.
===== Splitter Changes
The `FileSplitter` will now automatically close an (S)FTP session when the file is completely read.
This applies when the outbound gateway returns an `InputStream` or the new (S)FTP streaming channel adapters are being
used.
This applies when the outbound gateway returns an `InputStream` or the new (S)FTP streaming channel adapters are being used.
Also a new `markers-json` options has been introduced to convert `FileSplitter.FileMarker` to JSON `String` for relaxed downstream network interaction.
See <<file-splitter>> for more information.
==== AMQP Changes