INT-3737: (S)FTP Outbound Gateway Streaming GET

JIRA: https://jira.spring.io/browse/INT-3737

Support returning an `InputStream` from a GET operation.

This can be used in conjuction with a `<file:splitter/>` to stream a text file.

The user is responsible for releasing the session when the download is complete.

The session object is stored in the message header and `RemoteFileUtils.closeSession()` should be called
to clean up and close the session.

INT-3737: Polishing and Docs

Fix `CachedSession.close()` bug

Preventing `Caused by: java.io.IOException: Previous raw read was not finalized`
when `Session` is returned to the pool, but `readingRaw` hasn't been finalized
because the real `close()` isn't invoked in case of normal return to pool.

src/reference/asciidoc/ftp.adoc

@@ -328,10 +328,44 @@ _get_ retrieves a remote file and supports the following option:
 
 * -P - preserve the timestamp of the remote file
 
-The message payload resulting from a _get_ operation is a `File` object representing the retrieved file.
+* -stream - retrieve the remote file as a stream.
 
 The remote directory is provided in the `file_remoteDirectory` header, and the filename is provided in the `file_remoteFile` header.
 
+The message payload resulting from a _get_ operation is a `File` object representing the retrieved file, or
+an `InputStream` when the `-stream` option is provided.
+This option allows retrieving the file as a stream.
+For text files, a common use case is to combine this operation with a <<file-splitter>>.
+When consuming remote files as streams, the user is responsible for closing the `Session` after the stream is
+consumed.
+For convenience, the `Session` is provided in the `file_remoteSession` header.
+
+The following shows an example of consuming a file as a stream:
+
+[source, xml]
+----
+<int-ftp:outbound-gateway session-factory="ftpSessionFactory"
+							request-channel="inboundGetStream"
+							command="get"
+							command-options="-stream"
+							expression="payload"
+							remote-directory="ftpTarget"
+							reply-channel="stream" />
+
+<int:chain input-channel="stream">
+	<int-file:splitter markers="true" />
+	<int:payload-type-router resolution-required="false" default-output-channel="output">
+		<int:mapping type="org.springframework.integration.file.splitter.FileSplitter$FileMarker"
+					channel="markers" />
+	</int:payload-type-router>
+</int:chain>
+
+<int:service-activator input-channel="markers"
+	expression="payload.mark.toString().equals('END') ? headers['file_remoteSession'].close() : null"/>
+----
+
+The file lines are sent to the channel `output`.
+
 *mget*
 
 _mget_ retrieves multiple remote files based on a pattern and supports the following option:

src/reference/asciidoc/sftp.adoc

@@ -391,10 +391,44 @@ _get_ retrieves a remote file and supports the following option:
 
 * -P - preserve the timestamp of the remote file
 
-The message payload resulting from a _get_ operation is a `File` object representing the retrieved file.
+* -stream - retrieve the remote file as a stream.
 
 The remote directory is provided in the `file_remoteDirectory` header, and the filename is provided in the `file_remoteFile` header.
 
+The message payload resulting from a _get_ operation is a `File` object representing the retrieved file, or
+an `InputStream` when the `-stream` option is provided.
+This option allows retrieving the file as a stream.
+For text files, a common use case is to combine this operation with a <<file-splitter>>.
+When consuming remote files as streams, the user is responsible for closing the `Session` after the stream is
+consumed.
+For convenience, the `Session` is provided in the `file_remoteSession` header.
+
+The following shows an example of consuming a file as a stream:
+
+[source, xml]
+----
+<int-sftp:outbound-gateway session-factory="sftpSessionFactory"
+              request-channel="inboundGetStream"
+              command="get"
+              command-options="-stream"
+              expression="payload"
+              remote-directory="ftpTarget"
+              reply-channel="stream" />
+
+<int:chain input-channel="stream">
+  <int-file:splitter markers="true" />
+  <int:payload-type-router resolution-required="false" default-output-channel="output">
+    <int:mapping type="org.springframework.integration.file.splitter.FileSplitter$FileMarker"
+           channel="markers" />
+  </int:payload-type-router>
+</int:chain>
+
+<int:service-activator input-channel="markers"
+  expression="payload.mark.toString().equals('END') ? headers['file_remoteSession'].close() : null"/>
+----
+
+The file lines are sent to the channel `output`.
+
 *mget*
 
 _mget_ retrieves multiple remote files based on a pattern and supports the following option:

src/reference/asciidoc/whats-new.adoc

@@ -160,3 +160,9 @@ metadata store if it implements `Flushable` (e.g. the `PropertiesPersistenMetada
 
 When using Java 8, gateway methods can now return `CompletableFuture<?>`.
 See <<gw-completable-future>> for more information.
+
+[[x4.2-ftp-changes]]
+==== (S)FTP Changes
+
+The SFTP/FTP outbound gateway `get` operations now support the `-stream` option, allowing files to be retrieved as
+a stream. See <<ftp-outbound-gateway>> and <<sftp-outbound-gateway>> for more information.