<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="sftp"
    xmlns:xlink="http://www.w3.org/1999/xlink">
  <title>SFTP Adapters</title>
  <para>
    Spring Integration provides support for file transfer operations via SFTP.
  </para>
  <section id="sftp-intro">
    <title>Introduction</title>
    <para>
      The Secure File Transfer Protocol (SFTP) is a network protocol which allows you to transfer 
      files between two computers on the Internet over any reliable stream.
    </para>
    
    <para>
    The SFTP protocol requires a secure channel, such as SSH, as well as visibility to a client's identity throughout the SFTP session.
    </para>
    
    <para>
    Spring Integration supports sending and receiving files over SFTP by providing three <emphasis>client</emphasis>
    side endpoints:
    <emphasis>Inbound Channel Adapter</emphasis>, <emphasis>Outbound Channel Adapter</emphasis>, and <emphasis>Outbound Gateway</emphasis>
    It also provides convenient
    namespace configuration to define these <emphasis>client</emphasis> components.
    
    <programlisting language="xml"><![CDATA[xmlns:int-sftp="http://www.springframework.org/schema/integration/sftp"
xsi:schemaLocation="http://www.springframework.org/schema/integration/sftp 
	http://www.springframework.org/schema/integration/sftp/spring-integration-sftp.xsd"
]]></programlisting>
    </para>
  </section>
  
   <section id="sftp-session-factory">
  	<title>SFTP Session Factory</title>
  	<para>
  	Before configuring SFTP adapters you must configure an <emphasis>SFTP Session Factory</emphasis>. You can configure 
  	the <emphasis>SFTP Session Factory</emphasis> via a regular bean definition:
	Below is a basic configuration:
	
	<programlisting language="xml"><![CDATA[<beans:bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
		<beans:property name="host" value="loclahost"/>
		<beans:property name="privateKey" value="classpath:META-INF/keys/sftpTest"/>
		<beans:property name="privateKeyPassphrase" value="springIntegration"/>
		<beans:property name="port" value="22"/>
		<beans:property name="user" value="kermit"/>
</beans:bean>]]></programlisting>
	
	<para>
	Every time an adapter requests a session object from its <interfacename>SessionFactory</interfacename> the session is 
	returned from a session pool maintained by a caching wrapper around the factory. A Session in the session pool might go stale 
	(if it has been disconnected by the server due to inactivity) so the <interfacename>SessionFactory</interfacename> 
	will perform validation to make sure that it never returns a stale session to the adapter. If a stale session was encountered,
	it will be removed from the pool, and a new one will be created.
	<note>
	If you experience connectivity problems and would like to trace Session creation as well as see which Sessions are 
	polled you may enable it by setting the logger to TRACE level (e.g., log4j.category.org.springframework.integration.file=TRACE)
	</note>
	</para>

	Now all you need to do is inject this <emphasis>SFTP Session Factory</emphasis> into your adapters.
  	</para>
  	<para>
  	<note>
  	A more practical way to provide values for the <emphasis>SFTP Session Factory</emphasis> would be via Spring's property 
  	placeholder support (http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/beans.html#beans-factory-placeholderconfigurer)	
  	</note>
  </para>
  </section>
  
   <section id="sftp-inbound">
  	<title>SFTP Inbound Channel Adapter</title>
  	<para>
  	The <emphasis>SFTP Inbound Channel Adapter</emphasis> is a special listener that will connect to the server and listen for 
  	the remote directory events (e.g., new file created) at which point it will initiate a file transfer.
  	
  	<programlisting language="xml"><![CDATA[<int-sftp:inbound-channel-adapter id="sftpAdapterAutoCreate"
  			session-factory="sftpSessionFactory"
			channel="requestChannel"
			filename-pattern="*.txt"
			remote-directory="/foo/bar"
			local-directory="file:target/foo"
			auto-create-local-directory="true"
			local-filename-generator-expression="#this.toUpperCase() + '.a'"
			delete-remote-files="false">
		<int:poller fixed-rate="1000"/>
</int-sftp:inbound-channel-adapter>]]></programlisting>

	As you can see from the configuration above you can configure the <emphasis>SFTP Inbound Channel Adapter</emphasis> via the
	<code>inbound-channel-adapter</code> element while also providing values for various attributes such as <code>local-directory</code> 
	- where files are going to be transferred TO and <code>remote-directory</code> - the remote source directory where files are
	going to be transferred FROM -
	as well as other attributes including a <code>session-factory</code> reference to the bean we configured earlier. 
	</para>
	<para>
	By default the transferred file will carry the same name as the original file. If you want to override this behavior you 
	can set the <code>local-filename-generator-expression</code> attribute which allows you to provide a SpEL Expression to generate 
	the name of the local file. Unlike outbound gateways and adapters where the root object of the SpEL Evaluation Context 
	is a <classname>Message</classname>, this inbound adapter does not yet have the Message at the time of evaluation since
	that's what it ultimately generates with the transferred file as its payload. So, the root object of the SpEL Evaluation Context
	is the original name of the remote file (String).
	</para>
  	<para>
	Some times file filtering based on the simple pattern specified via <code>filename-pattern</code> attribute might not be 
	sufficient. If this is the case, you can use the <code>filename-regex</code> attribute to specify a Regular Expression 
	(e.g. <code>filename-regex=".*\.test$"</code>). And of course if you need complete control you can use the <code>filter</code> 
	attribute to provide a reference to a custom implementation of the 
	<classname>org.springframework.integration.file.filters.FileListFilter</classname> - a strategy interface for filtering a 
	list of files.
  	</para>
  	<para>
  	Please refer to the schema for more detail on these attributes. 
  	</para>
	
  	<para>
  	It is also important to understand that <emphasis>SFTP Inbound Channel Adapter</emphasis> is a Polling Consumer and therefore 
  	you must configure a poller (either a global default or a local sub-element).
	Once the file has been transferred to a local directory, a Message with <classname>java.io.File</classname> as its 
	payload type will be generated and sent to the channel identified by the <code>channel</code> attribute.
  	</para>
  	<para>
  		<emphasis>More on File Filtering and Large Files</emphasis>
  	</para>
  	 <para>
  	Some times a file that just appeared in the monitored (remote) directory is not complete. Typically such a file 
  	will be written with some temporary extension (e.g., foo.txt.writing) and then renamed after the writing process completes. 
  	As a user in most cases you are only interested in files that are complete and would like to filter only those files. 
  	To handle these scenarios, use filtering support provided via the <code>filename-pattern</code>, <code>filename-regex</code> 
  	and <code>filter</code> attributes.

	If you need a custom filter implementation simply include a reference in your adapter via the <code>filter</code> attribute.
	
	<programlisting language="xml"><![CDATA[<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
			channel="receiveChannel" 
			session-factory="sftpSessionFactory"
			filter="customFilter"
			local-directory="file:/local-test-dir"
			remote-directory="/remote-test-dir">
		<int:poller fixed-rate="1000" max-messages-per-poll="10" task-executor="executor"/>
</int-sftp:inbound-channel-adapter>
	
<bean id="customFilter" class="org.foo.CustomFilter"/>

]]></programlisting>
  	</para>
  </section>
  
  <section id="sftp-outbound">
  	<title>SFTP Outbound Channel Adapter</title>
  	
  	<para>
  	The <emphasis>SFTP Outbound Channel Adapter</emphasis>is a special <classname>MessageHandler</classname> that will connect to the 
  	remote directory and will initiate a file transfer for every file it will receive as the payload of an incoming <classname>Message</classname>. 
  	It also supports several representations of the File so you are not limited to the File object. 
  	Similar to the FTP outbound adapter, the <emphasis>SFTP Outbound Channel Adapter</emphasis> supports the following payloads: 
  	1) <classname>java.io.File</classname> - the actual file object; 2) <classname>byte[]</classname> - byte array that represents 
  	the file contents; 3) <classname>java.lang.String</classname> - text that represents the file contents.
  	
  	<programlisting language="xml"><![CDATA[<int-sftp:outbound-channel-adapter id="sftpOutboundAdapter"
				session-factory="sftpSessionFactory"
				channel="inputChannel"
				charset="UTF-8"
				remote-directory="foo/bar"
				remote-filename-generator-expression="payload.getName() + '-foo'"/>]]></programlisting>
			
	As you can see from the configuration above you can configure the <emphasis>SFTP Outbound Channel Adapter</emphasis> via 
	the <code>outbound-channel-adapter</code> element. 
	Please refer to the schema for more detail on these attributes. 
  	</para>
  	<para>
  		<emphasis>SpEL and the SFTP Outbound Adapter</emphasis>
  	</para>
  	<para>
  		As with many other components in Spring Integration, you can benefit from the Spring Expression Language (SpEL) support when configuring 
  		an <emphasis>SFTP Outbound Channel Adapter</emphasis>, by specifying two attributes <code>remote-directory-expression</code> and 
  		<code>remote-filename-generator-expression</code> (see above). The expression evaluation context will have the Message as its root object, thus allowing 
  		you to provide expressions which can dynamically compute the <emphasis>file name</emphasis> or the existing <emphasis>directory path</emphasis> 
  		based on the data in the Message (either from 'payload' or 'headers'). In the example above we are defining
        the <code>remote-filename-generator-expression</code> attribute with an expression 
  		value that computes the <emphasis>file name</emphasis> based on its original name while also appending a suffix: '-foo'.
  	</para>
  </section>

  <section id="sftp-outbound-gateway">
	<title>SFTP Outbound Gateway</title>

	<para>
	  The <emphasis>SFTP Outbound Gateway</emphasis> provides a limited set of commands to interact with a remote SFTP server.
	  <para>
	    Commands supported are:
	    <itemizedlist>
		  <listitem>ls (list files)</listitem>
		  <listitem>get (retrieve file)</listitem>
		  <listitem>mget (retrieve file(s))</listitem>
		  <listitem>rm (remove file(s))</listitem>
	    </itemizedlist>
	  </para>
	  <para><emphasis>ls</emphasis></para>
	  <para>
	    ls lists remote file(s) and supports the following options:
	    <itemizedlist>
		  <listitem>-1 - just retrieve a list of filenames, default is to retrieve a
		    list of <classname>FileInfo</classname> objects.</listitem>
		  <listitem>-a - include all files (including those starting with '.')</listitem>
		  <listitem>-f - do not sort the list</listitem>
		  <listitem>-dirs - include directories (excluded by default)</listitem>
		  <listitem>-links - include symbolic links (excluded by default)</listitem>
	    </itemizedlist>
	  </para>
	  <para>
	    In addition, filename filtering is provided, in the same manner as the
	    <classname>inbound-channel-adapter</classname>.
	  </para>
	  <para>
	    The message payload resulting from an <emphasis>ls</emphasis> operation is a list of file names,
	    or a list of <classname>FileInfo</classname> objects. These objects provide
	    information such as modified time, permissions etc.
	  </para>
	  <para>
	    The remote directory that the <emphasis>ls</emphasis> command acted on is provided
	    in the <classname>file_remoteDirectory</classname> header.
	  </para>
	  <para><emphasis>get</emphasis></para>
	  <para>
	    <emphasis>get</emphasis> retrieves a remote file and supports the following option:
	    <itemizedlist>
		  <listitem>-P - preserve the timestamp of the remote file</listitem>
	    </itemizedlist>
	  </para>
	  <para>
		The message payload resulting from a <emphasis>get</emphasis> operation is a
		<classname>File</classname>	object representing the retrieved file.
	  </para>
	  <para>
	    The remote directory is provided in the <classname>file_remoteDirectory</classname> header, and the filename is
	    provided in the <classname>file_remoteFile</classname> header.
	  </para>
	  <para><emphasis>mget</emphasis></para>
	  <para>
	    <emphasis>mget</emphasis> retrieves multiple remote files based on a pattern and supports the following option:
	    <itemizedlist>
		  <listitem>-x - Throw an exception if no files match the pattern (otherwise an empty
		  list is returned)</listitem>
	    </itemizedlist>
	  </para>
	  <para>
		The message payload resulting from an <emphasis>mget</emphasis> operation is a
		<classname>List&lt;File&gt;</classname>	object - a List of File objects, each representing
		a retrieved file.
	  </para>
	  <para>
	    The remote directory is provided in the <classname>file_remoteDirectory</classname> header, and the pattern
	    for the filenames is
	    provided in the <classname>file_remoteFile</classname> header.
	  </para>
	  <para><emphasis>rm</emphasis></para>
	  <para>
	    The <emphasis>rm</emphasis> command has no options.
	  </para>
	  <para>
		The message payload resulting from an <emphasis>rm</emphasis> operation is Boolean.TRUE if the
		remove was successful, Boolean.FALSE otherwise.
	    The remote directory is provided in the <classname>file_remoteDirectory</classname> header, and the filename is
	    provided in the <classname>file_remoteFile</classname> header.
	  </para>
	  <para>
	    In each case, the PATH that these commands act on is provided by the 'expression'
	    property of the gateway. For the mget command, the expression might evaluate to '*', meaning
	    retrieve all files, or 'somedirectory/*' etc.
	  </para>
	</para>
	<para>
	  Here is an example of a gateway configured for an ls command...
	  <programlisting language="xml"><![CDATA[<int-ftp:outbound-gateway id="gateway1"
		session-factory="ftpSessionFactory"
		request-channel="inbound1"
		command="ls"
		command-options="-1"
		expression="payload"
		reply-channel="toSplitter"/>
]]></programlisting>
	</para>
	<para>
	  The payload of the message sent to the toSplitter channel is a list of String objects
	  containing the filename of each file. If the <classname>command-options</classname> was
	  omitted, it would be a list of <classname>FileInfo</classname> objects. Options are
	  provided space-delimited, e.g. <classname>command-options="-1 -dirs -links"</classname>.
	</para>

  </section>

  <section id="sftp-jsch-logging">
  	<title>SFTP/JSCH Logging</title>
  	<para>
  	Since we use JSch libraries (http://www.jcraft.com/jsch/) to provide SFTP support, at times you may require
  	 more information from the JSch API itself, especially if something is not working properly 
  	 (e.g., Authentication exceptions). Unfortunately JSch does not use commons-logging but instead 
  	 relies on custom implementations of their <classname>com.jcraft.jsch.Logger</classname> interface. 
  	 As of Spring Integration 2.0.1, we have implemented this interface. So, now all you need to do to enable 
  	 JSch logging is to configure your logger the way you usually do. For example, here is valid configuration of a 
  	 logger using Log4J.
  	 
  	 <programlisting language="java"><![CDATA[log4j.category.com.jcraft.jsch=DEBUG]]></programlisting>
  	
  	</para>
  </section>
  <section id="sftp-session-caching">
  	<title>SFTP Session Caching</title>
	<para>
		As of version 2.1 we've exposed more flexibility with regard to session management for remote file adapters (e.g., FTP, SFTP etc).
	    In previous versions the sessions were cached automatically by default. We did expose a <code>cache-sessions</code> attribute for
	    disabling the auto caching, but that solution did not provide a way to configure other session caching attributes. For example, one
	    of the requested features was to support a limit on the number of sessions created since a remote server may impose a limit on the
	    number of client connections. To support that requirement and other configuration options, we decided to promote explicit definition
	    of the <classname>CachingSessionFactory</classname> instance. That provides the <code>sessionCacheSize</code> and <code>sessionWaitTimeout</code>
	    properties. As its name suggests, the  <code>sessionCacheSize</code> property controls how many active sessions this adapter will
	    maintain in its cache (the DEFAULT is unbounded). If the <code>sessionCacheSize</code> threshold has been reached, any attempt to
	    acquire another session will block until either one of the cached sessions becomes available or until the wait time for a Session 
	    expires (the DEFAULT wait time is Integer.MAX_VALUE). The <code>sessionWaitTimeout</code> property enables configuration of that value.
    </para>
    <para>
    If you want your Sessions to be cached, simply configure your default Session Factory as described above and then
    wrap it in an instance of <classname>CachingSessionFactory</classname> where you may provide those additional properties.

    <programlisting language="xml"><![CDATA[<bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
		<property name="host" value="localhost"/>
	</bean>
	
	<bean id="cachingSessionFactory" class="org.springframework.integration.file.remote.session.CachingSessionFactory">
		<constructor-arg ref="sftpSessionFactory"/>
		<property name="sessionCacheSize" value="10"/>
		<property name="sessionWaitTimeout" value="1000"/>
	</bean>]]></programlisting>
	
	In the above example you see a <classname>CachingSessionFactory</classname> created with the
	<code>sessionCacheSize</code> set to 10 and the <code>sessionWaitTimeout</code> set to 1 second (its value is in millliseconds).

  	</para>
  </section>
</chapter>
