240 lines
14 KiB
XML
240 lines
14 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="ftp"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink">
|
|
<title>FTP/FTPS Adapters</title>
|
|
<para>
|
|
Spring Integration provides support for file transfer operations via FTP and FTPS.
|
|
</para>
|
|
<section id="ftp-intro">
|
|
<title>Introduction</title>
|
|
<para>
|
|
The File Transfer Protocol (FTP) is a simple network protocol which allows you to transfer files between two computers on the Internet.
|
|
</para>
|
|
<para>
|
|
There are two actors when it comes to FTP communication: <emphasis>client</emphasis> and <emphasis>server</emphasis>.
|
|
To transfer files with FTP/FTPS, you use a <emphasis>client</emphasis> which initiates a connection to a remote computer
|
|
that is running an FTP <emphasis>server</emphasis>. After the connection is established, the <emphasis>client</emphasis> can choose
|
|
to send and/or receive copies of files.
|
|
</para>
|
|
|
|
<para>
|
|
Spring Integration supports sending and receiving files over FTP/FTPS by providing two types of <emphasis>client</emphasis>
|
|
side adapters: <emphasis>Inbound Channel Adapter</emphasis> and <emphasis>Outbound Channel Adapter</emphasis>. It also provides
|
|
convenient namespace-based configuration options for defining these <emphasis>client</emphasis> components.
|
|
</para>
|
|
<para>
|
|
To use the <emphasis>FTP</emphasis> namespace, add the following to the header of your XML file:
|
|
<programlisting language="xml"><![CDATA[xmlns:ftp="http://www.springframework.org/schema/integration/ftp"
|
|
xsi:schemaLocation="http://www.springframework.org/schema/integration/ftp
|
|
http://www.springframework.org/schema/integration/ftp/spring-integration-ftp-2.0.xsd"
|
|
]]></programlisting>
|
|
</para>
|
|
</section>
|
|
|
|
|
|
<section id="ftp-session-factory">
|
|
<title>FTP Session Factory</title>
|
|
<para>
|
|
Before configuring FTP adapters you must configure an <emphasis>FTP Session Factory</emphasis>. You can configure
|
|
the <emphasis>FTP Session Factory</emphasis> with a regular bean definition where the implementation class is <classname>org.springframework.integration.ftp.session.DefaultFtpSessionFactory</classname>:
|
|
Below is a basic configuration:
|
|
|
|
<programlisting language="xml"><![CDATA[<bean id="ftpClientFactory"
|
|
class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
|
|
<property name="host" value="localhost"/>
|
|
<property name="port" value="22"/>
|
|
<property name="username" value="kermit"/>
|
|
<property name="password" value="frog"/>
|
|
<property name="clientMode" value="0"/>
|
|
<property name="fileType" value="2"/>
|
|
<property name="bufferSize" value="100000"/>
|
|
</bean>]]></programlisting>
|
|
</para>
|
|
<para>
|
|
For FTPS connections all you need to do is use <classname>org.springframework.integration.ftp.session.DefaultFtpsSessionFactory</classname> instead.
|
|
Below is the complete configuration sample:
|
|
|
|
<programlisting language="xml"><![CDATA[<bean id="ftpClientFactory"
|
|
class="org.springframework.integration.ftp.client.DefaultFtpsClientFactory">
|
|
<property name="host" value="localhost"/>
|
|
<property name="port" value="22"/>
|
|
<property name="username" value="oleg"/>
|
|
<property name="password" value="password"/>
|
|
<property name="clientMode" value="1"/>
|
|
<property name="fileType" value="2"/>
|
|
<property name="useClientMode" value="true"/>
|
|
<property name="cipherSuites" value="a,b.c"/>
|
|
<property name="keyManager" ref="keyManager"/>
|
|
<property name="protocol" value="SSL"/>
|
|
<property name="trustManager" ref="trustManager"/>
|
|
<property name="prot" value="P"/>
|
|
<property name="needClientAuth" value="true"/>
|
|
<property name="authValue" value="oleg"/>
|
|
<property name="sessionCreation" value="true"/>
|
|
<property name="protocols" value="SSL, TLS"/>
|
|
<property name="implicit" value="true"/>
|
|
</bean>]]></programlisting>
|
|
</para>
|
|
|
|
<para>
|
|
Every time an adapter requests a session object from its <interfacename>SessionFactory</interfacename> the session is
|
|
returned from a session pool maintained by a caching wrapper around the factory. A Session in the session pool might go stale
|
|
(if it has been disconnected by the server due to inactivity) so the <interfacename>SessionFactory</interfacename>
|
|
will perform validation to make sure that it never returns a stale session to the adapter. If a stale session was encountered,
|
|
it will be removed from the pool, and a new one will be created.
|
|
<note>
|
|
If you experience connectivity problems and would like to trace Session creation as well as see which Sessions are
|
|
polled you may enable it by setting the logger to TRACE level (e.g., log4j.category.org.springframework.integration.file=TRACE)
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
Now all you need to do is inject these session factories into your adapters. Obviously the protocol (FTP or FTPS) that an adapter will
|
|
use depends on the type of session factory that has been injected into the adapter.
|
|
</para>
|
|
<para>
|
|
<note>
|
|
A more practical way to provide values for <emphasis>FTP/FTPS Session Factories</emphasis> is by using Spring's property
|
|
placeholder support (See: http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/beans.html#beans-factory-placeholderconfigurer).
|
|
</note>
|
|
</para>
|
|
</section>
|
|
|
|
<section id="ftp-inbound">
|
|
<title>FTP Inbound Channel Adapter</title>
|
|
<para>
|
|
The <emphasis>FTP Inbound Channel Adapter</emphasis> is a special listener that will connect to the FTP server and will listen
|
|
for the remote directory events (e.g., new file created) at which point it will initiate a file transfer.
|
|
|
|
<programlisting language="xml"><![CDATA[<int-ftp:inbound-channel-adapter id="ftpInbound"
|
|
channel="ftpChannel"
|
|
session-factory="ftpSessionFactory"
|
|
charset="UTF-8"
|
|
auto-create-local-directory="true"
|
|
delete-remote-files="true"
|
|
filename-pattern="*.txt"
|
|
remote-directory="some/remote/path"
|
|
local-directory=".">
|
|
<int:poller fixed-rate="1000"/>
|
|
</int-ftp:inbound-channel-adapter>]]></programlisting>
|
|
|
|
As you can see from the configuration above you can configure an <emphasis>FTP Inbound Channel Adapter</emphasis> via the <code>inbound-channel-adapter</code>
|
|
element while also providing values for various attributes such as <code>local-directory</code>, <code>filename-pattern</code>
|
|
(which is based on simple pattern matching, not regular expressions), and of course the reference to a <code>session-factory</code>.
|
|
</para>
|
|
<para>
|
|
Some times file filtering based on the simple pattern specified via <code>filename-pattern</code> attribute might not be
|
|
sufficient. If this is the case, you can use the <code>filename-regex</code> attribute to specify a Regular Expression
|
|
(e.g. <code>filename-regex=".*\.test$"</code>). And of course if you need complete control you can use <code>filter</code>
|
|
attribute and provide a reference to any custom implementation of the
|
|
<classname>org.springframework.integration.file.filters.FileListFilter</classname>, a strategy interface for filtering a
|
|
list of files.
|
|
</para>
|
|
<para>
|
|
Please refer to the schema for more details on these attributes.
|
|
</para>
|
|
<para>
|
|
It is also important to understand that the <emphasis>FTP Inbound Channel Adapter</emphasis> is a <emphasis>Polling Consumer</emphasis> and
|
|
therefore you must configure a poller (either via a global default or a local sub-element).
|
|
Once a file has been transferred, a Message with a <classname>java.io.File</classname> as its payload will be generated and sent to the channel
|
|
identified by the <code>channel</code> attribute.
|
|
</para>
|
|
<para>
|
|
<emphasis>More on File Filtering and Large Files</emphasis>
|
|
</para>
|
|
<para>
|
|
Some times the file that just appeared in the monitored (remote) directory is not complete. Typically such a file
|
|
will be written with temporary extension (e.g., foo.txt.writing) and then renamed after the writing process finished.
|
|
As a user in most cases you are only interested in files that are complete and would like to filter only files that are complete.
|
|
To handle these scenarios you can use the filtering support provided by the <code>filename-pattern</code>, <code>filename-regex</code>
|
|
and <code>filter</code> attributes. Here is an example that uses a custom Filter implementation.
|
|
|
|
<programlisting language="xml"><![CDATA[<int-ftp:inbound-channel-adapter
|
|
channel="ftpChannel"
|
|
session-factory="ftpSessionFactory"
|
|
filter="customFilter"
|
|
local-directory="file:/my_transfers">
|
|
remote-directory="some/remote/path"
|
|
<int:poller fixed-rate="1000"/>
|
|
</int-ftp:inbound-channel-adapter>
|
|
|
|
<bean id="customFilter" class="org.example.CustomFilter"/>
|
|
]]></programlisting>
|
|
</para>
|
|
<para>
|
|
<emphasis>Poller configuration notes for the inbound FTP adapter</emphasis>
|
|
</para>
|
|
<para>
|
|
The job of the inbound FTP adapter consists of two tasks:
|
|
<emphasis>1) Communicate with a remote server in order to transfer files from a remote directory to a local directory.</emphasis>
|
|
<emphasis>2) For each transferred file, generate a Message with that file as a payload and send it to the channel identified by the 'channel' attribute.</emphasis>
|
|
|
|
That is why they are called 'channel-adapters' rather than just 'adapters'. The main job of such an adapter is to generate a
|
|
Message to be sent to a Message Channel. Essentially, the second task mentioned above takes precedence in such a way that
|
|
*IF* your local directory already has one or more files it will first generate Messages from those, and *ONLY*
|
|
when all local files have been processed, will it initiate the remote communication to retrieve more files.
|
|
</para>
|
|
<para>
|
|
Also, when configuring a trigger on the poller you should pay close attention to the <code>max-messages-per-poll</code>
|
|
attribute. Its default value is 1 for all <classname>SourcePollingChannelAdapter</classname> instances (including FTP).
|
|
This means that as soon as one file is processed, it will wait for the next execution time as determined by your
|
|
trigger configuration. If you happened to have one or more files sitting in the <code>local-directory</code>, it would process
|
|
those files before it would initiate communication with the remote FTP server. And, if the <code>max-messages-per-poll</code>
|
|
were set to 1 (default), then it would be processing only one file at a time with intervals as defined by your trigger,
|
|
essentially working as <emphasis>one-poll = one-file</emphasis>.
|
|
</para>
|
|
<para>
|
|
For typical file-transfer use cases, you most likely want the opposite behavior: to process all the files you can for each
|
|
poll and only then wait for the next poll. If that is the case, set <code>max-messages-per-poll</code> to -1. Then, on
|
|
each poll, the adapter will attempt to generate as many Messages as it possibly can. In other words, it will process
|
|
everything in the local directory, and then it will connect to the remote directory to transfer everything that is available
|
|
there to be processed locally. Only then is the poll operation considered complete, and the poller will wait for the next execution time.
|
|
</para>
|
|
<para>
|
|
You can alternatively set the 'max-messages-per-poll' value to a positive value indicating the upward limit of Messages to be created
|
|
from files with each poll. For example, a value of 10 means that on each poll it will attempt to process no more than 10 files.
|
|
</para>
|
|
</section>
|
|
|
|
<section id="ftp-outbound">
|
|
<title>FTP Outbound Channel Adapter</title>
|
|
|
|
<para>
|
|
The <emphasis>FTP Outbound Channel Adapter</emphasis> relies upon a <classname>MessageHandler</classname> implementation that will connect to the
|
|
FTP server and initiate an FTP transfer for every file it receives in the payload of incoming Messages. It also supports several
|
|
representations of the <emphasis>File</emphasis> so you are not limited only to java.io.File typed payloads.
|
|
The <emphasis>FTP Outbound Channel Adapter</emphasis>
|
|
supports the following payloads: 1) <classname>java.io.File</classname> - the actual file object;
|
|
2) <classname>byte[]</classname> - a byte array that represents the file contents; and 3) <classname>java.lang.String</classname> -
|
|
text that represents the file contents.
|
|
|
|
<programlisting language="xml"><![CDATA[<int-ftp:outbound-channel-adapter id="ftpOutbound"
|
|
channel="ftpChannel"
|
|
session-factory="ftpSessionFactory"
|
|
charset="UTF-8"
|
|
filename-generator="fileNameGenerator"/>]]></programlisting>
|
|
|
|
As you can see from the configuration above you can configure an <emphasis>FTP Outbound Channel Adapter</emphasis> via the
|
|
<code>outbound-channel-adapter</code> element while also providing values for various attributes such as <code>filename-generator</code>
|
|
(an implementation of the <classname>org.springframework.integration.file.FileNameGenerator</classname> strategy interface),
|
|
a reference to a <code>client-factory</code>, as well as other attributes. Please refer to the schema for more details on
|
|
the available attributes.
|
|
<note>
|
|
By default Spring Integration will use <classname>org.springframework.integration.file.DefaultFileNameGenerator</classname> if none is specified.
|
|
<classname>DefaultFileNameGenerator</classname> will determine the file name based on the value of the <code>file_name</code> header (if it exists)
|
|
in the MessageHeaders, or if the payload of the Message is already a <classname>java.io.File</classname>, then it will use the original name of that file.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
<important>
|
|
Defining certain values (e.g., remote-directory) might be platform/ftp server dependent. For example as it
|
|
was reported on this forum http://forum.springsource.org/showthread.php?p=333478&posted=1#post333478 on some
|
|
platforms you must add slash to the end of the directory definition (e.g., remote-directory="/foo/bar/"
|
|
instead of remote-directory="/foo/bar")
|
|
</important>
|
|
</para>
|
|
|
|
</section>
|
|
</chapter>
|