<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="files"
    xmlns:xlink="http://www.w3.org/1999/xlink">
  <title>File Support</title>

  <section id="file-intro">
    <title>Introduction</title>
    <para>
      Spring Integration's File support extends the Spring Integration Core with
      a dedicated vocabulary to deal with reading, writing, and transforming files.
      It provides a namespace that enables elements defining Channel Adapters dedicated
      to files and support for Transformers that can read file contents into strings or
      byte arrays.
      </para>
    <para>
      This section will explain the workings of <classname>FileReadingMessageSource</classname>
      and <classname>FileWritingMessageHandler</classname> and how to configure them as
      <emphasis>beans</emphasis>. Also the support for dealing with files through file specific
      implementations of <interfacename>Transformer</interfacename> will be discussed. Finally the
      file specific namespace will be explained.
    </para>
  </section>

  <section id="file-reading">
    <title>Reading Files</title>
    <para>
      A <classname>FileReadingMessageSource</classname> can be used to consume files from the filesystem.
      This is an implementation of <interfacename>MessageSource</interfacename> that creates messages from
      a file system directory. <programlisting language="xml"><![CDATA[<bean id="pollableFileSource"
      class="org.springframework.integration.file.FileReadingMessageSource"
        p:inputDirectory="${input.directory}"/>]]></programlisting>
    </para>
    <para>
      To prevent creating messages for certain files, you may supply a
      <interfacename>FileListFilter</interfacename>. By default, an
      <classname>AcceptOnceFileListFilter</classname> is used. This filter
      ensures files are picked up only once from the directory.
      <programlisting language="xml"><![CDATA[<bean id="pollableFileSource"
      class="org.springframework.integration.file.FileReadingMessageSource"
        p:inputDirectory="${input.directory}"
        p:filter-ref="customFilterBean"/>]]></programlisting>
    </para>
    <para>
      A common problem with reading files is that a file may be detected before
      it is ready. The default <classname>AcceptOnceFileListFilter</classname>
      does not prevent this. In most cases, this can be prevented if the
      file-writing process renames each file as soon as it is ready for
      reading. A filename-pattern or filename-regex filter that accepts only files that are
      ready (e.g. based on a known suffix), composed with the default
      <classname>AcceptOnceFileListFilter</classname> allows for this.
      The <classname>CompositeFileListFilter</classname> enables the
      composition.
      <programlisting language="xml"><![CDATA[<bean id="pollableFileSource"
      class="org.springframework.integration.file.FileReadingMessageSource"
        p:inputDirectory="${input.directory}"
        p:filter-ref="compositeFilter"/>
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
    <constructor-arg>
        <list>
            <bean class="org.springframework.integration.file.filters.AcceptOnceFileListFilter" />
            <bean class="org.springframework.integration.file.filters.RegexPatternFileListFilter">
                <constructor-arg value="^test.*$"/>
            </bean>
        </list>
    </constructor-arg>
</bean>]]></programlisting>
    </para>
    <para>
      The configuration can be simplified using the file specific namespace. To do
      this use the following template.
      <programlisting language="xml"><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:integration="http://www.springframework.org/schema/integration"
  xmlns:file="http://www.springframework.org/schema/integration/file"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
      http://www.springframework.org/schema/integration
      http://www.springframework.org/schema/integration/spring-integration-2.0.xsd
      http://www.springframework.org/schema/integration/file
      http://www.springframework.org/schema/integration/file/spring-integration-file-2.0.xsd">
</beans>]]></programlisting>
      Within this namespace you can reduce the FileReadingMessageSource and wrap
      it in an inbound Channel Adapter like this:
      <programlisting language="xml"><![CDATA[ <file:inbound-channel-adapter id="filesIn1"
       directory="file:${input.directory}" prevent-duplicates="true"/>

 <file:inbound-channel-adapter id="filesIn2"
        directory="file:${input.directory}"
        filter="customFilterBean" />

 <file:inbound-channel-adapter id="filesIn3"
        directory="file:${input.directory}"
        filename-pattern="test*" />

 <file:inbound-channel-adapter id="filesIn4"
	        directory="file:${input.directory}"
	        filename-regex="test[0-9]+\.txt" />              ]]></programlisting>
      The first channel adapter is relying on the default filter that just prevents
      duplication, the second is using a custom filter, the third is using the
      <emphasis>filename-pattern</emphasis> attribute to add an <classname>AntPathMatcher</classname>
      based filter, and the fourth is using the <emphasis>filename-regex</emphasis> attribute to add a
      regular expression Pattern based filter to the <classname>FileReadingMessageSource</classname>.
      The <emphasis>filename-pattern</emphasis> and <emphasis>filename-regex</emphasis> attributes are
      each mutually exclusive with the regular <emphasis>filter</emphasis> reference attribute. However,
      you can use the <emphasis>filter</emphasis> attribute to reference an instance of
      <classname>CompositeFileListFilter</classname> that combines any number of filters, including one
      or more pattern based filters to fit your particular needs.
    </para>
        <para>
            When multiple processes are reading from the same directory it can be desirable to lock files to prevent
            them from being picked up concurrently. To do this you can use a <interfacename>FileLocker</interfacename>.
            There is a java.nio based implementation available out of the box, but it is also possible to implement your
            own locking scheme. The nio locker can be injected as follows
            <programlisting><![CDATA[ <file:inbound-channel-adapter id="filesIn"
       directory="file:${input.directory}" prevent-duplicates="true">
    <file:nio-locker/>
  </file:inbound-channel-adapter>]]>
            </programlisting>
            A custom locker you can configure like this:
             <programlisting><![CDATA[ <file:inbound-channel-adapter id="filesIn"
       directory="file:${input.directory}" prevent-duplicates="true">
    <file:locker ref="customLocker"/>
  </file:inbound-channel-adapter>]]>
            </programlisting>
        </para>
      <note>
          <para>
              When a file inbound adapter is configured with a locker, it will take the responsibility to acquire a
              lock before the file is allowed to be received.
              <strong>It will not assume the responsibility to unlock the file.</strong>
              If you have processed the file and keeping the locks hanging around you have a memory leak. If this is
              a problem in your case you should call FileLocker.unlock(File file) yourself at the appropriate time.
          </para>
      </note>
        <para>
            When filtering and locking files is not enough it might be needed to control the way files are listed entirely. To
            implement this type of requirement you can use an implementation of <interfacename>DirectoryScanner</interfacename>.
            This scanner allows you to determine entirely what files are listed each poll. This is also the interface
            that Spring Integration uses internally to wire FileListFilters FileLocker to the FileReadingMessageSource.
            A custom DirectoryScanner can be injected into the &lt;file:inbound-channel-adapter/&gt; on the <code>scanner</code>
            attribute.
            <programlisting><![CDATA[ <file:inbound-channel-adapter id="filesIn"
       directory="file:${input.directory}" prevent-duplicates="true" scanner="customDirectoryScanner"/>]]>
            </programlisting>
            This gives you full freedom to choose the ordering, listing and locking strategies.
        </para>
  </section>

  <section id="file-writing">
    <title>Writing files</title>
    <para>
      To write messages to the file system you can use a
      <classname>FileWritingMessageHandler</classname>. This class can deal with
      File, String, or byte array payloads. In its simplest form the
      <classname>FileWritingMessageHandler </classname> only requires a
      destination directory for writing the files. The name of the file to be
      written is determined by the handler's <classname>FileNameGenerator</classname>.
      The default implementation looks for a Message header whose key matches
      the constant defined as <code>FileHeaders.FILENAME</code>.
    </para>
    <para>
      Additionally, you can configure the encoding and the charset that
      will be used in case of a String payload.
    </para>
    <para>
      To make things easier you can configure the FileWritingMessageHandler as
      part of an outbound channel adapter using the namespace.
      <programlisting language="xml"><![CDATA[ <file:outbound-channel-adapter id="filesOut" directory="${input.directory.property}"/>]]></programlisting>
    </para>
    <para>
    The namespace based configuration also supports a <code>delete-source-files</code> attribute.
    If set to <code>true</code>, it will trigger deletion of the original source files after writing
    to a destination. The default value for that flag is <code>false</code>.
    <programlisting language="xml"><![CDATA[ <file:outbound-channel-adapter id="filesOut"
             directory="${output.directory}"
             delete-source-files="true"/>]]></programlisting>
    <note>
      <para>
        The <code>delete-source-files</code> attribute will only have an effect if the inbound
        Message has a File payload or if the <classname>FileHeaders.ORIGINAL_FILE</classname> header
        value contains either the source File instance or a String representing the original file path.
      </para>
    </note>
    </para>
    <para>
      In cases where you want to continue processing messages based on the written File you can use
      the <code>outbound-gateway</code> instead. It plays a very similar role as the
      <code>outbound-channel-adapter</code>. However after writing the File, it will also send it
      to the reply channel as the payload of a Message.
    <programlisting language="xml"><![CDATA[ <file:outbound-gateway id="mover" request-channel="moveInput"
            reply-channel="output"
            directory="${output.directory}"
            delete-source-files="true"/>]]></programlisting>
    </para>
    <note>
      The 'outbound-gateway' works well in cases where you want to first move a File and then send it
      through a processing pipeline. In such cases, you may connect the file namespace's
      'inbound-channel-adapter' element to the 'outbound-gateway' and then connect that gateway's
      reply-channel to the beginning of the pipeline.
    </note>
    <para>
      If you have more elaborate requirements or need to support additional payload types as input
      to be converted to file content you could extend the FileWritingMessageHandler, but a much
      better option is to rely on a <classname>Transformer</classname>.
    </para>
  </section>

  <section id="file-transforming">
    <title>File Transformers</title>
    <para>
      To transform data read from the file system to objects and the other way around you need
      to do some work. Contrary to <classname>FileReadingMessageSource</classname> and to a
      lesser extent <classname>FileWritingMessageHandler</classname>, it is very likely that you
      will need your own mechanism to get the job done. For this you can implement the
      <interfacename>Transformer</interfacename> interface. Or extend the
      <classname>AbstractFilePayloadTransformer</classname> for inbound messages. Some obvious
      implementations have been provided.
    </para>
    <para>
      <classname>FileToByteArrayTransformer</classname> transforms Files into byte[]s using
      Spring's <classname>FileCopyUtils</classname>. It is often better to use a sequence of
      transformers than to put all transformations in a single class. In that case the File to
      byte[] conversion might be a logical first step.
    </para>
    <para>
      <classname>FileToStringTransformer</classname> will convert Files to Strings as the name
      suggests. If nothing else, this can be useful for debugging (consider using with a Wire Tap).
    </para>
    <para>
      To configure File specific transformers you can use the appropriate elements from the file namespace.
      <programlisting language="xml"><![CDATA[ <file-to-bytes-transformer input-channel="input" output-channel="output"
             delete-files="true"/>

 <file:file-to-string-transformer input-channel="input" output-channel="output
             delete-files="true" charset="UTF-8"/>]]></programlisting>
      The <emphasis>delete-files</emphasis> option signals to the transformer that it should delete
      the inbound File after the transformation is complete. This is in no way a replacement for using the
      <classname>AcceptOnceFileListFilter</classname> when the FileReadingMessageSource is being used in a
      multi-threaded environment (e.g. Spring Integration in general).
    </para>
  </section>

</chapter>
