spring-integration/src/reference/docbook/mail.xml

<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="mail"
    xmlns:xlink="http://www.w3.org/1999/xlink">
  <title>Mail Support</title>

  <section id="mail-outbound">
    <title>Mail-Sending Channel Adapter</title>
    <para>
      Spring Integration provides support for outbound email with the
      <classname>MailSendingMessageHandler</classname>. It delegates to a configured instance of Spring's
      <interfacename>JavaMailSender</interfacename>:
      <programlisting language="java"> JavaMailSender mailSender = context.getBean("mailSender", JavaMailSender.class);

 MailSendingMessageHandler mailSendingHandler = new MailSendingMessageHandler(mailSender);</programlisting>
      <classname>MailSendingMessageHandler</classname> has various mapping strategies that use Spring's
      <interfacename>MailMessage</interfacename> abstraction. If the received Message's payload is already
      a <classname>MailMessage</classname> instance, it will be sent directly.
      Therefore, it is generally recommended to precede this
      consumer with a Transformer for non-trivial MailMessage construction requirements. However, a few simple
      Message mapping strategies are supported out-of-the-box. For example, if the message payload is a byte array,
      then that will be mapped to an attachment. For simple text-based emails, you can provide a String-based
      Message payload. In that case, a MailMessage will be created with that String as the text content. If you
      are working with a Message payload type whose toString() method returns appropriate mail text content, then
      consider adding Spring Integration's <emphasis>ObjectToStringTransformer</emphasis> prior to the outbound
      Mail adapter (see the example within <xref linkend="transformer-namespace"/> for more detail).
    </para>
    <para>
      The outbound MailMessage may also be configured with certain values from the
      <classname>MessageHeaders</classname>. If available, values will be mapped to the outbound mail's
      properties, such as the recipients (TO, CC, and BCC), the from/reply-to, and the subject. The header names are
      defined by the following constants:
      <programlisting language="java"> MailHeaders.SUBJECT
 MailHeaders.TO
 MailHeaders.CC
 MailHeaders.BCC
 MailHeaders.FROM
 MailHeaders.REPLY_TO</programlisting>
    </para>
    <note>
      <classname>MailHeaders</classname>  also allows you to override corresponding <classname>MailMessage</classname> values.
      For example: If <classname>MailMessage.to</classname> is set to 'foo@bar.com' and <classname>MailHeaders.TO</classname>
      Message header is provided it will take precedence and override the corresponding value in <classname>MailMessage</classname>
    </note>
  </section>

  <section id="mail-inbound">
    <title>Mail-Receiving Channel Adapter</title>
    <para>
      Spring Integration also provides support for inbound email with the
      <classname>MailReceivingMessageSource</classname>. It delegates to a configured instance of Spring
      Integration's own <interfacename>MailReceiver</interfacename> interface, and there are two implementations:
      <classname>Pop3MailReceiver</classname> and <classname>ImapMailReceiver</classname>. The easiest way to
      instantiate either of these is by passing the 'uri' for a Mail store to the receiver's constructor. For example:
      <programlisting language="java"><![CDATA[ MailReceiver receiver = new Pop3MailReceiver("pop3://usr:pwd@localhost/INBOX");
]]></programlisting>
    </para>
    <para>
      Another option for receiving mail is the IMAP "idle" command (if supported by the mail server you are using).
      Spring Integration provides the <classname>ImapIdleChannelAdapter</classname> which is itself a Message-producing
      endpoint. It delegates to an instance of the <classname>ImapMailReceiver</classname> but enables asynchronous
      reception of Mail Messages. There are examples in the next section of configuring both types of inbound Channel
      Adapter with Spring Integration's namespace support in the 'mail' schema.
    </para>
  </section>

  <section id="mail-namespace">
    <title>Mail Namespace Support</title>
    <para>
      Spring Integration provides a namespace for mail-related configuration. To use it, configure the following schema
      locations.<programlisting language="xml"><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/integration/mail
            http://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd">]]></programlisting>
    </para>
    <para>
      To configure an outbound Channel Adapter, provide the channel to receive from, and the MailSender:
      <programlisting language="xml"><![CDATA[<int-mail:outbound-channel-adapter channel="outboundMail"
                              mail-sender="mailSender"/>]]></programlisting>
      Alternatively, provide the host, username, and password:
      <programlisting language="xml"><![CDATA[<int-mail:outbound-channel-adapter channel="outboundMail"
                              host="somehost" username="someuser" password="somepassword"/>]]></programlisting>
      <note>
        Keep in mind, as with any outbound Channel Adapter, if the referenced channel is a PollableChannel, a
        &lt;poller&gt; sub-element should be provided with either an interval-trigger or cron-trigger.
      </note>
    </para>
    <para>
      To configure an Inbound Channel Adapter, you have the choice between polling or event-driven (assuming your
      mail server supports IMAP IDLE - if not, then polling is the only option). A polling Channel Adapter simply
      requires the store URI and the channel to send inbound Messages to. The URI may begin with "pop3" or "imap":
      <programlisting language="xml"><![CDATA[<int-mail:inbound-channel-adapter id="imapAdapter"
      store-uri="imaps://[username]:[password]@imap.gmail.com/INBOX"
      java-mail-properties="javaMailProperties"
      channel="recieveChannel"
      should-delete-messages="true"
      should-mark-messages-as-read="true"
      auto-startup="true">
      <int:poller max-messages-per-poll="1" fixed-rate="5000"/>
</int-mail:inbound-channel-adapter>]]></programlisting>
      If you do have IMAP idle support, then you may want to configure the "imap-idle-channel-adapter" element instead.
      Since the "idle" command enables event-driven notifications, no poller is necessary for this adapter. It will
      send a Message to the specified channel as soon as it receives the notification that new mail is available:
      <programlisting language="xml"><![CDATA[<int-mail:imap-idle-channel-adapter id="customAdapter"
      store-uri="imaps://[username]:[password]@imap.gmail.com/INBOX"
      channel="recieveChannel"
      auto-startup="true"
      should-delete-messages="false"
      should-mark-messages-as-read="true"
      java-mail-properties="javaMailProperties"/>]]></programlisting>
    ... where <emphasis>javaMailProperties</emphasis> could be provided by creating and populating
     a regular <classname>java.utils.Properties</classname> object. For example via <emphasis>util</emphasis> namespace
     provided by Spring.
     <important>
     	If your username contains the '@' character use '%40' instead of '@' to avoid parsing errors from the underlying JavaMail API.
     </important>
     <programlisting language="xml"><![CDATA[<util:properties id="javaMailProperties">
  <prop key="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
  <prop key="mail.imap.socketFactory.fallback">false</prop>
  <prop key="mail.store.protocol">imaps</prop>
  <prop key="mail.debug">false</prop>
</util:properties>]]></programlisting>
    </para>
    
    <para>
    By default, the <classname>ImapMailReceiver</classname> will search for Messages based on the default <classname>SearchTerm</classname> 
    which is <emphasis>All mails that are RECENT (if supported), that are NOT ANSWERED, that are NOT DELETED, that are NOT SEEN and have not 
    been processed by this mail receiver (enabled by the use of the custom USER flag or simply NOT FLAGGED if not supported)</emphasis>. 
    Since version 2.2, the <classname>SearchTerm</classname> used by the  <classname>ImapMailReceiver</classname> is fully configurable 
    via the <code>SearchTermStrategy</code> which you can inject via the <code>search-term-strategy</code> attribute.
    <classname>SearchTermStrategy</classname> is a simple strategy interface with a single method that allows you to create an 
    instance of the <classname>SearchTerm</classname> that will be used by the <classname>ImapMailReceiver</classname>.
    </para>
    <programlisting language="java"><![CDATA[public interface SearchTermStrategy {

	SearchTerm generateSearchTerm(Flags supportedFlags, Folder folder);

}]]></programlisting>
    <para>
For example:
    </para>
<programlisting language="xml"><![CDATA[<mail:imap-idle-channel-adapter id="customAdapter"
			store-uri="imap:foo"
			. . .	
			search-term-strategy="searchTermStrategy"/>
			
<bean id="searchTermStrategy" class="org.springframework.integration.mail.config.ImapIdleChannelAdapterParserTests.TestSearchTermStrategy"/>]]></programlisting>
    <para>
In the above example instead of relying on the default <classname>SearchTermStrategy</classname> the 
<classname>TestSearchTermStrategy</classname> will be used instead
    </para>
    
    <para><emphasis>IMAP IDLE and lost connection</emphasis></para>
    
    <para>
    When using IMAP IDLE channel adapter there might be situations where connection to the server may be lost 
    (e.g., network failure) and since Java Mail documentation explicitly states that the actual IMAP API is EXPERIMENTAL it is 
    important to understand the differences in the API and how to deal with them when configuring IMAP IDLE adapters.
    Currently Spring Integration Mail adapters was tested with Java Mail 1.4.1 and Java Mail 1.4.3 and depending on which one 
    is used special attention must be payed to some of the java mail properties that needs to be set with regard to auto-reconnect.
	</para>
	<para>
    <emphasis>
    The following behavior was observed with GMAIL but should provide you with some tips on how to solve re-connect 
    issue with other providers, however feedback is always welcome. Again, below notes are based on GMAIL.
    </emphasis>
    </para>
    <para>
    With Java Mail 1.4.1 if <code>mail.imaps.timeout</code> property is set for a relatively short period of time (e.g., ~ 5 min) 
    then <code>IMAPFolder.idle()</code> will throw <classname>FolderClosedException</classname> after this timeout. 
    However if this property is not set (should be indefinite) the behavior that was observed is that <code>IMAPFolder.idle()</code> 
    method never returns nor it throws an exception. It will however reconnect automatically if connection was lost for a short 
    period of time (e.g., under 10 min), but  if connection was lost for a long period of time (e.g., over 10 min), then 
    <code>IMAPFolder.idle()</code> will not throw <classname>FolderClosedException</classname> nor it will re-establish connection 
    and will remain in the blocked state indefinitely, thus leaving you no possibility to reconnect without restarting the adapter. 
    So the only way to make re-connect to work with Java Mail 1.4.1 is to set <code>mail.imaps.timeout</code> property explicitly 
    to some value, but it also means that such value shoudl be relatively short (under 10 min) and the connection should be 
    re-estabished relatively quickly. Again, it may be different with other providers.

	With Java Mail 1.4.3 there was significant improvements to the API ensuring that there will always be a condition which 
	will force <code>IMAPFolder.idle()</code> method to return via <classname>StoreClosedException</classname> or 
	<classname>FolderClosedException</classname> or simply return, thus allowing us to proceed with auto-reconnect.
    Currently auto-reconnect will run infinitely making attempts to reconnect every 10 sec. 
    </para>

    <important>
    In both configurations <code>channel</code> and <code>should-delete-messages</code> are the <emphasis>REQUIRED</emphasis>
    attributes. The important thing to understand is why <code>should-delete-messages</code> is required.
    The issue is with the POP3 protocol, which does NOT have any knowledge of messages that were READ. It can only know what's been read 
    within a single session. This means that when your POP3 mail adapter is running, emails are successfully consumed as as they become available during each poll
    and no single email message will be delivered more then once. However, as soon as you restart your adapter and begin a new session
    all the email messages that might have been retrieved in the previous session will be retrieved again. That is the nature of POP3. Some might argue
    that <code>should-delete-messages</code> should be TRUE by default. In other words, there are two valid and mutually exclusive use cases 
    which make it very hard to pick a single "best" default. You may want to configure your adapter as the only email receiver in which
    case you want to be able to restart such adapter without fear that messages that were delivered before will not be redelivered again. 
    In this case setting <code>should-delete-messages</code> to TRUE would make most sense. However, you may have another use case where 
    you may want to have multiple adapters that simply monitor email servers and their content. In other words you just want to 'peek but not touch'. 
    Then setting <code>should-delete-messages</code> to FALSE would be much more appropriate. So since it is hard to choose what should be
    the right default value for the <code>should-delete-messages</code> attribute, we simply made it a required attribute, to be set by the user.
    Leaving it up to the user also means, you will be less likely to end up with unintended behavior.
    </important>

     <note>When configuring a polling email adapter's <emphasis>should-mark-messages-as-read</emphasis> attribute,
    be aware of the protocol you are configuring to retrieve messages. For example POP3 does not support this flag
    which means setting it to either value will have no effect as messages will NOT be marked as read.</note>

    <important>
      <para>
        It is important to understand that that these actions (marking messages read, and deleting messages) are performed after
        the messages are received, but before they are processed. This can cause messages to be lost.
      </para>
      <para>
        You may wish to consider using transaction synchronization instead - see <xref linkend="mail-tx-sync"/>
      </para>
    </important>

    <para>
      When using the namespace support, a <emphasis>header-enricher</emphasis> Message Transformer is also available.
      This simplifies the application of the headers mentioned above to any Message prior to sending to the
      Mail-sending Channel Adapter.
      <programlisting language="xml"><![CDATA[<int-mail:header-enricher subject="Example Mail"
                     to="to@example.org"
                     cc="cc@example.org"
                     bcc="bcc@example.org"
                     from="from@example.org"
                     reply-to="replyTo@example.org"
                     overwrite="false"/>]]></programlisting>
    </para>
    <para>
      Finally, the &lt;imap-idle-channel-adapter/&gt; also accepts the 'error-channel' attribute. 
      If a downstream exception is thrown and an 'error-channel' is specified, 
      a MessagingException message containing the failed message and original exception, will be sent to this channel. 
      Otherwise, if the downstream channels are synchronous, any such exception will simply be logged as a 
      warning by the channel adapter.
    </para>
  </section>
  
  <section id="mail-filtering">
  	<title>Email Message Filtering</title>
  	<para>
  	Very often you may encounter a requirement to filter incoming messages 
  	(e.g., You want to only read emails that have 'Spring Integration' in the <emphasis>Subject</emphasis> line). This 
  	could be easily accomplished by connecting Inbound Mail adapter with an expression-based <emphasis>Filter</emphasis>. Although it would work, there is a 
  	downside to this approach. Since messages would be filtered after going through inbound mail adapter all such messages would be marked as read (SEEN) or 
  	Un-read (depending on the value of <code>should-mark-messages-as-read</code> attribute). However in reality what would be more useful is to mark messages 
  	as SEEN only if they passed the filtering criteria. This is very similar to looking at your email client while scrolling through all the 
  	messages in the preview pane, but only flagging messages as SEEN that were actually opened and read.
  	</para>
  	<para>
  	In Spring Integration 2.0.4 we've introduced <code>mail-filter-expression</code> attribute on <code>inbound-channel-adapter</code> and 
  	<code>imap-idle-channel-adapter</code>. This attribute allows you to provide an expression which is a combination of SpEL and Regular Expression. 
  	For example if you would like to read only emails that contain 'Spring Integration' in the Subject line, you would configure <code>mail-filter-expression</code> attribute
  	like this this: <code>mail-filter-expression="subject matches '(?i).*Spring Integration.*"</code>
  	</para>
  	<para>
  	Since <classname>javax.mail.internet.MimeMessage</classname> is the root context of SpEL Evaluation Context, you can filter on any value available 
  	through MimeMessage including the actual body of the message. This one is particularly important since reading the body of the message would 
  	typically result in such message to be marked as SEEN by default, but since we now setting PEAK flag of every incomming message to 'true', only 
  	messages that were explicitly marked as SEEN will be seen as read.
  	</para>
  	
  	<para>
  	So in the below example only messages that match the filter expression will be output by this adapter and only those messages will be marked as SEEN. 
  	In this case based on the <code>mail-filter-expression</code> only messages that contain 'Spring Integration' in the subject line will be 
  	produced by this adapter.
  	
  	<programlisting language="xml"><![CDATA[<int-mail:imap-idle-channel-adapter id="customAdapter"
	store-uri="imaps://some_google_address:${password}@imap.gmail.com/INBOX"
	channel="receiveChannel"	
	should-mark-messages-as-read="true"
	java-mail-properties="javaMailProperties"
	mail-filter-expression="subject matches '(?i).*Spring Integration.*'"/>]]></programlisting>
  	</para>
  	<para>
  	Another reasonable question is what happens on the next poll, or idle event, or what happens when such adapter is restarted. 
  	Will there be a potential duplication of massages to be filtered? In other words if on the last retrieval where you had 5 new messages and only 1 passed 
  	the filter what would happen with the other 4. Would they go through the filtering logic again on the next poll or idle? After all they were not marked 
  	as SEEN. The actual answer is no. They would not be subject of duplicate processing due to another flag (RECENT) that is set by the Email server and 
  	is used by Spring Integration mail search filter. Folder implementations set this flag to indicate that this message is new to this folder, that is, 
  	it has arrived since the last time this folder was opened. In other while our adapter may peek at the email it also lets the email server know that 
  	such email was touched and therefore will be marked as RECENT by the email server.
  	</para>
  </section>

  <section id="mail-tx-sync">
    <title>Transaction Synchronization</title>
      <para>
        Transaction synchronization for inbound adapters allows you to take different actions after a
        transaction commits, or rolls back. Transaction synchronization is enabled by adding a &lt;transactional/&gt; element
        to the poller for the polled &lt;inbound-adapter/&gt;, or to the &lt;imap-idle-inbound-adapter/&gt;. Even if there
        is no 'real' transaction involved, you can still enable this feature by using a
        <classname>PseudoTransactionManager</classname> with the &lt;transactional/&gt; element.
        For more information, see <xref linkend="transaction-synchronization" />.
      </para>
      <para>
        Because of the many different mail servers, and specifically the limitations that some have, at this time we
        only provide a strategy for these transaction synchronizations. You can send the messages to some other Spring
        Integration components, or invoke a custom bean to perform some action. For example, to move an IMAP message
        to a different folder after the transaction commits, you might use something similar to the following:
      </para>
      <programlisting><![CDATA[<int-mail:imap-idle-channel-adapter id="customAdapter"
			store-uri="imaps://foo.com:password@imap.foo.com/INBOX"
			channel="receiveChannel"
			auto-startup="true"
			should-delete-messages="false"		
			java-mail-properties="javaMailProperties">
	<int:transactional synchronization-factory="syncFactory"/>
</int-mail:imap-idle-channel-adapter>

<int:transaction-synchronization-factory id="syncFactory">
	<int:after-commit expression="@syncProcessor.process(payload)"/>
</int:transaction-synchronization-factory>

<bean id="syncProcessor" class="foo.bar.Mover"/>

public class Mover {

	public void process(MimeMessage message) throws Exception{
		Folder folder = message.getFolder();
		Store store = null;
		if (!folder.isOpen()){
			folder.open(Folder.READ_WRITE);
			store = folder.getStore();
		}
		message.setFlag(Flags.Flag.DELETED, true);
		Folder fooFolder = store.getFolder("FOO"));
		fooFolder.appendMessages(new MimeMessage[]{message});
		folder.expunge();
	}
}]]></programlisting>
    <important>
      For the message to be still available for manipulation after the transaction, <emphasis>should-delete-messages</emphasis>
      must be set to 'false'.
    </important>
  </section>
</chapter>