INT-3353 Concurrent RedisMetadataStore

JIRA: https://jira.spring.io/browse/INT-3353

Shared metadata for PersistentAcceptOnceFileListFilters.

INT-3353 Polishing; PR Comments

src/reference/docbook/aggregator.xml

@@ -535,6 +535,9 @@ then you should simply provide an implementation of the <classname>ReleaseStrate
 			used to obtain a <interfacename>Lock</interfacename> based on the <code>groupId</code> for
 			concurrent operations on the
 			<code>MessageGroup</code>. By default, an internal <classname>DefaultLockRegistry</classname> is used.
+			Use of a distributed <interfacename>LockRegistry</interfacename>, such as the <classname
+			>RedisLockRegistry</classname>, ensures only one instance of the aggregator will operate
+			on a group concurrently. See <xref linkend="redis-lock-registry"/> for more information.
         </para>
       </callout>
 

src/reference/docbook/file.xml

@@ -36,11 +36,21 @@
       <classname>AcceptOnceFileListFilter</classname> is used. This filter
       ensures files are picked up only once from the directory.
       <note>
+       <para>
         The <classname>AcceptOnceFileListFilter</classname> stores its state in memory. If you wish the
         state to survive a system restart, consider using the
         <classname>FileSystemPersistentAcceptOnceFileListFilter</classname> instead. This filter stores
-        the accepted file names in a <interfacename>MetadataStore</interfacename> strategy (<xref linkend="metadata-store"/>).
+        the accepted file names in a <interfacename>MetadataStore</interfacename> implementation
+        (<xref linkend="metadata-store"/>).
         This filter matches on the filename and modified time.
+       </para>
+       <para>
+        Since <emphasis>version 4.0</emphasis>, this filter requires a <interfacename
+        >ConcurrentMetadataStore</interfacename>. When used with a shared data store
+        (such as <code>Redis</code> with the <classname>RedisMetadataStore</classname>) this allows
+        filter keys to be shared across multiple application instances, or when a network
+        file share is being used by multiple servers.
+       </para>
       </note>
       <programlisting language="xml"><![CDATA[<bean id="pollableFileSource"
     class="org.springframework.integration.file.FileReadingMessageSource"

src/reference/docbook/ftp.xml

@@ -195,6 +195,12 @@ protected void postProcessClientBeforeConnect(T client) throws IOException {
         the accepted file names in an instance of the
         <interfacename>MetadataStore</interfacename> strategy (<xref linkend="metadata-store"/>).
         This filter matches on the filename and the remote modified time.
+    </para>
+    <para>
+        Since <emphasis>version 4.0</emphasis>, this filter requires a <interfacename
+        >ConcurrentMetadataStore</interfacename>. When used with a shared data store
+        (such as <code>Redis</code> with the <classname>RedisMetadataStore</classname>) this allows
+        filter keys to be shared across multiple application or server instances.
     </para>
 	<note>
 	  <para>

src/reference/docbook/meta-data-store.xml

@@ -40,6 +40,12 @@
 		<interfacename>MetadataStore</interfacename> interface (e.g. JdbcMetadataStore)
 		and configure it as a bean in the Application Context.
 	</para>
+	<para>
+		Starting with <emphasis>version 4.0</emphasis>, <classname>SimpleMetadataStore</classname>,
+		<classname>PropertiesPersistingMetadataStore</classname> and
+		<classname>RedisMetadataStore</classname> implement <interfacename>ConcurrentMetadataStore</interfacename>.
+		These provide for atomic updates and can be used across multiple component or application instances.
+	</para>
 	<section id="idempotent-receiver">
 		<title>Idempotent Receiver</title>
 		<para>

src/reference/docbook/redis.xml

@@ -489,6 +489,11 @@ rt.setConnectionFactory(redisConnectionFactory);]]></programlisting>
 			<code>key</code> plays the role of a <emphasis>region</emphasis>, which is useful in distributed environment,
 			when several applications use the same Redis server. By default this <code>key</code> has the value <code>MetaData</code>.
 		</para>
+		<para>
+			Starting with <emphasis>version 4.0</emphasis>, this store now implements <interfacename
+			>ConcurrentMetadataStore</interfacename>, allowing it to be reliably shared across multiple
+			application instances where only one instance will be allowed to store or modify a key's value.
+		</para>
 	</section>
   <section id="redis-store-inbound-channel-adapter">
         <title>RedisStore Inbound Channel Adapter</title>
@@ -770,4 +775,34 @@ the serialization of values, you may want to consider providing your own
 		  <ulink url="http://redis.io/commands">Redis Specification</ulink>.
 		</para>
 	</section>
+	<section id="redis-lock-registry">
+		<title>Redis Lock Registry</title>
+		<para>
+			Starting with <emphasis>version 4.0</emphasis>, the <classname>RedisLockRegistry</classname> is
+			available. Certain components (for example aggregator and resequencer) use a lock obtained from
+			a <interfacename>LockRegistry</interfacename> instance to ensure
+			that only one thread is manipulating a group at a time. The <classname>DefaultLockRegistry</classname>
+			performs this function within a single component; you can now configure an external lock registry
+			on these components. When used with a shared <interfacename>MessageGroupStore</interfacename>,
+			the <classname>RedisLockRegistry</classname> can be use to provide this functionality across
+			multiple application instances, such that only one instance can manipulate the group at a time.
+		</para>
+		<para>
+			When a lock is released by a local thread, another local thread will generally be able to acquire the
+			lock immediately. If a lock is released by a thread using a different registry instance, it can take up to
+			100ms to acquire the lock.
+		</para>
+		<para>
+			To avoid "hung" locks (when a server fails), the locks in this registry are expired after a default
+			60 seconds, but this can be configured on the registry. Locks are normally held for a much smaller
+			time. 
+		</para>
+		<important>
+			Because the keys can expire, an attempt to unlock an expired lock will result in an exception
+			being thrown. However, be aware that the resources protected by such a lock may have been
+			compromised so such exceptions should be considered severe. The expiry should be set at
+			a large enough value to prevent this condition, while small enough that the lock can
+			be recovered after a server failure in a reasonable amount of time. 
+		</important>
+	</section>
 </chapter>

src/reference/docbook/resequencer.xml

@@ -56,9 +56,9 @@
 
   lock-registry="lockRegistry" ]]><co id="resxml18" /><![CDATA[
 
-		group-timeout="60000" ]]><co id="resxml19" /><![CDATA[
-		group-timeout-expression="size() ge 2 ? 100 : -1" ]]><co id="resxml20" /><![CDATA[
-		scheduler="taskScheduler" /> ]]><co id="resxml21" /></programlisting>
+  group-timeout="60000" ]]><co id="resxml19" /><![CDATA[
+  group-timeout-expression="size() ge 2 ? 100 : -1" ]]><co id="resxml20" /><![CDATA[
+  scheduler="taskScheduler" /> ]]><co id="resxml21" /></programlisting>
     <para><calloutlist>
         <callout arearefs="resxml1-co" id="resxml1">
           <para>The id of the resequencer is

src/reference/docbook/sftp.xml

@@ -320,6 +320,12 @@ xsi:schemaLocation="http://www.springframework.org/schema/integration/sftp
         the accepted file names in an instance of the
         <interfacename>MetadataStore</interfacename> strategy (<xref linkend="metadata-store"/>).
         This filter matches on the filename and the remote modified time.
+    </para>
+    <para>
+        Since <emphasis>version 4.0</emphasis>, this filter requires a <interfacename
+        >ConcurrentMetadataStore</interfacename>. When used with a shared data store
+        (such as <code>Redis</code> with the <classname>RedisMetadataStore</classname>) this allows
+        filter keys to be shared across multiple application or server instances.
     </para>
 	<note>
 	  <para>

src/reference/docbook/whats-new.xml

@@ -133,6 +133,15 @@
 				For more information, see <xref linkend="redis-outbound-gateway"/>.
 			</para>
 		</section>
+		<section id="4.0-redis-lock-registry">
+			<title>RedisLockRegistry</title>
+			<para>
+				The <classname>RedisLockRegistry</classname> is now available supporting global locks visible
+				to multiple application instances/servers. This can be used with aggregating message handlers
+				across multiple application instances such that group release will occur on only one instance.
+				For more information, see <xref linkend="redis-lock-registry"/> and <xref linkend="aggregator"/>.
+			</para>
+		</section>
 	</section>
 
 	<section id="4.0-general">
@@ -220,5 +229,17 @@
 				For more information see <xref linkend="aggregator-config"/>.
 			</para>
 		</section>
+		<section id="4.0-redis-metadata">
+			<title>Redis Metadata Store</title>
+			<para>
+				The <classname>RedisMetadataStore</classname> now implements <interfacename
+				>ConcurrentMetadataStore</interfacename>, allowing it to be used, for example,
+				in a <classname>AbstractPersistentAcceptOnceFileListFilter</classname>
+				implementation in a multiple application instance/server environment.
+				For more information, see <xref linkend="redis-metadata-store"/>,  
+				<xref linkend="file-reading"/>, <xref linkend="ftp-inbound"/> and
+				<xref linkend="sftp-inbound"/>.
+			</para>
+		</section>
 	</section>
 </chapter>