GH-3241: MetadataStoreSelector - compare old/new (#3242)

Resolves: https://github.com/spring-projects/spring-integration/issues/3241

 * Docs and XML namespace support.

src/reference/asciidoc/file.adoc

@@ -1116,6 +1116,50 @@ public class FileSplitterApplication {
 ----
 ====
 
+[[idempotent-file-splitter]]
+==== Idempotent Downstream Processing a Split File
+
+When `apply-sequence` is true, the splitter adds the line number in the `SEQUENCE_NUMBER` header (when `markers` is true, the markers are counted as lines).
+The line number can be used with an <<./handler-advice.adoc#idempotent-receiver,Idempotent Receiver>> to avoid reprocessing lines after a restart.
+
+For example:
+
+====
+[source, java]
+----
+@Bean
+public ConcurrentMetadataStore store() {
+    return new ZookeeperMetadataStore();
+}
+
+@Bean
+public MetadataStoreSelector selector() {
+    return new MetadataStoreSelector(
+            message -> message.getHeaders().get(FileHeaders.ORIGINAL_FILE, File.class)
+                    .getAbsolutePath(),
+            message -> message.getHeaders().get(IntegrationMessageHeaderAccessor.SEQUENCE_NUMBER)
+                    .toString(),
+            store())
+                    .compareValues(
+                            (oldVal, newVal) -> Integer.parseInt(oldVal) < Integer.parseInt(newVal));
+}
+
+@Bean
+public IdempotentReceiverInterceptor idempotentReceiverInterceptor() {
+    return new IdempotentReceiverInterceptor(selector());
+}
+
+@Bean
+public IntegrationFlow flow() {
+    ...
+    .split(new FileSplitter())
+    ...
+    .handle("lineHandler", e -> e.advice(idempotentReceiverInterceptor()))
+    ...
+}
+----
+====
+
 [[remote-persistent-flf]]
 === Remote Persistent File List Filters
 

src/reference/asciidoc/handler-advice.adoc

@@ -872,6 +872,13 @@ See the https://docs.spring.io/spring-integration/api/org/springframework/integr
 You can also customize the `value` for `ConcurrentMetadataStore` by using an additional `MessageProcessor`.
 By default, `MetadataStoreSelector` uses the `timestamp` message header.
 
+Normally, the selector selects a message for acceptance if there is no existing value for the key.
+In some cases, it is useful to compare the current and new values for a key, to determine whether the message should be accepted.
+Starting with version 5.3, the `compareValues` property is provided which references a `BiPredicate<String, String>`; the first parameter is the old value; return `true` to accept the message and replace the old value with the new value in the `MetadataStore`.
+This can be useful to reduce the number of keys; for example, when processing lines in a file, you can store the file name in the key and the current line number in the value.
+Then, after a restart, you can skip lines that have already been processed.
+See <<./file.adoc#idempotent-file-splitter,Idempotent Downstream Processing a Split File>> for an example.
+
 For convenience, the `MetadataStoreSelector` options are configurable directly on the `<idempotent-receiver>` component.
 The following listing shows all the possible attributes:
 
@@ -888,7 +895,8 @@ The following listing shows all the possible attributes:
         key-expression=""  <7>
         value-strategy=""  <8>
         value-expression=""  <9>
-        throw-exception-on-rejection="" />  <10>
+        compare-values="" <10>
+        throw-exception-on-rejection="" />  <11>
 ----
 
 <1> The ID of the `IdempotentReceiverInterceptor` bean.
@@ -928,7 +936,8 @@ Used by the underlying `MetadataStoreSelector`.
 Evaluates a `value` for the `idempotentKey` by using the request message as the evaluation context root object.
 Mutually exclusive with `selector` and `value-strategy`.
 By default, the 'MetadataStoreSelector' uses the 'timestamp' message header as the metadata 'value'.
-<10> Whether to throw an exception if the `IdempotentReceiverInterceptor` rejects the message.
+<10> A reference to a `BiPredicate<String, String>` bean which allows you to optionally select a message by comparing the old and new values for the key; `null` by default.
+<11> Whether to throw an exception if the `IdempotentReceiverInterceptor` rejects the message.
 Defaults to `false`.
 It is applied regardless of whether or not a `discard-channel` is provided.
 ====

src/reference/asciidoc/whats-new.adoc

@@ -81,6 +81,9 @@ See also <<./transactions.adoc#reactive-transactions,Reactive Transactions>>.
 A new `intercept()` operator to register `ChannelInterceptor` instances without creating explicit channels was added into Java DSL.
 See <<./dsl.adoc#java-dsl-intercept,Operator intercept()>> for more information.
 
+The `MessageStoreSelector` has a new mechanism to compare an old and new value.
+See <<./handler-advice.adoc#idempotent-receiver,Idempotent Receiver Enterprise Integration Pattern>> for more information.
+
 [[x5.3-amqp]]
 === AMQP Changes