da9660a7ae
* GH-3846: Document Hazelcast module Fixes https://github.com/spring-projects/spring-integration/issues/3846 * Fix language in docs Co-authored-by: Gary Russell <grussell@vmware.com> Co-authored-by: Gary Russell <grussell@vmware.com>
88 lines
4.8 KiB
Plaintext
88 lines
4.8 KiB
Plaintext
[[metadata-store]]
|
|
=== Metadata Store
|
|
|
|
Many external systems, services, or resources are not transactional (Twitter, RSS, file systems, and so on), and there is not any ability to mark the data as read.
|
|
Also, sometimes, you may need to implement the Enterprise Integration Pattern https://www.enterpriseintegrationpatterns.com/IdempotentReceiver.html[idempotent receiver] in some integration solutions.
|
|
To achieve this goal and store some previous state of the endpoint before the next interaction with external system or to deal with the next message, Spring Integration provides the metadata store component as an an implementation of the `org.springframework.integration.metadata.MetadataStore` interface with a general key-value contract.
|
|
|
|
The metadata store is designed to store various types of generic metadata (for example, the published date of the last feed entry that has been processed) to help components such as the feed adapter deal with duplicates.
|
|
If a component is not directly provided with a reference to a `MetadataStore`, the algorithm for locating a metadata store is as follows: First, look for a bean with a `metadataStore` ID in the application context.
|
|
If one is found, use it.
|
|
Otherwise, create a new instance of `SimpleMetadataStore`, which is an in-memory implementation that persists only metadata within the lifecycle of the currently running application context.
|
|
This means that, upon restart, you may end up with duplicate entries.
|
|
|
|
If you need to persist metadata between application context restarts, the framework provides the following persistent `MetadataStores`:
|
|
|
|
* `PropertiesPersistingMetadataStore`
|
|
* <<./hazelcast.adoc#hazelcast-metadata-store,Hazelcast Metadata Store>>
|
|
* <<./jdbc.adoc#jdbc-metadata-store,JDBC Metadata Store>>
|
|
* <<./mongodb.adoc#mongodb-metadata-store,MongoDB Metadata Store>>
|
|
* <<./redis.adoc#redis-metadata-store,Redis Metadata Store>>
|
|
* <<./zookeeper.adoc#zk-metadata-store,Zookeeper Metadata Store>>
|
|
|
|
The `PropertiesPersistingMetadataStore` is backed by a properties file and a https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/util/PropertiesPersister.html[`PropertiesPersister`].
|
|
|
|
By default, it persists only the state when the application context is closed normally.
|
|
It implements `Flushable` so that you can persist the state at will, by invoking `flush()`.
|
|
The following example shows how to configure a 'PropertiesPersistingMetadataStore' with XML:
|
|
|
|
[source,xml]
|
|
----
|
|
<bean id="metadataStore"
|
|
class="org.springframework.integration.metadata.PropertiesPersistingMetadataStore"/>
|
|
----
|
|
|
|
Alternatively, you can provide your own implementation of the `MetadataStore` interface (for example, `JdbcMetadataStore`) and configure it as a bean in the application context.
|
|
|
|
Starting with version 4.0, `SimpleMetadataStore`, `PropertiesPersistingMetadataStore`, and `RedisMetadataStore` implement `ConcurrentMetadataStore`.
|
|
These provide for atomic updates and can be used across multiple component or application instances.
|
|
|
|
[[idempotent-receiver-pattern]]
|
|
==== Idempotent Receiver and Metadata Store
|
|
|
|
The metadata store is useful for implementing the EIP https://www.enterpriseintegrationpatterns.com/IdempotentReceiver.html[idempotent receiver] pattern when there is need to filter an incoming message if it has already been processed and you can discard it or perform some other logic on discarding.
|
|
The following configuration shows an example of how to do so:
|
|
|
|
====
|
|
[source,xml]
|
|
----
|
|
<int:filter input-channel="serviceChannel"
|
|
output-channel="idempotentServiceChannel"
|
|
discard-channel="discardChannel"
|
|
expression="@metadataStore.get(headers.businessKey) == null"/>
|
|
|
|
<int:publish-subscribe-channel id="idempotentServiceChannel"/>
|
|
|
|
<int:outbound-channel-adapter channel="idempotentServiceChannel"
|
|
expression="@metadataStore.put(headers.businessKey, '')"/>
|
|
|
|
<int:service-activator input-channel="idempotentServiceChannel" ref="service"/>
|
|
----
|
|
====
|
|
|
|
The `value` of the idempotent entry may be an expiration date, after which that entry should be removed from metadata store by some scheduled reaper.
|
|
|
|
See also <<./handler-advice.adoc#idempotent-receiver,Idempotent Receiver Enterprise Integration Pattern>>.
|
|
|
|
[[metadatastore-listener]]
|
|
==== `MetadataStoreListener`
|
|
|
|
Some metadata stores (currently only zookeeper) support registering a listener to receive events when items change, as the following example shows:
|
|
|
|
====
|
|
[source, java]
|
|
----
|
|
public interface MetadataStoreListener {
|
|
|
|
void onAdd(String key, String value);
|
|
|
|
void onRemove(String key, String oldValue);
|
|
|
|
void onUpdate(String key, String newValue);
|
|
}
|
|
----
|
|
====
|
|
|
|
See the https://docs.spring.io/spring-integration/api/org/springframework/integration/metadata/MetadataStoreListenerAdapter.html[Javadoc] for more information.
|
|
The `MetadataStoreListenerAdapter` can be subclassed if you are interested only in a subset of events.
|