INT-3147: (3167,3173,1941) Improve MetadataStore
Previously, `MetadataStore` couldn't be configured for Twitter Adapters - only a global one could be used. The `metadataKey` was generated automatically with a 'difficult' value. * Register all `MessageSource` for `SourcePollingChannelAdapter` as beans with id based on adapter id and prefix '.source' (INT-3147) * Polishing parser to get rid of explicit `MessageSource` beans. (INT-3147) * Make Feed and Twitter adapters `id` attribute as required - now it presents a `metadataKey` for `MetadataStore` (INT-3147) * Add to Twitter adapters a reference attribute for `MetadataStore` (INT-3173) * Add Twitter adapters `poll-skip-period` attribute (INT-3167) * Add and implement `MetadataStore#remove` (INT-1941) * Make `MetadataStore` as `@ManagedResource` (INT-1941) * Polishing tests JIRAs: https://jira.springsource.org/browse/INT-3147 https://jira.springsource.org/browse/INT-3167 https://jira.springsource.org/browse/INT-3173 https://jira.springsource.org/browse/INT-1941 INT-3147: Polishing and fixes * add domain suffix to `metadataKey` * change contract of `MetadataStore.remove` * remove timeout window from `AbstractTwitterMessageSource` * polishing and fix `SearchReceivingMessageSourceWithRedisTests` INT-3147: Rebasing and polishing INT-3147: fix 'metadata' package tangle INT-3147 Doc Polishing
This commit is contained in:
committed by
Gary Russell
parent
1fb838dd1a
commit
5be8ef3fd8
@@ -62,14 +62,14 @@ xsi:schemaLocation="http://www.springframework.org/schema/integration/feed
|
||||
Spring Integration provides a convenient mechanism to eliminate the need to worry about duplicate entries.
|
||||
Each feed entry will have a <emphasis>published date</emphasis> field. Every time a new Message is generated and sent,
|
||||
Spring Integration will store the value of the latest <emphasis>published date</emphasis> in an instance of the
|
||||
<classname>org.springframework.integration.store.MetadataStore</classname> strategy. The MetadataStore interface is
|
||||
<classname>org.springframework.integration.metadata.MetadataStore</classname> strategy. The MetadataStore interface is
|
||||
designed to store various types of generic meta-data (e.g., published date of the last feed entry that has been processed)
|
||||
to help components such as this Feed adapter deal with duplicates.
|
||||
</para>
|
||||
<para>
|
||||
The default rule for locating this metadata store is as follows:
|
||||
<emphasis>Spring Integration</emphasis> will look for a bean of type
|
||||
<classname>org.springframework.integration.store.MetadataStore</classname> in
|
||||
<classname>org.springframework.integration.metadata.MetadataStore</classname> in
|
||||
the ApplicationContext. If one is found then it will be used, otherwise
|
||||
it will create a new instance of <classname>SimpleMetadataStore</classname>
|
||||
which is an in-memory implementation that will only persist metadata within
|
||||
@@ -107,5 +107,10 @@ xsi:schemaLocation="http://www.springframework.org/schema/integration/feed
|
||||
<interfacename>MetadataStore</interfacename> interface (e.g. JdbcMetadataStore)
|
||||
and configure it as bean in the Application Context.
|
||||
</para>
|
||||
<note>
|
||||
The key used to persist the latest <emphasis>published date</emphasis> is the value of the (required)
|
||||
<code>id</code> attribute of the Feed Inbound Channel Adapter component plus the <code>feedUrl</code>
|
||||
from the adapter's configuration.
|
||||
</note>
|
||||
</section>
|
||||
</chapter>
|
||||
|
||||
@@ -141,11 +141,12 @@ twitter.oauth.accessTokenSecret=AbRxUAvyNCtqQtxFK8w5ZMtMj20KFhB6o]]></programlis
|
||||
criteria you'll end up with the same set of tweets unless some other new tweet that matches your search criteria was posted
|
||||
in between your searches. In that situation you'll get all the tweets you had before plus the new one. But what you really
|
||||
want is only the new tweet(s). Spring Integration provides an elegant mechanism for handling these situations.
|
||||
The latest Tweet timestamp will be stored in an instance of the <classname>org.springframework.integration.store.MetadataStore</classname> which is a
|
||||
The latest Tweet id will be stored in an instance of the <classname>org.springframework.integration.metadata.MetadataStore</classname> which is a
|
||||
strategy interface designed for storing various types of metadata (e.g., last retrieved tweet in this case). That strategy helps components such as
|
||||
these Twitter adapters avoid duplicates. By default, Spring Integration will look for a bean of type
|
||||
<classname>org.springframework.integration.store.MetadataStore</classname> in the ApplicationContext.
|
||||
If one is found then it will be used, otherwise it will create a new instance of <classname>SimpleMetadataStore</classname>
|
||||
<classname>org.springframework.integration.metadata.MetadataStore</classname> in the ApplicationContext. Alternatively,
|
||||
you can configure an explicit <classname>MetadataStore</classname> on the adapter.
|
||||
If there is no explicit or default store, the adapter will create a new instance of <classname>SimpleMetadataStore</classname>
|
||||
which is a simple in-memory implementation that will only persist metadata within the lifecycle of the currently running application context.
|
||||
That means upon restart you may end up with duplicate entries. If you need to persist metadata between Application Context
|
||||
restarts, you may use the <classname>PropertiesPersistingMetadataStore</classname> (which is backed by a properties file, and a persister
|
||||
@@ -165,8 +166,16 @@ twitter.oauth.accessTokenSecret=AbRxUAvyNCtqQtxFK8w5ZMtMj20KFhB6o]]></programlis
|
||||
</warning>
|
||||
<programlisting language="xml"><![CDATA[<bean id="metadataStore" class="o.s.i.store.PropertiesPersistingMetadataStore"/>
|
||||
]]></programlisting>
|
||||
The Poller that is configured as part of any Inbound Twitter Adapter (see below) will simply poll from this MetadataStore to determine the latest tweet
|
||||
received.
|
||||
<para>
|
||||
If the <classname>MetadataStore</classname> is persistent, during initialization, any Inbound Twitter Adapter (see below)
|
||||
will retrieve the latest tweet id that has already been sent by the adapter.
|
||||
</para>
|
||||
<note>
|
||||
The key used to persist the latest <emphasis>twitter id</emphasis> is the value of the (required)
|
||||
<code>id</code> attribute of the Twitter Inbound Channel Adapter component plus the <code>profileId</code>
|
||||
of the Twitter user.
|
||||
</note>
|
||||
|
||||
<section id="inbound-twitter-update">
|
||||
<title>Inbound Message Channel Adapter</title>
|
||||
<para>
|
||||
|
||||
Reference in New Issue
Block a user