Merge pull request #42 from dturanski/INT-2082

SpelMessageProducerSupport abstract and package-private
  INT-2082 added supported event types
  INT-2081 added gemfire documentation to the reference. Also fixed a typo in the schema
  INT-2082 added support for evaluating payload expression in ContinuousQueryMessageProducer
This commit is contained in:
Mark Fisher
2011-08-25 22:45:54 -04:00
12 changed files with 317 additions and 70 deletions

View File

@@ -0,0 +1,129 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" xml:id="gemfire"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude">
<title>GemFire Support</title>
<para>
Spring Integration provides support for VMWare vFabric GemFire
</para>
<section id="gemfire-intro">
<title>Introduction</title>
<para>
VMWare vFabric GemFire (GemFire) is a distributed data management platform providing a key-value data grid along with advanced distributed system features such as event processing, continuous querying, and
remote function execution. This guide assumes
some familiarity with <ulink url="http://www.gemstone.com/docs/6.6.RC/product/docs/html/user_guide/UserGuide_GemFire.html#Getting%20Started%20with%20Gemfire">GemFire</ulink>
and its <ulink url="http://www.gemstone.com/docs/6.6.RC/product/docs/japi/index.html">API</ulink>.
</para>
<para>
Spring integration provides support for GemFire by providing inbound adapters for entry and continuous query events,
and an outbound adapter to write entries to the cache. Spring integration leverages the
<ulink url="http://www.springsource.org/spring-gemfire">Spring Gemfire</ulink> project, providing a thin
wrapper over its components.
</para>
<para>
To configure the 'int-gfe' namespace, include the following elements within the headers of your XML configuration file:
<programlisting language="xml"><![CDATA[xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire
http://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire-2.1.xsd"]]></programlisting>
</para>
</section>
<section>
<title>Inbound Channel Adapter</title>
<para>
The <emphasis>inbound-channel-adapter</emphasis> produces messages on a channel triggered by a GemFire <classname>EntryEvent</classname>. GemFire
generates events whenever an entry is CREATED, UPDATED, DESTROYED, or INVALIDATED in the associated region. The inbound channel adapter allows you to filter on a subset of these
events. For example, you may want to only produce messages in response to an entry being CREATED. In addition, the inbound channel adapter can evaluate a SpEL
expression if, for example, you want your message payload to contain an event property such as the new entry value.
<programlisting language="xml">
<![CDATA[<gfe:cache/>
<gfe:replicated-region id="region"/>
<int-gfe:inbound-channel-adapter id="inputChannel" region="region"
cache-events="CREATED" expression="newValue"/>]]>
</programlisting>
In the above configuration, we are creating a GemFire <classname>Cache</classname> and <classname>Region</classname> using Spring GemFire's 'gfe' namespace.
The inbound-channel-adapter requires a reference to the GemFire region for which the adapter will be listening for events. Optional attributes include <code>cache-events</code>
which can contain a comma separated list of event types for which a message will be produced on the input channel. By default all event types are enabled.
Note that this adapter conforms to Spring integration conventions.
If no <code>channel</code> attribute is provided, the channel will be created from the <code>id</code> attribute. This adapter also supports an <code>error-channel</code>.
If <code>expression</code> is not provided the message payload will be a GemFire <classname>EntryEvent</classname>
</para>
</section>
<section>
<title>Continuous Query Inbound Channel Adapter</title>
<para>
The <emphasis>cq-inbound-channel-adapter</emphasis> produces messages a channel triggered by a GemFire continuous query or <classname>CqEvent</classname> event. Spring GemFire introduced
continuous query support in release 1.1, including a <classname>QueryListenerContainer</classname> which provides a nice abstraction over the GemFire native API. This adapter requires a
reference to a QueryListenerContainer, and creates a listener for a given <code>query</code> and executes the query. The continuous query acts as an event source that will fire whenever its
result set changes state.
<note>
GemFire queries are written in OQL and are scoped to the entire cache (not just one region). Additionally, continuous queries require a remote (i.e., running in a separate process or remote host)
cache server. Please consult the <ulink url="http://www.gemstone.com/docs/6.6.RC/product/docs/html/user_guide/UserGuide_GemFire.html#Continuous%20Querying">GemFire documentation</ulink> for more information on
implementing continuous queries.
</note>
<programlisting language="xml">
<![CDATA[<gfe:cache id="client-cache"/>
<gfe:pool id="client-pool" subscription-enabled="true" >
<!--configure server or locator here required to address the cache server -->
</gfe:pool>
<gfe:client-region id="test" cache-ref="client-cache" pool-name="client-pool"/>
<bean id="queryListenerContainer"
class="org.springframework.data.gemfire.listener.QueryListenerContainer">
<property name="cache" ref="client-cache"/>
</bean>
<int-gfe:cq-inbound-channel-adapter id="inputChannel"
query-listener-container="queryListenerContainer"
query="select * from /test"/>
]]>
</programlisting>
In the above configuration, we are creating a GemFire client cache
(recall a cache server is required for this implementation and its address is configured as a sub-element of the pool), a client region and a <classname>QueryListenerContainer</classname>
using Spring GemFire. The continuous query inbound channel adapter requires a <code>query-listener-container</code> attribute which contains a reference to the <classname>QueryListenerContainer</classname>. Optionally,
it accepts an <code>expression</code> attribute which uses SpEL to transform the <code>CqEvent</code> or extract an individual property as needed. The cq-inbound-channel-adapter also supports a
<code>query-events</code> attribute, containing a comma separated list of event types for which a message will be produced on the input channel (all events are enabled by default), <code>query-name</code> which provides an optional query name, and
<code>expression</code> which works as described in the above section.
Note that this adapter conforms to Spring integration conventions.
If no <code>channel</code> attribute is provided, the channel will be created from the <code>id</code> attribute. This adapter also supports an <code>error-channel</code>
</para>
</section>
<section>
<title>Outbound Channel Adapter</title>
<para>
The <emphasis>outbound-channel-adapter</emphasis> writes cache entries mapped from the message payload. In its simplest form, it expects a
payload of type <classname>java.util.Map</classname> and puts the map entries into its configured region.
<programlisting language="xml">
<![CDATA[<int-gfe:outbound-channel-adapter id="cacheChannel" region="region"/>]]>
</programlisting>
Given the above configuration, an exception will be thrown if the payload is not a Map. Additionally, the outbound channel adapter can be configured to create a
map of cache entries using SpEL of course.
<programlisting language="xml">
<![CDATA[<int-gfe:outbound-channel-adapter id="cacheChannel" region="region">
<int-gfe:cache-entries>
<entry key="payload.toUpperCase()" value="payload.toLowerCase()"/>
<entry key="'foo'" value="'bar'"/>
</int-gfe:cache-entries>
</int-gfe:outbound-channel-adapter>
]]>
</programlisting>
In the above configuration, the inner element <code>cache-entries</code> is semantically equivalent to Spring 'map' element. The adapter interprets the <code>key</code> and
<code>value</code> attributes as SpEL expressions with the message as the evaluation context. Note that this contain
arbitrary cache entries (not only those derived from the message) and that literal values must be enclosed in single quotes. In the above example, if the message sent to
<code>cacheChannel</code> has a String payload with a value "Hello", two entries <code>[HELLO:hello, foo:bar]</code> will be written (created or updated) in the cache region.
This adapter also supports the <code>order</code> attribute which may be useful if it is bound to a PublishSubscribeChannel.
</para>
</section>
</chapter>

View File

@@ -135,7 +135,8 @@
<xi:include href="./ws.xml"/>
<xi:include href="./xml.xml"/>
<xi:include href="./xmpp.xml"/>
<xi:include href="./amqp.xml"/>
<xi:include href="./amqp.xml"/>
<xi:include href="./gemfire.xml"/>
</part>
<part id="spring-integration-appendices">
<title>Appendices</title>

View File

@@ -22,10 +22,6 @@ import java.util.Set;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.expression.Expression;
import org.springframework.expression.spel.standard.SpelExpressionParser;
import org.springframework.integration.endpoint.MessageProducerSupport;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.util.Assert;
@@ -47,7 +43,7 @@ import com.gemstone.gemfire.cache.util.CacheListenerAdapter;
* @since 2.1
*/
@SuppressWarnings({"rawtypes", "unchecked"})
public class CacheListeningMessageProducer extends MessageProducerSupport {
public class CacheListeningMessageProducer extends SpelMessageProducerSupport {
private final Log logger = LogFactory.getLog(this.getClass());
@@ -58,9 +54,6 @@ public class CacheListeningMessageProducer extends MessageProducerSupport {
private volatile Set<EventType> supportedEventTypes =
new HashSet<EventType>(Arrays.asList(EventType.CREATED, EventType.UPDATED));
private volatile Expression payloadExpression;
private final SpelExpressionParser parser = new SpelExpressionParser();
public CacheListeningMessageProducer(Region<?, ?> region) {
Assert.notNull(region, "region must not be null");
@@ -74,14 +67,6 @@ public class CacheListeningMessageProducer extends MessageProducerSupport {
this.supportedEventTypes = new HashSet<EventType>(Arrays.asList(eventTypes));
}
public void setPayloadExpression(String payloadExpression) {
if (payloadExpression == null) {
this.payloadExpression = null;
}
else {
this.payloadExpression = this.parser.parseExpression(payloadExpression);
}
}
@Override
protected void doStart() {
@@ -105,8 +90,7 @@ public class CacheListeningMessageProducer extends MessageProducerSupport {
}
}
private class MessageProducingCacheListener extends CacheListenerAdapter {
@Override
@@ -137,14 +121,9 @@ public class CacheListeningMessageProducer extends MessageProducerSupport {
}
}
private void processEvent(EntryEvent event) {
if (payloadExpression != null) {
Object evaluationResult = payloadExpression.getValue(event);
this.publish(evaluationResult);
}
else {
this.publish(event);
}
private void processEvent(EntryEvent event) {
this.publish(evaluationResult(event));
}
private void publish(Object payload) {

View File

@@ -16,15 +16,19 @@
package org.springframework.integration.gemfire.inbound;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.data.gemfire.listener.CqQueryDefinition;
import org.springframework.data.gemfire.listener.QueryListener;
import org.springframework.data.gemfire.listener.QueryListenerContainer;
import org.springframework.integration.Message;
import org.springframework.integration.endpoint.MessageProducerSupport;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.util.Assert;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import com.gemstone.gemfire.cache.query.CqEvent;
/**
@@ -38,17 +42,24 @@ import com.gemstone.gemfire.cache.query.CqEvent;
* @since 2.1
*
*/
public class ContinuousQueryMessageProducer extends MessageProducerSupport implements QueryListener {
public class ContinuousQueryMessageProducer extends SpelMessageProducerSupport implements QueryListener {
private static Log logger = LogFactory.getLog(ContinuousQueryMessageProducer.class);
private final String query;
private final QueryListenerContainer queryListenerContainer;
private volatile String queryName;
private boolean durable;
private volatile Set<CqEventType> supportedEventTypes = new HashSet<CqEventType>(Arrays.asList(CqEventType.CREATED,
CqEventType.UPDATED));
/**
*
* @param queryListenerContainer a {@link org.springframework.data.gemfire.listener.QueryListenerContainer}
* @param queryListenerContainer a
* {@link org.springframework.data.gemfire.listener.QueryListenerContainer}
* @param query the query string
*/
public ContinuousQueryMessageProducer(QueryListenerContainer queryListenerContainer, String query) {
@@ -57,7 +68,7 @@ public class ContinuousQueryMessageProducer extends MessageProducerSupport imple
this.queryListenerContainer = queryListenerContainer;
this.query = query;
}
/**
*
* @param queryName optional query name
@@ -65,7 +76,7 @@ public class ContinuousQueryMessageProducer extends MessageProducerSupport imple
public void setQueryName(String queryName) {
this.queryName = queryName;
}
/**
*
* @param durable true if the query is a durable subscription
@@ -74,16 +85,22 @@ public class ContinuousQueryMessageProducer extends MessageProducerSupport imple
this.durable = durable;
}
public void setSupportedEventTypes(CqEventType... eventTypes) {
Assert.notEmpty(eventTypes, "eventTypes must not be empty");
this.supportedEventTypes = new HashSet<CqEventType>(Arrays.asList(eventTypes));
}
@Override
protected void onInit() {
super.onInit();
if (queryName == null){
if (queryName == null) {
queryListenerContainer.addListener(new CqQueryDefinition(this.query, this, this.durable));
} else {
}
else {
queryListenerContainer.addListener(new CqQueryDefinition(this.queryName, this.query, this, this.durable));
}
}
/*
* (non-Javadoc)
*
@@ -92,11 +109,24 @@ public class ContinuousQueryMessageProducer extends MessageProducerSupport imple
* .gemfire.cache.query.CqEvent)
*/
public void onEvent(CqEvent event) {
if (logger.isDebugEnabled()){
logger.debug(String.format("processing cq event key [%s] event [%s]",event.getBaseOperation().toString(),event.getKey()));
if (isEventSupported(event)) {
if (logger.isDebugEnabled()) {
logger.debug(String.format("processing cq event key [%s] event [%s]", event.getBaseOperation()
.toString(), event.getKey()));
}
Message<?> cqEventMessage = MessageBuilder.withPayload(evaluationResult(event)).build();
sendMessage(cqEventMessage);
}
Message<CqEvent> cqEventMessage = MessageBuilder.withPayload(event).build();
sendMessage(cqEventMessage);
}
/**
* @param event
* @return
*/
private boolean isEventSupported(CqEvent event) {
String eventName = event.getBaseOperation().toString()+"D";
CqEventType eventType = CqEventType.valueOf(eventName);
return supportedEventTypes.contains(eventType);
}
}

View File

@@ -0,0 +1,30 @@
/*
* Copyright 2002-2011 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.springframework.integration.gemfire.inbound;
/**
* Enumeration of GemFire Continuous Query Event Types
* @author David Turanski
* @since 2.1
*/
public enum CqEventType {
CREATED,
UPDATED,
DESTROYED,
REGION_CLEARED,
REGION_INVALIDATED
}

View File

@@ -0,0 +1,54 @@
/*
* Copyright 2002-2011 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.springframework.integration.gemfire.inbound;
import org.springframework.expression.Expression;
import org.springframework.expression.spel.standard.SpelExpressionParser;
import org.springframework.integration.endpoint.MessageProducerSupport;
/**
* @author David Turanski
* @since 2.1
*
*/
abstract class SpelMessageProducerSupport extends MessageProducerSupport {
private volatile Expression payloadExpression;
private final SpelExpressionParser parser = new SpelExpressionParser();
@Override
protected void onInit(){
super.onInit();
}
public void setPayloadExpression(String payloadExpression) {
if (payloadExpression == null) {
this.payloadExpression = null;
}
else {
this.payloadExpression = this.parser.parseExpression(payloadExpression);
}
}
protected Object evaluationResult(Object payload){
Object evaluationResult = payload;
if (payloadExpression != null) {
evaluationResult = payloadExpression.getValue(payload);
}
return evaluationResult;
}
}

View File

@@ -1,2 +1,2 @@
http\://www.springframework.org/schema/integration/scripting/spring-integration-gemfire.xsd=org/springframework/integration/gemfire/config/xml/spring-integration-gemfire-2.1.xsd
http\://www.springframework.org/schema/integration/scripting/spring-integration-gemfire-2.1.xsd=org/springframework/integration/gemfire/config/xml/spring-integration-gemfire-2.1.xsd
http\://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire.xsd=org/springframework/integration/gemfire/config/xml/spring-integration-gemfire-2.1.xsd
http\://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire-2.1.xsd=org/springframework/integration/gemfire/config/xml/spring-integration-gemfire-2.1.xsd

View File

@@ -2,9 +2,9 @@
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:gfe="http://www.springframework.org/schema/gemfire"
xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xmlns:int="http://www.springframework.org/schema/integration"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/scripting/spring-integration-gemfire.xsd
xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire.xsd
http://www.springframework.org/schema/gemfire http://www.springframework.org/schema/gemfire/spring-gemfire.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

View File

@@ -23,7 +23,6 @@ import org.springframework.integration.MessagingException;
import org.springframework.integration.core.MessageHandler;
import org.springframework.integration.core.SubscribableChannel;
import org.springframework.integration.message.ErrorMessage;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

View File

@@ -3,9 +3,9 @@
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:gfe="http://www.springframework.org/schema/gemfire"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/scripting/spring-integration-gemfire.xsd
xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire.xsd
http://www.springframework.org/schema/gemfire http://www.springframework.org/schema/gemfire/spring-gemfire-1.1.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
@@ -23,14 +23,26 @@
<property name="cache" ref="client-cache"/>
</bean>
<bean class="org.springframework.integration.gemfire.inbound.ContinuousQueryMessageProducer">
<bean id="cqMessageProducer" class="org.springframework.integration.gemfire.inbound.ContinuousQueryMessageProducer">
<constructor-arg ref="queryListenerContainer"/>
<constructor-arg value="select * from /test"/>
<property name="outputChannel" ref="outputChannel"/>
<property name="outputChannel" ref="outputChannel1"/>
<property name="durable" value="true"/>
</bean>
<int:channel id="outputChannel">
<int:channel id="outputChannel1">
<int:queue/>
</int:channel>
<bean id="spelCqMessageProducer" class="org.springframework.integration.gemfire.inbound.ContinuousQueryMessageProducer">
<constructor-arg ref="queryListenerContainer"/>
<constructor-arg value="select * from /test"/>
<property name="outputChannel" ref="outputChannel2"/>
<property name="payloadExpression" value="newValue"/>
</bean>
<int:channel id="outputChannel2">
<int:queue/>
</int:channel>
</beans>

View File

@@ -12,6 +12,7 @@
*/
package org.springframework.integration.gemfire.inbound.cq;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
@@ -19,6 +20,7 @@ import java.io.IOException;
import java.io.OutputStream;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
@@ -41,6 +43,8 @@ import com.gemstone.gemfire.internal.cache.LocalRegion;
@ContextConfiguration
public class ContinuousQueryMessageProducerTests {
static ConfigurableApplicationContext staticCtx;
@Autowired
LocalRegion region;
@@ -48,7 +52,10 @@ public class ContinuousQueryMessageProducerTests {
ConfigurableApplicationContext applicationContext;
@Autowired
PollableChannel outputChannel;
PollableChannel outputChannel1;
@Autowired
PollableChannel outputChannel2;
static OutputStream os;
@BeforeClass
@@ -56,30 +63,36 @@ public class ContinuousQueryMessageProducerTests {
os = ForkUtil.cacheServer();
}
@Before
public void setUp() {
staticCtx = applicationContext;
}
@Test
public void test() throws InterruptedException {
public void testCqEvent() throws InterruptedException {
region.put("one",1);
Message<?> msg = outputChannel.receive(1000);
Message<?> msg = outputChannel1.receive(1000);
assertNotNull(msg);
assertTrue(msg.getPayload() instanceof CqEvent);
/*
* Avoid shutdown errors
*/
applicationContext.close();
}
@Test
public void testPayloadExpression() throws InterruptedException {
region.put("one",1);
Message<?> msg = outputChannel2.receive(1000);
assertNotNull(msg);
assertEquals(1,msg.getPayload());
}
@AfterClass
public static void cleanUp() {
try {
Thread.sleep(3000);
}
catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
/*
* Avoid shutdown errors
*/
staticCtx.close();
sendSignal();
}

View File

@@ -4,7 +4,7 @@
xmlns:int-gfe="http://www.springframework.org/schema/integration/gemfire"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:gfe="http://www.springframework.org/schema/gemfire"
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/scripting/spring-integration-gemfire.xsd
xsi:schemaLocation="http://www.springframework.org/schema/integration/gemfire http://www.springframework.org/schema/integration/gemfire/spring-integration-gemfire.xsd
http://www.springframework.org/schema/gemfire http://www.springframework.org/schema/gemfire/spring-gemfire-1.1.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">