Spring/spring-data-examples
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

#146 - Add example to demonstrate Redis Cluster support.

Browse Source 
Example demonstrating the basic usage of Spring Data Redis in a clustered environment using Jedis.
This commit is contained in:
Christoph Strobl
2015-12-21 17:51:23 +01:00
committed by Oliver Gierke
parent 9cb44b3166
commit dfc1bf0390
8 changed files with 400 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
redis/cluster/pom.xml Normal file
View File

@@ -0,0 +1,36 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>spring-data-redis-cluster-example</artifactId>
<name>Spring Data Redis - Cluster Example</name>
<parent>
<groupId>org.springframework.data.examples</groupId>
<artifactId>spring-data-redis-examples</artifactId>
<version>1.0.0.BUILD-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-redis</artifactId>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>spring-data-redis-example-utils</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<!-- remove this one when DATAREDIS-315 has been merged -->
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<version>1.7.0.DATAREDIS-315-SNAPSHOT</version>
</dependency>
</dependencies>
</project>

122
redis/cluster/readme.md Normal file
View File

@@ -0,0 +1,122 @@
# Spring Data Redis - Cluster Examples #
This project contains Redis 3 Cluster specific features of Spring Data Redis.
To run the code in this sample a running cluster environment is required. Please refer to the [redis cluster-tutorial](http://redis.io/topics/cluster-tutorial) for detailed information or check the Cluster Setup section below.
## Support for Cluster ##
Cluster Support uses the same building blocks as the non clustered counterpart. We use `application.properties` to point to an initial set of known cluster nodes.
```properties
spring.redis.cluster.nodes[0]=127.0.0.1:30001
spring.redis.cluster.nodes[1]=127.0.0.1:30002
spring.redis.cluster.nodes[2]=127.0.0.1:30003
```
Additionally we need to have the `RedisConnectionFactory` set up with the according `RedisClusterConfiguration`.
```java
@Configuration
@EnableConfigurationProperties(ClusterConfigurationProperties.class)
public class AppConfig {
/**
* Type safe representation of application.properties
*/
@Autowired ClusterConfigurationProperties clusterProperties;
/**
* The connection factory used for obtaining RedisConnection
* uses a RedisClusterConfiguration that points
* to the initial set of nodes.
*/
@Bean
RedisConnectionFactory connectionFactory() {
return new JedisConnectionFactory(
new RedisClusterConfiguration(clusterProperties.getNodes()));
}
/**
* RedisTemplate can be configured with RedisSerializer if needed.
* NOTE: be careful using JSON serializers for key serialization.
*/
@Bean
RedisTemplate<String, String> redisTemplate() {
return new StringRedisTemplate(connectionFactory());
}
}
```
**INFORMATION:** The tests flush the db of all known instances during the JUnit _setup_ phase to allow inspecting data directly on the cluster nodes after a test is run.
## Cluster Setup ##
To quickly set up a cluster of 6 nodes (3 master | 3 slave) go to the `redis/utils/create-cluster` directory.
```bash
redis/utils/create-cluster $ ./create-cluster start
Starting 30001
Starting 30002
Starting 30003
Starting 30004
Starting 30005
Starting 30006
```
On first initialization cluster nodes need to form the cluster by joining and assigning slot allocations.
**INFO**: This has to be done only once.
```bash
redis/utils/create-cluster $ ./create-cluster create
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:30001
127.0.0.1:30002
127.0.0.1:30003
Adding replica 127.0.0.1:30004 to 127.0.0.1:30001
Adding replica 127.0.0.1:30005 to 127.0.0.1:30002
Adding replica 127.0.0.1:30006 to 127.0.0.1:30003
M: 10696916f57e58c5edce34127b23ca7af1b669a0 127.0.0.1:30001
slots:0-5460 (5461 slots) master
M: 5b0e1b4cc87175326ba79d00ecfc6f5dbdb424a7 127.0.0.1:30002
slots:5461-10922 (5462 slots) master
M: 5f3e978fb40b1d9c910d904ea19a0494b78668aa 127.0.0.1:30003
slots:10923-16383 (5461 slots) master
S: d1717c418d03db93183ce2d791ba6f48be5cf028 127.0.0.1:30004
replicates 10696916f57e58c5edce34127b23ca7af1b669a0
S: c7dfcdb9cd1105e4251de51c4ade54de59bb063c 127.0.0.1:30005
replicates 5b0e1b4cc87175326ba79d00ecfc6f5dbdb424a7
S: 3219785a9145717f30648a27a2dd07359e9dd46f 127.0.0.1:30006
replicates 5f3e978fb40b1d9c910d904ea19a0494b78668aa
Can I set the above configuration? (type 'yes' to accept): yes
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
```
It is now possible to connect to the cluster using the `redis-cli`.
```bash
redis/src $ ./redis-cli -c -p 30001
127.0.0.1:30001> cluster nodes
106969... 127.0.0.1:30001 myself,master - 0 0 1 connected 0-5460
5b0e1b... 127.0.0.1:30002 master - 0 1450765112345 2 connected 5461-10922
5f3e97... 127.0.0.1:30003 master - 0 1450765112345 3 connected 10923-16383
d1717c... 127.0.0.1:30004 slave 106969... 0 1450765112345 4 connected
c7dfcd... 127.0.0.1:30005 slave 5b0e1b... 0 1450765113050 5 connected
321978... 127.0.0.1:30006 slave 5f3e97... 0 1450765113050 6 connected
```
To shutdown the cluster use the `create-cluster stop` command.
```bash
redis/utils/create-cluster $ ./create-cluster stop
```

62
redis/cluster/src/main/java/example/springdata/redis/cluster/AppConfig.java Normal file
View File

@@ -0,0 +1,62 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package example.springdata.redis.cluster;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisClusterConfiguration;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.RedisSerializer;
/**
* Application context configuration setting up {@link RedisConnectionFactory} and {@link RedisTemplate} according to
* {@link ClusterConfigurationProperties}.
*
* @author Christoph Strobl
*/
@Configuration
@EnableConfigurationProperties(ClusterConfigurationProperties.class)
public class AppConfig {
/**
* Type safe representation of application.properties
*/
@Autowired ClusterConfigurationProperties clusterProperties;
/**
* The connection factory used for obtaining {@link RedisConnection} uses a {@link RedisClusterConfiguration} that
* points to the initial set of nodes.
*/
@Bean
RedisConnectionFactory connectionFactory() {
return new JedisConnectionFactory(new RedisClusterConfiguration(clusterProperties.getNodes()));
}
/**
* {@link RedisTemplate} can be configured with {@link RedisSerializer} if needed. <br />
* <b>NOTE:</b> be careful using JSON @link RedisSerializer} for key serialization.
*/
@Bean
RedisTemplate<String, String> redisTemplate() {
return new StringRedisTemplate(connectionFactory());
}
}

47
redis/cluster/src/main/java/example/springdata/redis/cluster/ClusterConfigurationProperties.java Normal file
View File

@@ -0,0 +1,47 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package example.springdata.redis.cluster;
import java.util.List;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
/**
* Type safe representation of {@code spring.redis.cluster.*} properties in {@literal application.properties}.
*
* @author Christoph Strobl
*/
@Component
@ConfigurationProperties(prefix = "spring.redis.cluster")
public class ClusterConfigurationProperties {
List<String> nodes;
/**
* Get initial collection of known cluster nodes in format {@code host:port}.
*
* @return
*/
public List<String> getNodes() {
return nodes;
}
public void setNodes(List<String> nodes) {
this.nodes = nodes;
}
}

122
redis/cluster/src/test/java/example/springdata/redis/cluster/BasicUsageTests.java Normal file
View File

@@ -0,0 +1,122 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package example.springdata.redis.cluster;
import static org.hamcrest.core.Is.*;
import static org.hamcrest.core.IsCollectionContaining.*;
import static org.junit.Assert.*;
import java.util.Arrays;
import org.junit.Before;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.dao.DataAccessException;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import example.springdata.redis.test.util.RequiresRedisServer;
/**
* {@link BasicUsageTests} shows general usage of {@link RedisTemplate} and {@link RedisOperations} in a clustered
* environment.
*
* @author Christoph Strobl
*/
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = { AppConfig.class })
public class BasicUsageTests {
@Autowired RedisTemplate<String, String> template;
public static @ClassRule RequiresRedisServer redisServerAvailable = RequiresRedisServer.listeningAt("127.0.0.1",
30001);
@Before
public void setUp() {
template.execute(new RedisCallback<String>() {
@Override
public String doInRedis(RedisConnection connection) throws DataAccessException {
connection.flushDb();
return "FLUSHED";
}
});
}
/**
* Operation executed on a single node and slot. <br />
* -&gt; {@code SLOT 5798} served by {@code 127.0.0.1:30002}
*/
@Test
public void singleSlotOperation() {
template.opsForValue().set("name", "rand al'thor"); // slot 5798
assertThat(template.opsForValue().get("name"), is("rand al'thor"));
}
/**
* Operation executed on multiple nodes and slots. <br />
* -&gt; {@code SLOT 5798} served by {@code 127.0.0.1:30002} <br />
* -&gt; {@code SLOT 14594} served by {@code 127.0.0.1:30003}
*/
@Test
public void multiSlotOperation() {
template.opsForValue().set("name", "matrim cauthon"); // slot 5798
template.opsForValue().set("nickname", "prince of the ravens"); // slot 14594
assertThat(template.opsForValue().multiGet(Arrays.asList("name", "nickname")),
hasItems("matrim cauthon", "prince of the ravens"));
}
/**
* Operation executed on a single node and slot because of pinned slot key <br />
* -&gt; {@code SLOT 5798} served by {@code 127.0.0.1:30002}
*/
@Test
public void fixedSlotOperation() {
template.opsForValue().set("{user}.name", "perrin aybara"); // slot 5474
template.opsForValue().set("{user}.nickname", "wolfbrother"); // slot 5474
assertThat(template.opsForValue().multiGet(Arrays.asList("{user}.name", "{user}.nickname")),
hasItems("perrin aybara", "wolfbrother"));
}
/**
* Operation executed across the cluster to retrieve cumulated result. <br />
* -&gt; {@code KEY age} served by {@code 127.0.0.1:30001} <br />
* -&gt; {@code KEY name} served by {@code 127.0.0.1:30002} <br />
* -&gt; {@code KEY nickname} served by {@code 127.0.0.1:30003}
*/
@Test
public void multiNodeOperation() {
template.opsForValue().set("name", "rand al'thor"); // slot 5798
template.opsForValue().set("nickname", "dragon reborn"); // slot 14594
template.opsForValue().set("age", "23"); // slot 741;
assertThat(template.keys("*"), hasItems("name", "nickname", "age"));
}
}

3
redis/cluster/src/test/resources/application.properties Normal file
View File

@@ -0,0 +1,3 @@
spring.redis.cluster.nodes[0]=127.0.0.1:30001
spring.redis.cluster.nodes[1]=127.0.0.1:30002
spring.redis.cluster.nodes[2]=127.0.0.1:30003

3
redis/pom.xml
View File

@@ -18,6 +18,7 @@
<module>util</module>
<module>cluster-sentinel</module>
<module>example</module>
<module>cluster</module>
</modules>
<dependencies>
@@ -30,7 +31,7 @@
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.7.2</version>
<version>2.8.0</version>
</dependency>
</dependencies>

7
redis/util/src/main/java/example/springdata/redis/test/util/RequiresRedisServer.java
View File

@@ -18,8 +18,9 @@ package example.springdata.redis.test.util;
import java.net.InetSocketAddress;
import java.net.Socket;
import org.junit.internal.AssumptionViolatedException;
import org.junit.AssumptionViolatedException;
import org.junit.rules.ExternalResource;
import org.springframework.util.StringUtils;
/**
* Implementation of junit rule {@link ExternalResource} to verify Redis (or at least something on the defined host and
@@ -44,6 +45,10 @@ public class RequiresRedisServer extends ExternalResource {
return new RequiresRedisServer("localhost", 6379);
}
public static RequiresRedisServer listeningAt(String host, int port) {
return new RequiresRedisServer(StringUtils.hasText(host) ? host : "127.0.0.1", port);
}
/*
* (non-Javadoc)
* @see org.junit.rules.ExternalResource#before()