DATAREDIS-682 - Connect to Redis using unix domain sockets.

We now support Redis connections using OS-native transports through unix domain sockets when using the Lettuce driver. Domain sockets do not require a roundtrip through the networking layer but directly use native epoll or kqueue interfaces.

LettuceConnectionFactory factory = new LettuceConnectionFactory(new RedisSocketConfiguration("/var/run/redis.sock"));

Unix domain socket support requires netty's native epoll/kqueue dependencies matching the runtime environment:

<dependency>
	<groupId>io.netty</groupId>
	<artifactId>netty-transport-native-epoll</artifactId>
	<classifier>linux-x86_64</classifier>
	<version>${netty}</version>
</dependency>

<dependency>
	<groupId>io.netty</groupId>
	<artifactId>netty-transport-native-kqueue</artifactId>
	<classifier>osx-x86_64</classifier>
	<version>${netty}</version>
</dependency>

Original Pull Request: #286
This commit is contained in:
Mark Paluch
2017-10-13 10:46:04 +02:00
committed by Christoph Strobl
parent 5c16a6044c
commit d08ab93421
10 changed files with 386 additions and 53 deletions

View File

@@ -29,6 +29,8 @@ work/redis-%.conf:
echo notify-keyspace-events Ex >> $@
echo pidfile $(shell pwd)/work/redis-$*.pid >> $@
echo logfile $(shell pwd)/work/redis-$*.log >> $@
echo unixsocket $(shell pwd)/work/redis-$*.sock >> $@
echo unixsocketperm 755 >> $@
echo save \"\" >> $@
echo slaveof 127.0.0.1 6379 >> $@
@@ -42,6 +44,8 @@ work/redis-6379.conf:
echo notify-keyspace-events Ex >> $@
echo pidfile $(shell pwd)/work/redis-6379.pid >> $@
echo logfile $(shell pwd)/work/redis-6379.log >> $@
echo unixsocket $(shell pwd)/work/redis-6379.sock >> $@
echo unixsocketperm 755 >> $@
echo save \"\" >> $@
work/redis-%.pid: work/redis-%.conf work/redis/bin/redis-server
@@ -122,7 +126,7 @@ cluster-init: cluster-start cluster-meet cluster-slots
# Global
########
clean:
rm -rf work/*.conf work/*.log
rm -rf work/*.conf work/*.log dump.rdb
clobber:
rm -rf work

17
pom.xml
View File

@@ -24,6 +24,7 @@
<lettuce>5.0.1.RELEASE</lettuce>
<jedis>2.9.0</jedis>
<multithreadedtc>1.01</multithreadedtc>
<netty>4.1.15.Final</netty>
<java-module-name>spring.data.redis</java-module-name>
</properties>
@@ -85,6 +86,22 @@
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<classifier>linux-x86_64</classifier>
<version>${netty}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-kqueue</artifactId>
<classifier>osx-x86_64</classifier>
<version>${netty}</version>
<scope>test</scope>
</dependency>
<!-- reactive -->
<dependency>
<groupId>io.projectreactor</groupId>

View File

@@ -3,7 +3,11 @@
New and noteworthy in the latest releases.
[[new-in-2.0.0]]
[[new-in-2.1.0]]
== New in Spring Data Redis 2.1
* Unix domain socket connections.
== New in Spring Data Redis 2.0
* Upgrade to Java 8.

View File

@@ -39,64 +39,78 @@ NOTE: Depending on the underlying configuration, the factory can return a new co
The easiest way to work with a `RedisConnectionFactory` is to configure the appropriate connector through the IoC container and inject it into the using class.
IMPORTANT: Unfortunately, currently, not all connectors support all Redis features. When invoking a method on the Connection API that is unsupported by the underlying library, an `UnsupportedOperationException` is thrown.
This situation is likely to be fixed in the future, as the various connectors mature.
[[redis:connectors:jedis]]
=== Configuring Jedis connector
http://github.com/xetorthio/jedis[Jedis] is one of the connectors supported by the Spring Data Redis module through the `org.springframework.data.redis.connection.jedis` package. In its simplest form, the Jedis configuration looks as follow:
[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- Jedis ConnectionFactory -->
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"/>
</beans>
----
For production use however, one might want to tweak the settings such as the host or password:
[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory" p:host-name="server" p:port="6379" />
</beans>
----
IMPORTANT: Unfortunately, currently, not all connectors support all Redis features. When invoking a method on the Connection API that is unsupported by the underlying library, an `UnsupportedOperationException` is thrown.
[[redis:connectors:lettuce]]
=== Configuring Lettuce connector
https://github.com/mp911de/lettuce[Lettuce] is a http://netty.io/[netty]-based open-source connector supported by Spring Data Redis through the `org.springframework.data.redis.connection.lettuce` package.
https://github.com/lettuce-io/lettuce-core[Lettuce] is a http://netty.io/[netty]-based open-source connector supported by Spring Data Redis through the `org.springframework.data.redis.connection.lettuce` package.
Its configuration is probably easy to guess:
[source,xml]
[source,java]
----
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
@Configuration
class AppConfig {
<bean id="lettuceConnectionFactory" class="org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory" p:host-name="server" p:port="6379"/>
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
</beans>
return new LettuceConnectionFactory(new RedisStandaloneConfiguration("server", 6379));
}
}
----
There are also a few Lettuce-specific connection parameters that can be tweaked. By default, all `LettuceConnection` s created by the `LettuceConnectionFactory` share the same thread-safe native connection for all non-blocking and non-transactional operations. Set `shareNativeConnection` to false to use a dedicated connection each time. `LettuceConnectionFactory` can also be configured with a `LettucePool` to use for pooling blocking and transactional connections, or all connections if `shareNativeConnection` is set to false.
Lettuce integrates with netty's http://netty.io/wiki/native-transports.html[native transports] allowing to use unix domain sockets to communicate with Redis. Make sure to include the appropriate native transport dependencies that match your runtime environment.
[source,java]
----
@Configuration
class AppConfig {
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory(new RedisSocketConfiguration("/var/run/redis.sock"));
}
}
----
NOTE: Netty currently supports epoll (Linux) and kqueue (BSD/macOS) interfaces for OS-native transport.
[[redis:connectors:jedis]]
=== Configuring Jedis connector
http://github.com/xetorthio/jedis[Jedis] is a community-driven connector supported by the Spring Data Redis module through the `org.springframework.data.redis.connection.jedis` package. In its simplest form, the Jedis configuration looks as follow:
[source,java]
----
@Configuration
class AppConfig {
@Bean
public JedisConnectionFactory redisConnectionFactory() {
return new JedisConnectionFactory();
}
}
----
For production use however, one might want to tweak the settings such as the host or password:
[source,java]
----
@Configuration
class RedisConfiguration {
@Bean
public JedisConnectionFactory redisConnectionFactory() {
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration("server", 6379);
return new JedisConnectionFactory(config);
}
}
----
[[redis:sentinel]]
== Redis Sentinel Support
@@ -105,7 +119,7 @@ For dealing with high available Redis there is support for http://redis.io/topic
[source,java]
----
/**
* jedis
* Jedis
*/
@Bean
public RedisConnectionFactory jedisConnectionFactory() {

View File

@@ -0,0 +1,101 @@
/*
* Copyright 2017 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.redis.connection;
import org.springframework.util.Assert;
/**
* Configuration class used for setting up {@link RedisConnection} via {@link RedisConnectionFactory} by connecting to a
* single node <a href="http://redis.io/">Redis</a> using a local unix domain socket.
*
* @author Mark Paluch
* @since 2.1
*/
public class RedisSocketConfiguration {
private static final String DEFAULT_SOCKET = "/tmp/redis.sock";
private String socket = DEFAULT_SOCKET;
private int database;
private RedisPassword password = RedisPassword.none();
/**
* Create a new default {@link RedisSocketConfiguration}.
*/
public RedisSocketConfiguration() {}
/**
* Create a new {@link RedisSocketConfiguration} given {@code socket}.
*
* @param socket must not be {@literal null} or empty.
*/
public RedisSocketConfiguration(String socket) {
Assert.hasText(socket, "Socket path must not be null or empty!");
this.socket = socket;
}
/**
* @return path to the Redis socket.
*/
public String getSocket() {
return socket;
}
/**
* @param socket path to the Redis socket.
*/
public void setSocket(String socket) {
this.socket = socket;
}
/**
* @return the db index.
*/
public int getDatabase() {
return database;
}
/**
* Sets the index of the database used by this connection factory. Default is 0.
*
* @param index database index.
*/
public void setDatabase(int index) {
Assert.isTrue(index >= 0, () -> String.format("Invalid DB index '%s' (a positive index required)", index));
this.database = index;
}
/**
* @return never {@literal null}.
*/
public RedisPassword getPassword() {
return password;
}
/**
* @param password must not be {@literal null}.
*/
public void setPassword(RedisPassword password) {
Assert.notNull(password, "RedisPassword must not be null!");
this.password = password;
}
}

View File

@@ -67,6 +67,7 @@ import org.springframework.util.ClassUtils;
* {@link LettuceConnectionFactory client configuration}. Lettuce supports the following environmental configurations:
* <ul>
* <li>{@link RedisStandaloneConfiguration}</li>
* <li>{@link RedisSocketConfiguration}</li>
* <li>{@link RedisSentinelConfiguration}</li>
* <li>{@link RedisClusterConfiguration}</li>
* </ul>
@@ -99,6 +100,7 @@ public class LettuceConnectionFactory
private final Object connectionMonitor = new Object();
private boolean convertPipelineAndTxResults = true;
private RedisStandaloneConfiguration standaloneConfig = new RedisStandaloneConfiguration("localhost", 6379);
private @Nullable RedisSocketConfiguration socketConfiguration;
private @Nullable RedisSentinelConfiguration sentinelConfiguration;
private @Nullable RedisClusterConfiguration clusterConfiguration;
private @Nullable ClusterCommandExecutor clusterCommandExecutor;
@@ -138,7 +140,17 @@ public class LettuceConnectionFactory
}
/**
* Constructs a new {@link LettuceConnectionFactory} instance using the given {@link RedisSentinelConfiguration}
* Constructs a new {@link LettuceConnectionFactory} instance using the given {@link RedisSocketConfiguration}.
*
* @param socketConfiguration must not be {@literal null}.
* @since 2.1
*/
public LettuceConnectionFactory(RedisSocketConfiguration socketConfiguration) {
this(socketConfiguration, new MutableLettuceClientConfiguration());
}
/**
* Constructs a new {@link LettuceConnectionFactory} instance using the given {@link RedisSentinelConfiguration}.
*
* @param sentinelConfiguration must not be {@literal null}.
* @since 1.6
@@ -186,6 +198,24 @@ public class LettuceConnectionFactory
this.standaloneConfig = standaloneConfig;
}
/**
* Constructs a new {@link LettuceConnectionFactory} instance using the given {@link RedisSocketConfiguration} and
* {@link LettuceClientConfiguration}.
*
* @param socketConfiguration must not be {@literal null}.
* @param clientConfig must not be {@literal null}.
* @since 2.1
*/
public LettuceConnectionFactory(RedisSocketConfiguration socketConfiguration,
LettuceClientConfiguration clientConfig) {
this(clientConfig);
Assert.notNull(socketConfiguration, "RedisSocketConfiguration must not be null!");
this.socketConfiguration = socketConfiguration;
}
/**
* Constructs a new {@link LettuceConnectionFactory} instance using the given {@link RedisSentinelConfiguration} and
* {@link LettuceClientConfiguration}.
@@ -588,6 +618,10 @@ public class LettuceConnectionFactory
*/
public int getDatabase() {
if (isDomainSocketAware()) {
return socketConfiguration.getDatabase();
}
if (isRedisSentinelAware()) {
return sentinelConfiguration.getDatabase();
}
@@ -604,6 +638,11 @@ public class LettuceConnectionFactory
Assert.isTrue(index >= 0, "invalid DB index (a positive index required)");
if (isDomainSocketAware()) {
socketConfiguration.setDatabase(index);
return;
}
if (isRedisSentinelAware()) {
sentinelConfiguration.setDatabase(index);
return;
@@ -648,6 +687,10 @@ public class LettuceConnectionFactory
private RedisPassword getRedisPassword() {
if (isDomainSocketAware()) {
return socketConfiguration.getPassword();
}
if (isRedisSentinelAware()) {
return sentinelConfiguration.getPassword();
}
@@ -669,6 +712,11 @@ public class LettuceConnectionFactory
@Deprecated
public void setPassword(String password) {
if (isDomainSocketAware()) {
socketConfiguration.setPassword(RedisPassword.of(password));
return;
}
if (isRedisSentinelAware()) {
sentinelConfiguration.setPassword(RedisPassword.of(password));
return;
@@ -745,6 +793,15 @@ public class LettuceConnectionFactory
return standaloneConfig;
}
/**
* @return the {@link RedisSocketConfiguration}.
* @since 2.1
*/
@Nullable
public RedisSocketConfiguration getSocketConfiguration() {
return socketConfiguration;
}
/**
* @return the {@link RedisStandaloneConfiguration}, may be {@literal null}.
* @since 2.0
@@ -793,6 +850,14 @@ public class LettuceConnectionFactory
return sentinelConfiguration != null;
}
/**
* @return true when {@link RedisSocketConfiguration} is present.
* @since 2.1
*/
private boolean isDomainSocketAware() {
return socketConfiguration != null;
}
/**
* @return true when {@link RedisClusterConfiguration} is present.
* @since 1.7
@@ -833,6 +898,7 @@ public class LettuceConnectionFactory
}
private LettuceConnectionProvider doConnectionProvider(AbstractRedisClient client, RedisCodec<?, ?> codec) {
if (isClusterAware()) {
return new ClusterConnectionProvider((RedisClusterClient) client, codec);
}
@@ -871,7 +937,8 @@ public class LettuceConnectionFactory
return clusterClient;
}
RedisURI uri = createRedisURIAndApplySettings(getHostName(), getPort());
RedisURI uri = isDomainSocketAware() ? createRedisSocketURIAndApplySettings(socketConfiguration.getSocket())
: createRedisURIAndApplySettings(getHostName(), getPort());
RedisClient redisClient = clientConfiguration.getClientResources() //
.map(clientResources -> RedisClient.create(clientResources, uri)) //
.orElseGet(() -> RedisClient.create(uri));
@@ -906,6 +973,16 @@ public class LettuceConnectionFactory
return builder.build();
}
private RedisURI createRedisSocketURIAndApplySettings(String socketPath) {
RedisURI.Builder builder = RedisURI.Builder.socket(socketPath);
getRedisPassword().toOptional().ifPresent(builder::withPassword);
builder.withTimeout(clientConfiguration.getCommandTimeout());
return builder.build();
}
@Override
public RedisSentinelConnection getSentinelConnection() {
return new LettuceSentinelConnection(connectionProvider);

View File

@@ -1,5 +1,5 @@
/*
* Copyright 2011-2013 the original author or authors.
* Copyright 2011-2017 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -17,19 +17,25 @@ package org.springframework.data.redis;
import java.util.Properties;
import org.springframework.data.redis.connection.RedisSocketConfiguration;
import org.springframework.data.redis.connection.RedisStandaloneConfiguration;
/**
* Utility class exposing connection settings to connect Redis instances during test execution. Settings can be adjusted
* by overriding these in {@literal org/springframework/data/redis/test.properties}.
*
* @author Costin Leau
* @author Mark Paluch
*/
public abstract class SettingsUtils {
private final static Properties DEFAULTS = new Properties();
private static final Properties SETTINGS;
static {
DEFAULTS.put("host", "127.0.0.1");
DEFAULTS.put("port", "6379");
DEFAULTS.put("socket", "work/redis-6379.sock");
SETTINGS = new Properties(DEFAULTS);
@@ -40,15 +46,44 @@ public abstract class SettingsUtils {
}
}
private SettingsUtils() {}
/**
* @return the Redis hostname.
*/
public static String getHost() {
return SETTINGS.getProperty("host");
}
/**
* @return the Redis port.
*/
public static int getPort() {
return Integer.valueOf(SETTINGS.getProperty("port"));
}
/**
* @return path to the unix domain socket.
*/
public static String getSocket() {
return SETTINGS.getProperty("socket");
}
/**
* Construct a new {@link RedisStandaloneConfiguration} initialized with test endpoint settings.
*
* @return a new {@link RedisStandaloneConfiguration} initialized with test endpoint settings.
*/
public static RedisStandaloneConfiguration standaloneConfiguration() {
return new RedisStandaloneConfiguration(getHost(), getPort());
}
/**
* Construct a new {@link RedisSocketConfiguration} initialized with test endpoint settings.
*
* @return a new {@link RedisSocketConfiguration} initialized with test endpoint settings.
*/
public static RedisSocketConfiguration socketConfiguration() {
return new RedisSocketConfiguration(getSocket());
}
}

View File

@@ -19,11 +19,15 @@ import static org.hamcrest.core.Is.*;
import static org.hamcrest.core.IsEqual.*;
import static org.hamcrest.core.IsNull.*;
import static org.junit.Assert.*;
import static org.junit.Assume.*;
import io.lettuce.core.EpollProvider;
import io.lettuce.core.KqueueProvider;
import io.lettuce.core.RedisException;
import io.lettuce.core.api.async.RedisAsyncCommands;
import io.lettuce.core.api.reactive.BaseRedisReactiveCommands;
import java.io.File;
import java.time.Duration;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
@@ -362,6 +366,25 @@ public class LettuceConnectionFactoryTests {
factory.destroy();
}
@Test // DATAREDIS-687
public void connectsThroughRedisSocket() {
assumeTrue(EpollProvider.isAvailable() || KqueueProvider.isAvailable());
assumeTrue(new File(SettingsUtils.getSocket()).exists());
LettuceClientConfiguration configuration = LettuceTestClientConfiguration.create();
LettuceConnectionFactory factory = new LettuceConnectionFactory(SettingsUtils.socketConfiguration(), configuration);
factory.setShareNativeConnection(false);
factory.afterPropertiesSet();
RedisConnection connection = factory.getConnection();
assertThat(connection.ping(), is(equalTo("PONG")));
connection.close();
factory.destroy();
}
@Test // DATAREDIS-576
public void connectionAppliesClientName() {

View File

@@ -43,6 +43,7 @@ import org.springframework.data.redis.connection.RedisClusterConfiguration;
import org.springframework.data.redis.connection.RedisClusterConnection;
import org.springframework.data.redis.connection.RedisPassword;
import org.springframework.data.redis.connection.RedisSentinelConfiguration;
import org.springframework.data.redis.connection.RedisSocketConfiguration;
import org.springframework.data.redis.connection.RedisStandaloneConfiguration;
import org.springframework.test.util.ReflectionTestUtils;
@@ -285,6 +286,21 @@ public class LettuceConnectionFactoryUnitTests {
}
}
@Test // DATAREDIS-682
public void socketShouldBeSetOnStandaloneClient() {
LettuceConnectionFactory connectionFactory = new LettuceConnectionFactory(new RedisSocketConfiguration());
connectionFactory.afterPropertiesSet();
ConnectionFactoryTracker.add(connectionFactory);
AbstractRedisClient client = (AbstractRedisClient) getField(connectionFactory, "client");
assertThat(client, instanceOf(RedisClient.class));
RedisURI redisUri = (RedisURI) getField(client, "redisURI");
assertThat(redisUri.getSocket(), is("/tmp/redis.sock"));
}
@Test // DATAREDIS-574
public void shouldReadStandalonePassword() {
@@ -337,6 +353,20 @@ public class LettuceConnectionFactoryUnitTests {
assertThat(envConfig.getPassword(), is(equalTo(RedisPassword.of("bar"))));
}
@Test // DATAREDIS-682
public void shouldWriteSocketPassword() {
RedisSocketConfiguration envConfig = new RedisSocketConfiguration();
envConfig.setPassword(RedisPassword.of("foo"));
LettuceConnectionFactory connectionFactory = new LettuceConnectionFactory(envConfig,
LettuceClientConfiguration.defaultConfiguration());
connectionFactory.setPassword("bar");
assertThat(connectionFactory.getPassword(), is(equalTo("bar")));
assertThat(envConfig.getPassword(), is(equalTo(RedisPassword.of("bar"))));
}
@Test // DATAREDIS-574
public void shouldReadClusterPassword() {
@@ -415,6 +445,20 @@ public class LettuceConnectionFactoryUnitTests {
assertThat(envConfig.getDatabase(), is(3));
}
@Test // DATAREDIS-682
public void shouldWriteSocketDatabaseIndex() {
RedisSocketConfiguration envConfig = new RedisSocketConfiguration();
envConfig.setDatabase(2);
LettuceConnectionFactory connectionFactory = new LettuceConnectionFactory(envConfig,
LettuceClientConfiguration.defaultConfiguration());
connectionFactory.setDatabase(3);
assertThat(connectionFactory.getDatabase(), is(3));
assertThat(envConfig.getDatabase(), is(3));
}
@Test // DATAREDIS-574
public void shouldApplyClientConfiguration() {
@@ -456,6 +500,19 @@ public class LettuceConnectionFactoryUnitTests {
assertThat(connectionFactory.getClusterConfiguration(), is(nullValue()));
}
@Test // DATAREDIS-682
public void shouldReturnSocketConfiguration() {
RedisSocketConfiguration configuration = new RedisSocketConfiguration("/var/redis/socket");
LettuceConnectionFactory connectionFactory = new LettuceConnectionFactory(configuration,
LettuceClientConfiguration.defaultConfiguration());
assertThat(connectionFactory.getStandaloneConfiguration(), is(notNullValue()));
assertThat(connectionFactory.getSocketConfiguration(), is(configuration));
assertThat(connectionFactory.getSentinelConfiguration(), is(nullValue()));
assertThat(connectionFactory.getClusterConfiguration(), is(nullValue()));
}
@Test // DATAREDIS-574
public void shouldReturnSentinelConfiguration() {

View File

@@ -1,3 +1,4 @@
# redis connection properties
host=127.0.0.1
port=6379
port=6379
socket=work/redis-6379.sock