Commit Graph

1086 Commits

Author SHA1 Message Date
Christoph Strobl
bd7728e6ad DATAREDIS-425 - Add Support for basic CRUD and finder Operations backed by Hashes and Sets.
We now enable storing domain object as a flat Redis 'HASH' and maintain additional 'SET' structures to enable finder operations on simple properties.

  @RedisHash("persons");
  class Person {

    @id String id;
    @Indexed String firstname;
    String lastname;

    Map<String, String> attributes;

    City city;

    @Reference Person mother;
  }

The above is stored in the HASH with key 'persons:1' as

  _class = org.example.Person
  id = 1
  firstname = rand
  lastname = al’thor
  attributes.[eye-color] = grey
  attributes.[hair-color] = red
  city.name = emond's field
  city.region = two rivers
  mother = persons:2

Complex types are flattened out to their full property path for each of the values provided. If the properties actual value type does not match the declared one the '_class' type hint is added to the entry.

  city._class = CityInAndor.class
  city.name = emond's field
  city.region = two rivers
  city.country = andor

Map and Collection like structures are stored with their key/index values as part of the property path. If the map/collection value type does not match the actutal objects one the '_class' type hint is added to the entry.

  list.[0]._class = DomainType.class
  list.[0].property1 = ...

  map.[key-1]._class = DomainType.class
  map.[key-1].property1 = ...

Properties marked with '@Reference' are stored as semantic references by just storing the key to the referenced object 'HASH' instead of embedding its values.

   mother = persons:2

Please note that referenced objects are not transitively updated/saved and that lazy loading of references will be part of future development.

A 'save' operation therefore executes the following:

  # flatten domain type and add as hash
  HMSET persons:1 id 1 firstname rand …

  # add the newly inserted entry to the list of all entries of that type
  SADD persons 1

  # index the firstname for finder lookup
  SADD persons.firstname:rand 1

Simple finder operation like 'findByFirstname' use 'SINTER' to find matching

  SINTER persons.firstname:rand
  HGETALL persons:1

Besides resolving an index via the '@Index' annotation we also allow to add custom configuration via the 'indexConfiguration' attribute of '@EnableRedisRepositories'.

  @Configuration
  @EnableRedisRepositories(indexConfiguration = CustomIndexConfiguration.class)
  class Config { }

  static class CustomIndexConfiguration extends IndexConfiguration {

    @Override
    protected Iterable<RedisIndexDefinition> initialConfiguration() {
      return Arrays.asList(
        new SimpleIndexDefinition("persons", "lastname"),
      );
    }
  }

The '@TimeToLive' annotation allows to define a property or method providing an expiration time when storing the key in redis.

@RedisHash
class Person {

  @Id String id;
  @TimeToLive Long ttl;
}

Original Pull Request: #156
2016-03-14 10:37:53 +01:00
Christoph Strobl
5a85ac47a3 DATAREDIS-472 - Add guards to JedisConnection before casting long to int. 2016-03-04 14:19:41 +01:00
Milan Agatonovic
b630744564 DATAREDIS-472 - Remove guards on pExpire & pSetEx in JedisConnection for values exceeding Integer.MAX_VALUE.
Remove guards and use native driver api for pexpire and psetex which is available in the current jedis distribution.

Original Pull Request: #175
CLA: 165820160303082625 (Milan Agatonovic)
2016-03-04 13:26:05 +01:00
Mark Paluch
0f035982b2 DATAREDIS-468 - Polishing.
Add DecoratedRedisConnection interface to unwrap decorated connections. When running on Redis Cluster, use keys/del instead of lua script because cluster connections do not support lua script execution.

Original pull request: #173.
2016-02-26 09:07:07 +01:00
Christoph Strobl
7024ef33e0 DATAREDIS-468 - Guard multi/exec bocks in RedisCache to allow usage with cluster.
We now guard multi/exec blocks by checking the connection type. This allows to skip those commands when running in cluster.

Original pull request: #173.
2016-02-26 09:07:02 +01:00
Christoph Strobl
02742e3b49 DATAREDIS-467 - Fix cluster multi key commands.
We moved away from returning raw map types and now return dedicated objects for command executions in cluster environment. This allows to maintain node information and collecting results from each and every callback. MGET now returns values according to the key position.

Additionally we now prefix info commands in the cluster with the nodes host:port.

Original pull request: #174.
2016-02-26 08:46:05 +01:00
Christoph Strobl
1e5772d00b DATAREDIS-462 - Polishing.
Updated javadoc, fixed indentation, organize imports and some minor refactoring.

Original Pull Request: #169
2016-02-18 07:41:23 +01:00
Mark Paluch
b58c18f85b DATAREDIS-462 - Support lettuce ClientResources.
Add ClientResources to LettuceConnectionFactory. Add TestClientResources for managed resources during tests and reuse ClientResources as much as possible in tests.

Original Pull Request: #169
2016-02-17 17:26:20 +01:00
Christoph Strobl
0fa1319b8f DATAREDIS-465 - Guard lettuce client shutdown during ConnectionFactory.destroy.
We now guard and log errors during lettuce client shutdown when destroying the ConnectionFactory on ApplicationContext shutdown.
2016-02-17 17:26:20 +01:00
Christoph Strobl
bdea5f6544 DATAREDIS-415 - Polishing.
Update license headers and fix minor code format issues.

Original Pull Request: #168
2016-02-17 10:00:16 +01:00
Mark Paluch
7da09eaa51 DATAREDIS-415 - Handle InterruptedException correctly in RedisMessageListenerContainer.
Set interrupted bit when catching InterruptedException. Exit loops when catching InterruptedException to free resources.

Original Pull Request: #168
2016-02-17 09:59:36 +01:00
Mark Paluch
80f8867763 DATAREDIS-464 - Enhance JavaDoc on RedisStringCommands and StringRedisConnection. 2016-02-16 15:20:54 +01:00
Mark Paluch
1151bcf136 DATAREDIS-316 - Polishing.
Extend JavaDoc documentation. Spelling fixes. Add Expiration.from factory method to create Expiration from all available time units.

Original pull request: #170.
2016-02-16 15:14:26 +01:00
Christoph Strobl
0c6c127b5e DATAREDIS-316 - Add EX/PX, NX/XX options to SET command.
We now support EX/PX and NX/XX arguments along with the SET command for both xetorthio/jedis and mp911de/lettuce.

Jedis 2.8 does not support all combinations of EX/PX and NX/XX. To work around those limitations we delegate to the according set methods to execute related commands or fail fast if there is no way for delegation.

Original pull request: #170.
2016-02-16 15:14:14 +01:00
Christoph Strobl
32c5512dda DATAREDIS-461 - Deprecate support for org.codehaus.jackson based Serializers and Mappers. 2016-02-12 09:50:18 +01:00
Vladimir Tsanev
1781e80717 DATAREDIS-456 - Fix misleading log message created by RedisMessageListenerContainer.
Original Pull Request: #161 (Vladimir Tsanev)
2016-02-10 13:34:36 +01:00
Christoph Strobl
97b88979e7 DATAREDIS-407 - Polishing.
Updated license header, added missing Javadoc, fix indentation and organized imports.

Original Pull Request: #166
2016-02-10 11:09:00 +01:00
Mark Paluch
4131f501a6 DATAREDIS-407 - Add zRangeByLex to ZSetOperations.
We now offer methods for ZRANGEBYLEX on ZSetOperations, BoundZSetOperations as well as via RedisZSet.

Original Pull Request: #166
2016-02-10 11:08:51 +01:00
Christoph Strobl
008836066f DATAREDIS-315 - Remove timeout/password from RedisClusterConfiguration.
Removed duplicate configuration options for timeout and password from RedisClusterConfiguration and use the ones provided via the RedisConnectionFactory.

Original pull request: #158.
2016-02-09 15:40:38 +01:00
Mark Paluch
0ac118fcc0 DATAREDIS-315 - Support password protected Redis Cluster.
Add new config property to configure a password for operations with Redis Cluster. The only driver to support Redis Cluster with password is lettuce but 3.4.Final has a blocker bug that prevents the usage

Original pull request: #158.
2016-02-09 15:40:38 +01:00
Mark Paluch
f18def013a DATAREDIS-315 - Support Redis 3.2 CLUSTER NODES output format.
The CLUSTER NODES converter supports the formats for Redis 3.0 and Redis 3.2.

Original pull request: #158.
2016-02-09 15:40:33 +01:00
Mark Paluch
c105cbb7b6 DATAREDIS-315 - Polishing.
Update documentation and fix spelling in JavaDoc and reference documentation.

Allow node lookup by host/port and node Id to enable flexibility when passing references to Redis Cluster nodes. Parse multiple slot ranges for a cluster node and move CLUSTER NODES parser to Converters.

Use fixed line separator instead of OS-specific line separator, consolidate duplicate code, support clusterSetSlotStable with lettuce 3.4 and set lettuce driver shutdown timeout to zero for long running tests.

Original pull request: #158.
2016-02-09 15:38:52 +01:00
Christoph Strobl
c5047f40c6 DATAREDIS-315 - Add initial support for redis cluster (jedis/lettuce).
Cluster support is based on the very same building blocks as non clustered communication. RedisClusterConnection and extension to RedisConnection handles the communication with the Redis Cluster and translates errors into the Spring DAO exception hierarchy.

Redirects for to a specific keys to the corresponding slot serving node are handled by the driver libraries, higher level functions like collecting information accross nodes, or sending commands to all nodes in the cluster that are covered by RedisClusterConnection utilizing a ClusterCommandExecutor distributing commands accross the cluster and collecting results.

RedisTemplate provides access to cluster specific operations via the ClusterOperations interface that can be obtained via RedisTemplate.opsForCluster(). This allows to execute commands explicitly on a single node within the cluster while retaining de-/serialization features configured for the template.

Original pull request: #158.
2016-02-09 14:53:33 +01:00
Christoph Strobl
1f97623a7c DATAREDIS-443 - Add Support for Spring 4.3 synchronized mode to RedisCache.
As of Spring Framework 4.3.RC1, the `Cache` interface has a new `<T> T get(Object key, Callable<T> valueLoader);` method (see SPR-9254).

If no entry for the given key is found, the `Callable` is invoked to compute/load the value that is then put into redis and returned. Additionally concurrent calls get synchronized so that the `Callable` is only called once.

Using Spring Framework 4.3 failures result in `o.s.c.Cache$ValueRetrievalException` prior versions use `RedisSystemException`.

Original pull request: #162.
2016-02-04 09:13:42 +01:00
Christoph Strobl
239d97195a DATAREDIS-405 - Deprecate support for jredis and srp.
Original pull request: #163.
2016-02-03 11:17:07 +01:00
Christoph Strobl
a446dd3619 DATAREDIS-448 - Fix DefaultSetOperations.intersectAndStore not returning command result.
We now properly return the number of elements in the destination set, has always been null so far, for DefaultSetOperations.intersectAndStore.
2016-01-28 07:59:32 +01:00
Christoph Strobl
d078a63f4c DATAREDIS-431 - Autoselect database configured in LettuceConnectionFactory.
We now actively select the predefined db set via the dbIndex on LettuceConnectionFactory. Fixed some spelling issues along the way.

Original pull request: #157.
2015-11-15 13:37:24 +01:00
欧夺标
130c057acd DATAREDIS-417 - Fix ScanCursor throwing NoSuchElementException for empty scan result.
ScanCursor now retriggers SCAN command for results having a new cursorId but no actual values. This avoids NoSuchElementException for server calls like:

    scan 0 match key*9
      1) 1966080
      2) (empty list or set)

Original Pull Request: #154
CLA Number: 143520151016081625 (Duobiao Ou)
2015-10-16 11:31:36 +02:00
Christoph Strobl
74ea61c51c DATAREDIS-421 - Fix NPE when using HashMapper implementations.
We now make sure the available HashMapper implementations avoid exposing null values. Prior to this null values would have been added to the converted Hash causing trouble when saving those to redis since hashes are expected to not contain any null values
2015-08-31 14:19:39 +02:00
Christoph Strobl
35424bb9da DATAREDIS-392 - Add Gerneric Jackson2 RedisSerializer.
We introduced GenericJackson2JsonRedisSerializer holding a preconfigured ObjectMapper which writes Class type informations into the JSON structure. This enables polymorphic deserialization and allows RedisCache manager to operate upon a RedisTemplate storing data in JSON format, which had until this only been possible using the Jdk Serializer.
2015-08-31 14:09:22 +02:00
Christoph Strobl
444b290863 DATAREDIS-348 - Polishing.
Fixed connection problems leaving sockets open. Added missing JUnit rules for Sentinels. Added missing author information update, license header and Javadoc.

Upgraded test script to use Redis 3.0.2. Upgrade test script to shut down Redis in case of test errors. Added Redis 3.0 to TestProfileValueSource. Enabled ZRANGEBYLEX tests for lettuce.

Original pull request: #144.
Related pull request: #104.
2015-08-04 10:12:39 +02:00
Mark Paluch
7426b27688 DATAREDIS-348 - Switch to mp911de/lettuce 3.0.
Updated lettuce redis client to latest snapshot, adopt changed API and use API for commands which were emulated using LUA script

Implemente Sentinel support using lettuce library, adopt finer grained exceptions and aligned exception behavior to match other libraries. Added shutdown-timeout property in order to control shutdown duration of the netty framework which reduces test run duration. Documentation updated as well. Contains also a fix to use psetex instead of setex (typo).

Switch to Jedis for Redis Version discovery in tests. Polished documentation. Use LettuceConverter and native time() method.

Original pull request: #144.
Related pull request: #104.
2015-08-04 10:12:39 +02:00
Christoph Strobl
a08e90a1f7 DATAREDIS-414 - Serialization now occurs after pooled connection was released.
Moved key serialization and value deserialization outside of callback so that pooled connections get released more quickly.

Original pull request: #153.
2015-07-28 14:18:26 +02:00
Christoph Strobl
7073502e42 DATAREDIS-416 - RedisCache.putIfAbsent(…) now returns null for first addition.
We now return null for RedisCache.putIfAbesent(…) when value is set to meet the contract defined by org.springframework.cache.Cache.putIfAbsent(…).

Original pull request: #150.
2015-07-28 13:42:10 +02:00
Christoph Strobl
b2bae8f8fb DATAREDIS-412 - Assert compatibility with Jedis 2.7.1, 2.7.2 and 2.7.3.
Bump Jedis version and fix SendCommand class type differences between versions.

We now also lazily init the transaction on first call of multi() since pre initialization causes issues releasing Jedis connections and returning them to pool.

Original pull request: #149.
Related ticket: #139.
2015-07-22 14:39:30 +02:00
Christoph Strobl
1ad3392a72 DATAREDIS-401 - RedisCacheManager should operate on RedisOperations.
We now no longer require RedisCache and RedisCacheManager to be initialized with a RedisTemplate but use RedisOperations instead.

Original pull request: #142.
2015-07-09 19:10:13 +02:00
Christoph Strobl
aa04e73dbd DATAREDIS-402 - RedisCache should not expire known keys in case ttl of element is not set.
We now skip defining expire on known keys set in case element is eternal.

Original pull request: #141.
2015-06-16 09:38:54 +02:00
Christoph Strobl
283a9638f2 DATAREDIS-352 - Enhance range options for ZSET.
We now allow usage of Range and Limit types (newly introduced for ZRANGEBYLEX) on mostly all ZSET range operations.
Extracted constants for plus/minus and -inf/+inf expressions.

Original pull request: #138.
2015-05-22 15:16:08 +02:00
Thomas Darimont
3d4f62f5b7 DATAREDIS-396 - Update Jedis driver to version 2.7.2.
Adapted sendCommand lookup in JedisConnection to honor the changed signature of redis.clients.jedis.Connection.sendCommand(Command, String…) to redis.clients.jedis.Connection.sendCommand(ProtocolCommand, String…).

Original pull request: #140.
2015-05-19 13:26:33 +02:00
Christoph Strobl
ff37da5197 DATAREDIS-378 - Add support for ZRANGEBYLEX.
We now support ZRANGEBYLEX on RedisConnection and StringRedisConnection when using the Jedis driver. The upper and lower bounds can be defined using the Range type which will be converted to the according binary representation including prefix and/or infinite operators.

	range().gt("a")          => (a +
	range().gte("a")         => [a +
	range().lt("z")          => - (z
	range().lte("z")         => - [z
	range().gte("a").lt("z") => [a (z
2015-04-13 14:15:10 +02:00
Thomas Darimont
cf4fa9e4fb DATAREDIS-332 - @Transactional operation should use the same connection.
We only unbind the current redisConnectionFactory from the current thread if the transaction is not active anymore.
Previously we performed the unbinding in RedisTemplate.execute(RedisCallback, boolean, boolean) overtime which lead to the fact that the next redis operation within the same transaction used a different redis connection, which could lead to exhaustion of the connection pool.
We also ensure that we correctly unbind the RedisConnectionFactory resource from the TransactionSynchronizationManager on transaction completion in RedisTransactionSynchronizer.afterCompletion(int).
Added test case supplied from issue.

Original pull request: #134.
2015-04-13 12:58:54 +02:00
Thomas Darimont
c47292d173 DATAREDIS-374 - Adapt JedisConnectionFactory for changes in Jedis 2.7.
We now support both Jedis 2.6.2 (current release) as well as the upcoming 2.7 version. Therefore we call the in 2.7 renamed `getTimeout` / `setTimeout` Methods on `JedisShardInfo` via reflection to read/set `soTimeout`.

Since there’s no change in r.c.j.JedisFactory that would allow to individually configure connection and socket timeout, we use the socketTimeout of `JedisShardInfo`. This will cause `r.c.j.JedisFactory` to use the socket timeout for the connection as well, which is the same behavior as in 2.6.2.

Tested against Jedis 2.7 branch with Redis 3.0.0.RC4.

Original Pull Request: #127
2015-03-19 11:41:57 +01:00
Stephane Nicoll
d0a0a6a78b DATAREDIS-383 - Improve JavaDoc of RedisCacheManager.
Extended description on setCacheNames(..).

Original pull request: #132.
2015-03-18 13:10:10 +01:00
Thomas Darimont
a8e4659f75 DATAREDIS-375 - Avoid repeatedly decorating a cache.
Previously a transactional RedisCacheManager decorated a cache with the TransactionAwareCacheDecorator twice. We now make sure that a particular cache is only decorated once with a TransactionAwareCacheDecorator.

Original Pull Request: #128
2015-03-04 17:28:32 +01:00
Christoph Strobl
c662c46d02 DATAREDIS-372 - Allow easier RedisSentinelConfiguration setup.
We now allow to initialize a RedisSentinelConfiguration based on a given String collection of host:ports.
  sentinelHostAndPorts[0] = 127.0.0.1:23679
  sentinelHostAndPorts[1] = 127.0.0.1:23680

Further on it’s possible to pass in a PropertySource containing master name and sentinel nodes.
  spring.redis.sentinel.master=myMaster
  spring.redis.sentinel.nodes=127.0.0.1:23679,127.0.0.1:23680

Original pull request: #126.
2015-02-16 15:16:09 +01:00
Christoph Strobl
7a6b127355 DATAREDIS-369 - Avoid resource leak in RedisCache.
Introduce dedicated types RedisCache is supposed to work upon. We still stick to the contract defined by previous versions to assure source and binary compatibility.
In a next step RedisCache should be decoupled from o.s.cache.Cache by an additional indirection via RedisCacheCache.

Original Pull Request: #122
2015-02-05 08:32:50 +01:00
Christoph Strobl
b242d4ac6d DATAREDIS-369 - Avoid resource leak in RedisCache.
We now check the presence of a prefix which allows to identify the keys associated with the cache to either keep track of keys or use the prefix to remove them on clear.

This is a small fix intended to be back ported to the Evans (1.4.x) release train. Additional refactoring will be done for the Fowler release.

Original Pull Request: #122
2015-02-05 08:32:36 +01:00
Thomas Darimont
0ea0c4bf67 DATAREDIS-328 - RedisCacheManager should not instantiate caches in setCacheNames().
We now construct the caches for the configured cache-names in afterPropertiesSet(). Previously the caches were created in the setter which lead to unwanted property-set order dependencies.

Original pull request: #123.
2015-02-02 14:54:05 +01:00
Thomas Darimont
89d5ac8840 DATAREDIS-364 - Open-up API in RedisCacheManager for easier extensibility.
We now allow users to build custom CacheManagers based on RedisCacheManager more easily by making a broader API accessible to sub-classes.
In that sense we also allowing RedisCache to be used in custom implementations by making it public.

Original pull request: #120.
2015-01-19 14:04:00 +01:00
Thomas Darimont
e72514143e DATAREDIS-356 - Check NonTransientDataAccessException for NOSCRIPT error.
We now check for a NonTransientDataAccessException instead of just an UncategorizedDataAccessException to conver all relevant exception types.
Corrected typo in method name. Added infrastructure for driver specific DefaultScriptExecutorTests.
Removed usage of RelaxedJunit4ClassRuner and using MinimumRedisRule
instead.
Removed obsolete test context configuration for ScriptExecutorTests.

Original pull request: #115.
2014-11-28 10:10:01 +01:00