Commit Graph

794 Commits

Author SHA1 Message Date
Christoph Strobl
bd7728e6ad DATAREDIS-425 - Add Support for basic CRUD and finder Operations backed by Hashes and Sets.
We now enable storing domain object as a flat Redis 'HASH' and maintain additional 'SET' structures to enable finder operations on simple properties.

  @RedisHash("persons");
  class Person {

    @id String id;
    @Indexed String firstname;
    String lastname;

    Map<String, String> attributes;

    City city;

    @Reference Person mother;
  }

The above is stored in the HASH with key 'persons:1' as

  _class = org.example.Person
  id = 1
  firstname = rand
  lastname = al’thor
  attributes.[eye-color] = grey
  attributes.[hair-color] = red
  city.name = emond's field
  city.region = two rivers
  mother = persons:2

Complex types are flattened out to their full property path for each of the values provided. If the properties actual value type does not match the declared one the '_class' type hint is added to the entry.

  city._class = CityInAndor.class
  city.name = emond's field
  city.region = two rivers
  city.country = andor

Map and Collection like structures are stored with their key/index values as part of the property path. If the map/collection value type does not match the actutal objects one the '_class' type hint is added to the entry.

  list.[0]._class = DomainType.class
  list.[0].property1 = ...

  map.[key-1]._class = DomainType.class
  map.[key-1].property1 = ...

Properties marked with '@Reference' are stored as semantic references by just storing the key to the referenced object 'HASH' instead of embedding its values.

   mother = persons:2

Please note that referenced objects are not transitively updated/saved and that lazy loading of references will be part of future development.

A 'save' operation therefore executes the following:

  # flatten domain type and add as hash
  HMSET persons:1 id 1 firstname rand …

  # add the newly inserted entry to the list of all entries of that type
  SADD persons 1

  # index the firstname for finder lookup
  SADD persons.firstname:rand 1

Simple finder operation like 'findByFirstname' use 'SINTER' to find matching

  SINTER persons.firstname:rand
  HGETALL persons:1

Besides resolving an index via the '@Index' annotation we also allow to add custom configuration via the 'indexConfiguration' attribute of '@EnableRedisRepositories'.

  @Configuration
  @EnableRedisRepositories(indexConfiguration = CustomIndexConfiguration.class)
  class Config { }

  static class CustomIndexConfiguration extends IndexConfiguration {

    @Override
    protected Iterable<RedisIndexDefinition> initialConfiguration() {
      return Arrays.asList(
        new SimpleIndexDefinition("persons", "lastname"),
      );
    }
  }

The '@TimeToLive' annotation allows to define a property or method providing an expiration time when storing the key in redis.

@RedisHash
class Person {

  @Id String id;
  @TimeToLive Long ttl;
}

Original Pull Request: #156
2016-03-14 10:37:53 +01:00
Christoph Strobl
5a85ac47a3 DATAREDIS-472 - Add guards to JedisConnection before casting long to int. 2016-03-04 14:19:41 +01:00
Christoph Strobl
7024ef33e0 DATAREDIS-468 - Guard multi/exec bocks in RedisCache to allow usage with cluster.
We now guard multi/exec blocks by checking the connection type. This allows to skip those commands when running in cluster.

Original pull request: #173.
2016-02-26 09:07:02 +01:00
Christoph Strobl
02742e3b49 DATAREDIS-467 - Fix cluster multi key commands.
We moved away from returning raw map types and now return dedicated objects for command executions in cluster environment. This allows to maintain node information and collecting results from each and every callback. MGET now returns values according to the key position.

Additionally we now prefix info commands in the cluster with the nodes host:port.

Original pull request: #174.
2016-02-26 08:46:05 +01:00
Christoph Strobl
1e5772d00b DATAREDIS-462 - Polishing.
Updated javadoc, fixed indentation, organize imports and some minor refactoring.

Original Pull Request: #169
2016-02-18 07:41:23 +01:00
Mark Paluch
b58c18f85b DATAREDIS-462 - Support lettuce ClientResources.
Add ClientResources to LettuceConnectionFactory. Add TestClientResources for managed resources during tests and reuse ClientResources as much as possible in tests.

Original Pull Request: #169
2016-02-17 17:26:20 +01:00
Christoph Strobl
30b06a9202 DATAREDIS-463 - Prevent tests cleanup from failing.
Use more general Exception during tearDown to prevent tests from failing.
2016-02-17 17:26:20 +01:00
Mark Paluch
cc6bde665a DATAREDIS-463 - Improve test synchronization.
Several tests rely heavily on time to perform synchronization. Therfore we remove Thread.sleep from tests which do not require a sleep at all, use a condition where possible and increase timeouts on tests known to fail very likely due to a short Thread.sleep.

Original Pull Request: #171
2016-02-17 15:26:46 +01:00
Christoph Strobl
bdea5f6544 DATAREDIS-415 - Polishing.
Update license headers and fix minor code format issues.

Original Pull Request: #168
2016-02-17 10:00:16 +01:00
Mark Paluch
7da09eaa51 DATAREDIS-415 - Handle InterruptedException correctly in RedisMessageListenerContainer.
Set interrupted bit when catching InterruptedException. Exit loops when catching InterruptedException to free resources.

Original Pull Request: #168
2016-02-17 09:59:36 +01:00
Mark Paluch
1151bcf136 DATAREDIS-316 - Polishing.
Extend JavaDoc documentation. Spelling fixes. Add Expiration.from factory method to create Expiration from all available time units.

Original pull request: #170.
2016-02-16 15:14:26 +01:00
Christoph Strobl
0c6c127b5e DATAREDIS-316 - Add EX/PX, NX/XX options to SET command.
We now support EX/PX and NX/XX arguments along with the SET command for both xetorthio/jedis and mp911de/lettuce.

Jedis 2.8 does not support all combinations of EX/PX and NX/XX. To work around those limitations we delegate to the according set methods to execute related commands or fail fast if there is no way for delegation.

Original pull request: #170.
2016-02-16 15:14:14 +01:00
Christoph Strobl
b59c79f20a DATAREDIS-452 - Polishing.
Fix indentation and organized imports.

Original Pull Request: #165
2016-02-10 14:59:26 +01:00
Mark Paluch
be40b0853a DATAREDIS-452 - Improve thread synchronization in RedisCacheTest.testCacheGetSynchronized.
Original Pull Request: #165
2016-02-10 14:57:51 +01:00
Christoph Strobl
e0ff8fcf1d DATAREDIS-455 - Increase expire time in RedisTemplateTests.
Increase the expiration time to prevent test from failing randomly when executed in cluster environment. Needs to be done as a cluster refresh on driver level might take longer than the actual expiration, which then causes the key to already be expired when checking the assertion.
2016-02-10 13:07:47 +01:00
Christoph Strobl
97b88979e7 DATAREDIS-407 - Polishing.
Updated license header, added missing Javadoc, fix indentation and organized imports.

Original Pull Request: #166
2016-02-10 11:09:00 +01:00
Mark Paluch
4131f501a6 DATAREDIS-407 - Add zRangeByLex to ZSetOperations.
We now offer methods for ZRANGEBYLEX on ZSetOperations, BoundZSetOperations as well as via RedisZSet.

Original Pull Request: #166
2016-02-10 11:08:51 +01:00
Christoph Strobl
008836066f DATAREDIS-315 - Remove timeout/password from RedisClusterConfiguration.
Removed duplicate configuration options for timeout and password from RedisClusterConfiguration and use the ones provided via the RedisConnectionFactory.

Original pull request: #158.
2016-02-09 15:40:38 +01:00
Mark Paluch
0ac118fcc0 DATAREDIS-315 - Support password protected Redis Cluster.
Add new config property to configure a password for operations with Redis Cluster. The only driver to support Redis Cluster with password is lettuce but 3.4.Final has a blocker bug that prevents the usage

Original pull request: #158.
2016-02-09 15:40:38 +01:00
Mark Paluch
f18def013a DATAREDIS-315 - Support Redis 3.2 CLUSTER NODES output format.
The CLUSTER NODES converter supports the formats for Redis 3.0 and Redis 3.2.

Original pull request: #158.
2016-02-09 15:40:33 +01:00
Mark Paluch
c105cbb7b6 DATAREDIS-315 - Polishing.
Update documentation and fix spelling in JavaDoc and reference documentation.

Allow node lookup by host/port and node Id to enable flexibility when passing references to Redis Cluster nodes. Parse multiple slot ranges for a cluster node and move CLUSTER NODES parser to Converters.

Use fixed line separator instead of OS-specific line separator, consolidate duplicate code, support clusterSetSlotStable with lettuce 3.4 and set lettuce driver shutdown timeout to zero for long running tests.

Original pull request: #158.
2016-02-09 15:38:52 +01:00
Christoph Strobl
c5047f40c6 DATAREDIS-315 - Add initial support for redis cluster (jedis/lettuce).
Cluster support is based on the very same building blocks as non clustered communication. RedisClusterConnection and extension to RedisConnection handles the communication with the Redis Cluster and translates errors into the Spring DAO exception hierarchy.

Redirects for to a specific keys to the corresponding slot serving node are handled by the driver libraries, higher level functions like collecting information accross nodes, or sending commands to all nodes in the cluster that are covered by RedisClusterConnection utilizing a ClusterCommandExecutor distributing commands accross the cluster and collecting results.

RedisTemplate provides access to cluster specific operations via the ClusterOperations interface that can be obtained via RedisTemplate.opsForCluster(). This allows to execute commands explicitly on a single node within the cluster while retaining de-/serialization features configured for the template.

Original pull request: #158.
2016-02-09 14:53:33 +01:00
Christoph Strobl
1f97623a7c DATAREDIS-443 - Add Support for Spring 4.3 synchronized mode to RedisCache.
As of Spring Framework 4.3.RC1, the `Cache` interface has a new `<T> T get(Object key, Callable<T> valueLoader);` method (see SPR-9254).

If no entry for the given key is found, the `Callable` is invoked to compute/load the value that is then put into redis and returned. Additionally concurrent calls get synchronized so that the `Callable` is only called once.

Using Spring Framework 4.3 failures result in `o.s.c.Cache$ValueRetrievalException` prior versions use `RedisSystemException`.

Original pull request: #162.
2016-02-04 09:13:42 +01:00
Christoph Strobl
a446dd3619 DATAREDIS-448 - Fix DefaultSetOperations.intersectAndStore not returning command result.
We now properly return the number of elements in the destination set, has always been null so far, for DefaultSetOperations.intersectAndStore.
2016-01-28 07:59:32 +01:00
Christoph Strobl
d078a63f4c DATAREDIS-431 - Autoselect database configured in LettuceConnectionFactory.
We now actively select the predefined db set via the dbIndex on LettuceConnectionFactory. Fixed some spelling issues along the way.

Original pull request: #157.
2015-11-15 13:37:24 +01:00
Christoph Strobl
3d7a524474 DATAREDIS-435 - Fix potential NPE in RedisTestProfileValueSource. 2015-10-27 09:30:32 +01:00
Christoph Strobl
df4ad35057 DATAREDIS-417 - Add failing tests for ScanCursor throwing NoSuchElementException.
Add failing tests for scanning key sets where Redis returns a scan iteration pointing to the next cursor but having no actual values itself like in:

    scan 0 match key*9
      1) 1966080
      2) (empty list or set)
2015-10-16 11:31:28 +02:00
Christoph Strobl
74ea61c51c DATAREDIS-421 - Fix NPE when using HashMapper implementations.
We now make sure the available HashMapper implementations avoid exposing null values. Prior to this null values would have been added to the converted Hash causing trouble when saving those to redis since hashes are expected to not contain any null values
2015-08-31 14:19:39 +02:00
Christoph Strobl
35424bb9da DATAREDIS-392 - Add Gerneric Jackson2 RedisSerializer.
We introduced GenericJackson2JsonRedisSerializer holding a preconfigured ObjectMapper which writes Class type informations into the JSON structure. This enables polymorphic deserialization and allows RedisCache manager to operate upon a RedisTemplate storing data in JSON format, which had until this only been possible using the Jdk Serializer.
2015-08-31 14:09:22 +02:00
Christoph Strobl
444b290863 DATAREDIS-348 - Polishing.
Fixed connection problems leaving sockets open. Added missing JUnit rules for Sentinels. Added missing author information update, license header and Javadoc.

Upgraded test script to use Redis 3.0.2. Upgrade test script to shut down Redis in case of test errors. Added Redis 3.0 to TestProfileValueSource. Enabled ZRANGEBYLEX tests for lettuce.

Original pull request: #144.
Related pull request: #104.
2015-08-04 10:12:39 +02:00
Mark Paluch
7426b27688 DATAREDIS-348 - Switch to mp911de/lettuce 3.0.
Updated lettuce redis client to latest snapshot, adopt changed API and use API for commands which were emulated using LUA script

Implemente Sentinel support using lettuce library, adopt finer grained exceptions and aligned exception behavior to match other libraries. Added shutdown-timeout property in order to control shutdown duration of the netty framework which reduces test run duration. Documentation updated as well. Contains also a fix to use psetex instead of setex (typo).

Switch to Jedis for Redis Version discovery in tests. Polished documentation. Use LettuceConverter and native time() method.

Original pull request: #144.
Related pull request: #104.
2015-08-04 10:12:39 +02:00
Christoph Strobl
7073502e42 DATAREDIS-416 - RedisCache.putIfAbsent(…) now returns null for first addition.
We now return null for RedisCache.putIfAbesent(…) when value is set to meet the contract defined by org.springframework.cache.Cache.putIfAbsent(…).

Original pull request: #150.
2015-07-28 13:42:10 +02:00
Christoph Strobl
aa04e73dbd DATAREDIS-402 - RedisCache should not expire known keys in case ttl of element is not set.
We now skip defining expire on known keys set in case element is eternal.

Original pull request: #141.
2015-06-16 09:38:54 +02:00
Thomas Darimont
31351b0463 DATAREDIS-352 - Enhance range options for ZSET.
Raising required redis version for zRangeByLexTest to fix failing test on CI server.
2015-05-22 18:59:54 +02:00
Christoph Strobl
283a9638f2 DATAREDIS-352 - Enhance range options for ZSET.
We now allow usage of Range and Limit types (newly introduced for ZRANGEBYLEX) on mostly all ZSET range operations.
Extracted constants for plus/minus and -inf/+inf expressions.

Original pull request: #138.
2015-05-22 15:16:08 +02:00
Christoph Strobl
ff37da5197 DATAREDIS-378 - Add support for ZRANGEBYLEX.
We now support ZRANGEBYLEX on RedisConnection and StringRedisConnection when using the Jedis driver. The upper and lower bounds can be defined using the Range type which will be converted to the according binary representation including prefix and/or infinite operators.

	range().gt("a")          => (a +
	range().gte("a")         => [a +
	range().lt("z")          => - (z
	range().lte("z")         => - [z
	range().gte("a").lt("z") => [a (z
2015-04-13 14:15:10 +02:00
Thomas Darimont
cf4fa9e4fb DATAREDIS-332 - @Transactional operation should use the same connection.
We only unbind the current redisConnectionFactory from the current thread if the transaction is not active anymore.
Previously we performed the unbinding in RedisTemplate.execute(RedisCallback, boolean, boolean) overtime which lead to the fact that the next redis operation within the same transaction used a different redis connection, which could lead to exhaustion of the connection pool.
We also ensure that we correctly unbind the RedisConnectionFactory resource from the TransactionSynchronizationManager on transaction completion in RedisTransactionSynchronizer.afterCompletion(int).
Added test case supplied from issue.

Original pull request: #134.
2015-04-13 12:58:54 +02:00
Thomas Darimont
a8e4659f75 DATAREDIS-375 - Avoid repeatedly decorating a cache.
Previously a transactional RedisCacheManager decorated a cache with the TransactionAwareCacheDecorator twice. We now make sure that a particular cache is only decorated once with a TransactionAwareCacheDecorator.

Original Pull Request: #128
2015-03-04 17:28:32 +01:00
Christoph Strobl
c662c46d02 DATAREDIS-372 - Allow easier RedisSentinelConfiguration setup.
We now allow to initialize a RedisSentinelConfiguration based on a given String collection of host:ports.
  sentinelHostAndPorts[0] = 127.0.0.1:23679
  sentinelHostAndPorts[1] = 127.0.0.1:23680

Further on it’s possible to pass in a PropertySource containing master name and sentinel nodes.
  spring.redis.sentinel.master=myMaster
  spring.redis.sentinel.nodes=127.0.0.1:23679,127.0.0.1:23680

Original pull request: #126.
2015-02-16 15:16:09 +01:00
Christoph Strobl
b242d4ac6d DATAREDIS-369 - Avoid resource leak in RedisCache.
We now check the presence of a prefix which allows to identify the keys associated with the cache to either keep track of keys or use the prefix to remove them on clear.

This is a small fix intended to be back ported to the Evans (1.4.x) release train. Additional refactoring will be done for the Fowler release.

Original Pull Request: #122
2015-02-05 08:32:36 +01:00
Thomas Darimont
0ea0c4bf67 DATAREDIS-328 - RedisCacheManager should not instantiate caches in setCacheNames().
We now construct the caches for the configured cache-names in afterPropertiesSet(). Previously the caches were created in the setter which lead to unwanted property-set order dependencies.

Original pull request: #123.
2015-02-02 14:54:05 +01:00
Thomas Darimont
e72514143e DATAREDIS-356 - Check NonTransientDataAccessException for NOSCRIPT error.
We now check for a NonTransientDataAccessException instead of just an UncategorizedDataAccessException to conver all relevant exception types.
Corrected typo in method name. Added infrastructure for driver specific DefaultScriptExecutorTests.
Removed usage of RelaxedJunit4ClassRuner and using MinimumRedisRule
instead.
Removed obsolete test context configuration for ScriptExecutorTests.

Original pull request: #115.
2014-11-28 10:10:01 +01:00
Christoph Strobl
cb070c1375 DATAREDIS-358 - Precise version resolution within RedisTestProfileValueSource.
We now resolve the precise redis version including patch level for the 2.8 line. This allows us to disable tests when building against older versions that might not have the complete featureset available.
2014-11-27 22:03:24 +01:00
Christoph Strobl
22aa5df73e DATAREDIS-308 - Add support for HyperLogLog.
We added methods for PFADD, PFCOUNT and PFMERGE to RedisConnection and StringRedisConnection. HyperLogLogOperations available via RedisTemplate allow more indrect usage of HLL.

Please note that currently Jedis is the only driver that can be used for HyperLogLog commands.

Original pull request: #116.
2014-11-26 15:59:11 +01:00
Christoph Strobl
57fc58fcff DATAREDIS-288 - Polishing.
Added javadoc and assertions to ensure failing before actually sending data to redis and backed those with tests.

Original pull request: #114.
2014-11-17 13:13:08 +01:00
David Liu
d89773387a DATAREDIS-288 - Improved support for collection parameters in ListOperations.
Added additional leftPushAll and rightPushAll method overloads that accept a Collection parameter.
Previously we only accepted varargs which required users to convert collections to arrays.

Based on David Liu's pull request: #100.

Original pull request: #114.
2014-11-17 13:03:13 +01:00
Thomas Darimont
cf46d6e766 DATAREDIS-106 - Support for open interval for scores for sorted sets.
We now support open zset's interval for JedisConnection, JredisConnection, LettuceConnection, SrpConnection and DefaultStringRedisConnection.

Original pull request: #97.
2014-10-29 11:53:03 +01:00
Christoph Strobl
5ca672d180 DATAREDIS-347 - Fix unintended duplicate script execution.
We now inspect the message of the exception thrown to determine whether the error resulted from a script not being present or an other error. In case the script is not present we simply invoke eval.

Original pull request: #108
2014-10-24 19:24:56 +02:00
David Liu
4711693669 DATAREDIS-334 - Add binary alternative for RedisScriptingCommands.evalSha.
Introduced new evalSha method that accepts a byte[] as scriptSha as well
as a variable number of byte[] as keyValueArgs.

Original pull request: #99.
2014-10-09 16:13:27 +02:00
David Liu
9ecad07587 DATAREDIS-197 - Expose bit-related ops via ValueOperations.
Polishing. Added -S switch to gradlew in makefile for better Stacktrace reporting.

Original pull request: #96.
2014-09-30 10:28:53 +02:00