Add builder pattern to RedisVectorStore and refactor package name
Refactors RedisVectorStore to use the builder pattern for improved configuration and usability. The changes include: * Move classes to org.springframework.ai.vectorstore.redis package * Add RedisBuilder with comprehensive configuration options * Deprecate RedisVectorStoreConfig in favor of builder pattern * Enhance documentation with detailed usage examples * Improve error handling and parameter validation This change makes RedisVectorStore configuration more intuitive and consistent with other vector stores in the project.
This commit is contained in:
Soby Chacko
committed by
Mark Pollack
Mark Pollack
parent
cde8f7446c
commit
f69d879ec9
@@ -45,41 +45,15 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
|
||||
The vector store implementation can initialize the requisite schema for you, but you must opt-in by specifying the `initializeSchema` boolean in the appropriate constructor or by setting `...initialize-schema=true` in the `application.properties` file.
|
||||
|
||||
NOTE: this is a breaking change! In earlier versions of Spring AI, this schema initialization happened by default.
|
||||
|
||||
Please have a look at the list of <<redisvector-properties,configuration parameters>> for the vector store to learn about the default values and configuration options.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other EmbeddingModel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
To connect to Redis you need to provide access details for your instance.
|
||||
A simple configuration can either be provided via Spring Boot's _application.properties_,
|
||||
|
||||
[source,properties]
|
||||
----
|
||||
spring.ai.vectorstore.redis.uri=<your redis instance uri>
|
||||
spring.ai.vectorstore.redis.index=<your index name>
|
||||
spring.ai.vectorstore.redis.prefix=<your prefix>
|
||||
|
||||
# API key if needed, e.g. OpenAI
|
||||
spring.ai.openai.api.key=<api-key>
|
||||
----
|
||||
|
||||
Please have a look at the list of xref:#_configuration_properties[configuration parameters] for the vector store to learn about the default values and configuration options.
|
||||
|
||||
Now you can Auto-wire the Redis Vector Store in your application and use it
|
||||
Now you can auto-wire the `RedisVectorStore` as a vector store in your application.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -99,146 +73,144 @@ vectorStore.add(documents);
|
||||
List<Document> results = this.vectorStore.similaritySearch(SearchRequest.query("Spring").withTopK(5));
|
||||
----
|
||||
|
||||
=== Configuration properties
|
||||
[[redisvector-properties]]
|
||||
=== Configuration Properties
|
||||
|
||||
You can use the following properties in your Spring Boot configuration to customize the Redis vector store.
|
||||
To connect to Redis and use the `RedisVectorStore`, you need to provide access details for your instance.
|
||||
A simple configuration can be provided via Spring Boot's `application.yml`,
|
||||
|
||||
[stripes=even]
|
||||
[source,yaml]
|
||||
----
|
||||
spring:
|
||||
data:
|
||||
redis:
|
||||
uri: <redis instance uri>
|
||||
ai:
|
||||
vectorstore:
|
||||
redis:
|
||||
initialize-schema: true
|
||||
index-name: custom-index
|
||||
prefix: custom-prefix
|
||||
batching-strategy: TOKEN_COUNT # Optional: Controls how documents are batched for embedding
|
||||
----
|
||||
|
||||
Properties starting with `spring.ai.vectorstore.redis.*` are used to configure the `RedisVectorStore`:
|
||||
|
||||
[cols="2,5,1",stripes=even]
|
||||
|===
|
||||
|Property| Description | Default value
|
||||
|
||||
|`spring.ai.vectorstore.redis.uri`| Server connection URI | `redis://localhost:6379`
|
||||
|`spring.ai.vectorstore.redis.index`| Index name | `default-index`
|
||||
|`spring.ai.vectorstore.redis.initialize-schema`| Whether to initialize the required schema | `false`
|
||||
|`spring.ai.vectorstore.redis.prefix`| Prefix | `default:`
|
||||
|Property | Description | Default Value
|
||||
|
||||
|`spring.ai.vectorstore.redis.initialize-schema`| Whether to initialize the required schema | `false`
|
||||
|`spring.ai.vectorstore.redis.index-name` | The name of the index to store the vectors | `spring-ai-index`
|
||||
|`spring.ai.vectorstore.redis.prefix` | The prefix for Redis keys | `embedding:`
|
||||
|`spring.ai.vectorstore.redis.batching-strategy` | Strategy for batching documents when calculating embeddings. Options are `TOKEN_COUNT` or `FIXED_SIZE` | `TOKEN_COUNT`
|
||||
|===
|
||||
|
||||
== Metadata filtering
|
||||
== Metadata Filtering
|
||||
|
||||
You can leverage the generic, portable link:https://docs.spring.io/spring-ai/reference/api/vectordbs.html#_metadata_filters[metadata filters] with RedisVectorStore as well.
|
||||
You can leverage the generic, portable xref:api/vectordbs.adoc#metadata-filters[metadata filters] with Redis as well.
|
||||
|
||||
For example, you can use either the text expression language:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
vectorStore.similaritySearch(
|
||||
SearchRequest
|
||||
.query("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression("country in ['UK', 'NL'] && year >= 2020"));
|
||||
vectorStore.similaritySearch(SearchRequest.defaults()
|
||||
.withQuery("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression("country in ['UK', 'NL'] && year >= 2020"));
|
||||
----
|
||||
|
||||
or programmatically using the expression DSL:
|
||||
or programmatically using the `Filter.Expression` DSL:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
FilterExpressionBuilder b = new FilterExpressionBuilder();
|
||||
|
||||
vectorStore.similaritySearch(
|
||||
SearchRequest
|
||||
.query("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression(b.and(
|
||||
b.in("country", "UK", "NL"),
|
||||
b.gte("year", 2020)).build()));
|
||||
vectorStore.similaritySearch(SearchRequest.defaults()
|
||||
.withQuery("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression(b.and(
|
||||
b.in("country", "UK", "NL"),
|
||||
b.gte("year", 2020)).build()));
|
||||
----
|
||||
|
||||
The portable filter expressions get automatically converted into link:https://redis.io/docs/interact/search-and-query/query/[Redis search queries].
|
||||
For example, the following portable filter expression:
|
||||
NOTE: Those (portable) filter expressions get automatically converted into link:https://redis.io/docs/interact/search-and-query/query/[Redis search queries].
|
||||
|
||||
For example, this portable filter expression:
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
country in ['UK', 'NL'] && year >= 2020
|
||||
----
|
||||
|
||||
is converted into Redis query:
|
||||
is converted into the proprietary Redis filter format:
|
||||
|
||||
[source]
|
||||
[source,text]
|
||||
----
|
||||
@country:{UK | NL} @year:[2020 inf]
|
||||
----
|
||||
|
||||
== Manual configuration
|
||||
== Manual Configuration
|
||||
|
||||
If you prefer not to use the auto-configuration, you can manually configure the Redis Vector Store.
|
||||
Add the Redis Vector Store and Jedis dependencies
|
||||
Instead of using the Spring Boot auto-configuration, you can manually configure the Redis vector store. For this you need to add the `spring-ai-redis-store` to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-redis-store</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>redis.clients</groupId>
|
||||
<artifactId>jedis</artifactId>
|
||||
<version>5.1.0</version>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-redis-store</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
Then, create a `RedisVectorStore` bean in your Spring configuration:
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-redis-store'
|
||||
}
|
||||
----
|
||||
|
||||
Create a `JedisPooled` bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(EmbeddingModel embeddingModel) {
|
||||
RedisVectorStoreConfig config = RedisVectorStoreConfig.builder()
|
||||
.withURI("redis://localhost:6379")
|
||||
// Define the metadata fields to be used
|
||||
// in the similarity search filters.
|
||||
.withMetadataFields(
|
||||
MetadataField.tag("country"),
|
||||
MetadataField.numeric("year"))
|
||||
.build();
|
||||
public JedisPooled jedisPooled() {
|
||||
return new JedisPooled("<host>", 6379);
|
||||
}
|
||||
----
|
||||
|
||||
return new RedisVectorStore(config, embeddingModel);
|
||||
Then create the `RedisVectorStore` bean using the builder pattern:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(JedisPooled jedisPooled, EmbeddingModel embeddingModel) {
|
||||
return RedisVectorStore.builder()
|
||||
.jedis(jedisPooled)
|
||||
.embeddingModel(embeddingModel)
|
||||
.indexName("custom-index") // Optional: defaults to "spring-ai-index"
|
||||
.prefix("custom-prefix") // Optional: defaults to "embedding:"
|
||||
.metadataFields( // Optional: define metadata fields for filtering
|
||||
MetadataField.tag("country"),
|
||||
MetadataField.numeric("year"))
|
||||
.initializeSchema(true) // Optional: defaults to false
|
||||
.batchingStrategy(new TokenCountBatchingStrategy()) // Optional: defaults to TokenCountBatchingStrategy
|
||||
.build();
|
||||
}
|
||||
|
||||
// This can be any EmbeddingModel implementation
|
||||
@Bean
|
||||
public EmbeddingModel embeddingModel() {
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
It is more convenient and preferred to create the `RedisVectorStore` as a Bean.
|
||||
But if you decide to create it manually, then you must call the `RedisVectorStore#afterPropertiesSet()` after setting the properties and before using the client.
|
||||
You must list explicitly all metadata field names and types (`TAG`, `TEXT`, or `NUMERIC`) for any metadata field used in filter expressions.
|
||||
The `metadataFields` above registers filterable metadata fields: `country` of type `TAG`, `year` of type `NUMERIC`.
|
||||
====
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
You must list explicitly all metadata field names and types (`TAG`, `TEXT`, or `NUMERIC`) for any metadata field used in filter expression.
|
||||
The `withMetadataFields` above registers filterable metadata fields: `country` of type `TAG`, `year` of type `NUMERIC`.
|
||||
====
|
||||
|
||||
Then in your main code, create some documents:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
List<Document> documents = List.of(
|
||||
new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("country", "UK", "year", 2020)),
|
||||
new Document("The World is Big and Salvation Lurks Around the Corner", Map.of()),
|
||||
new Document("You walk forward facing the past and you turn back toward the future.", Map.of("country", "NL", "year", 2023)));
|
||||
----
|
||||
|
||||
Now add the documents to your vector store:
|
||||
|
||||
|
||||
[source,java]
|
||||
----
|
||||
vectorStore.add(documents);
|
||||
----
|
||||
|
||||
And finally, retrieve documents similar to a query:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
List<Document> results = vectorStore.similaritySearch(
|
||||
SearchRequest
|
||||
.query("Spring")
|
||||
.withTopK(5));
|
||||
----
|
||||
|
||||
If all goes well, you should retrieve the document containing the text "Spring AI rocks!!".
|
||||
|
||||
@@ -22,8 +22,7 @@ import redis.clients.jedis.JedisPooled;
|
||||
import org.springframework.ai.embedding.BatchingStrategy;
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.embedding.TokenCountBatchingStrategy;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.RedisVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.redis.RedisVectorStore;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationConvention;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
import org.springframework.boot.autoconfigure.AutoConfiguration;
|
||||
@@ -61,15 +60,16 @@ public class RedisVectorStoreAutoConfiguration {
|
||||
ObjectProvider<VectorStoreObservationConvention> customObservationConvention,
|
||||
BatchingStrategy batchingStrategy) {
|
||||
|
||||
var config = RedisVectorStoreConfig.builder()
|
||||
.withIndexName(properties.getIndex())
|
||||
.withPrefix(properties.getPrefix())
|
||||
return RedisVectorStore.builder()
|
||||
.jedis(new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()))
|
||||
.embeddingModel(embeddingModel)
|
||||
.initializeSchema(properties.isInitializeSchema())
|
||||
.observationRegistry(observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP))
|
||||
.customObservationConvention(customObservationConvention.getIfAvailable(() -> null))
|
||||
.batchingStrategy(batchingStrategy)
|
||||
.indexName(properties.getIndex())
|
||||
.prefix(properties.getPrefix())
|
||||
.build();
|
||||
|
||||
return new RedisVectorStore(config, embeddingModel,
|
||||
new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()),
|
||||
properties.isInitializeSchema(), observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP),
|
||||
customObservationConvention.getIfAvailable(() -> null), batchingStrategy);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.redis;
|
||||
|
||||
import java.text.MessageFormat;
|
||||
import java.util.List;
|
||||
@@ -22,7 +22,7 @@ import java.util.Map;
|
||||
import java.util.function.Function;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.redis.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.Expression;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.ExpressionType;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.Group;
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.redis;
|
||||
|
||||
import java.text.MessageFormat;
|
||||
import java.util.ArrayList;
|
||||
@@ -54,6 +54,9 @@ import org.springframework.ai.embedding.EmbeddingOptionsBuilder;
|
||||
import org.springframework.ai.embedding.TokenCountBatchingStrategy;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreProvider;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreSimilarityMetric;
|
||||
import org.springframework.ai.vectorstore.AbstractVectorStoreBuilder;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.ai.vectorstore.filter.FilterExpressionConverter;
|
||||
import org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationContext;
|
||||
@@ -61,21 +64,114 @@ import org.springframework.ai.vectorstore.observation.VectorStoreObservationConv
|
||||
import org.springframework.beans.factory.InitializingBean;
|
||||
import org.springframework.util.Assert;
|
||||
import org.springframework.util.CollectionUtils;
|
||||
import org.springframework.util.StringUtils;
|
||||
|
||||
/**
|
||||
* The RedisVectorStore is for managing and querying vector data in a Redis database. It
|
||||
* offers functionalities like adding, deleting, and performing similarity searches on
|
||||
* documents.
|
||||
* Redis-based vector store implementation using Redis Stack with RediSearch and
|
||||
* RedisJSON.
|
||||
*
|
||||
* <p>
|
||||
* The store uses Redis JSON documents to persist vector embeddings along with their
|
||||
* associated document content and metadata. It leverages RediSearch for creating and
|
||||
* querying vector similarity indexes. The RedisVectorStore manages and queries vector
|
||||
* data, offering functionalities like adding, deleting, and performing similarity
|
||||
* searches on documents.
|
||||
* </p>
|
||||
*
|
||||
* <p>
|
||||
* The store utilizes RedisJSON and RedisSearch to handle JSON documents and to index and
|
||||
* search vector data. It supports various vector algorithms (e.g., FLAT, HSNW) for
|
||||
* search vector data. It supports various vector algorithms (e.g., FLAT, HNSW) for
|
||||
* efficient similarity searches. Additionally, it allows for custom metadata fields in
|
||||
* the documents to be stored alongside the vector and content data.
|
||||
* </p>
|
||||
*
|
||||
* This class requires a RedisVectorStoreConfig configuration object for initialization,
|
||||
* which includes settings like Redis URI, index name, field names, and vector algorithms.
|
||||
* It also requires an EmbeddingModel to convert documents into embeddings before storing
|
||||
* them.
|
||||
* <p>
|
||||
* Features:
|
||||
* </p>
|
||||
* <ul>
|
||||
* <li>Automatic schema initialization with configurable index creation</li>
|
||||
* <li>Support for HNSW and FLAT vector indexing algorithms</li>
|
||||
* <li>Cosine similarity metric for vector comparisons</li>
|
||||
* <li>Flexible metadata field types (TEXT, TAG, NUMERIC) for advanced filtering</li>
|
||||
* <li>Configurable similarity thresholds for search results</li>
|
||||
* <li>Batch processing support with configurable batching strategies</li>
|
||||
* </ul>
|
||||
*
|
||||
* <p>
|
||||
* Basic usage example:
|
||||
* </p>
|
||||
* <pre>{@code
|
||||
* RedisVectorStore vectorStore = RedisVectorStore.builder()
|
||||
* .jedis(jedisPooled)
|
||||
* .embeddingModel(embeddingModel)
|
||||
* .indexName("custom-index") // Optional: defaults to "spring-ai-index"
|
||||
* .prefix("custom-prefix") // Optional: defaults to "embedding:"
|
||||
* .vectorAlgorithm(Algorithm.HNSW)
|
||||
* .build();
|
||||
*
|
||||
* // Add documents
|
||||
* vectorStore.add(List.of(
|
||||
* new Document("content1", Map.of("meta1", "value1")),
|
||||
* new Document("content2", Map.of("meta2", "value2"))
|
||||
* ));
|
||||
*
|
||||
* // Search with filters
|
||||
* List<Document> results = vectorStore.similaritySearch(
|
||||
* SearchRequest.query("search text")
|
||||
* .withTopK(5)
|
||||
* .withSimilarityThreshold(0.7)
|
||||
* .withFilterExpression("meta1 == 'value1'")
|
||||
* );
|
||||
* }</pre>
|
||||
*
|
||||
* <p>
|
||||
* Advanced configuration example:
|
||||
* </p>
|
||||
* <pre>{@code
|
||||
* RedisVectorStore vectorStore = RedisVectorStore.builder()
|
||||
* .jedis(jedisPooled)
|
||||
* .embeddingModel(embeddingModel)
|
||||
* .indexName("custom-index")
|
||||
* .prefix("custom-prefix")
|
||||
* .contentFieldName("custom_content")
|
||||
* .embeddingFieldName("custom_embedding")
|
||||
* .vectorAlgorithm(Algorithm.FLAT)
|
||||
* .metadataFields(
|
||||
* MetadataField.tag("category"),
|
||||
* MetadataField.numeric("year"),
|
||||
* MetadataField.text("description"))
|
||||
* .initializeSchema(true)
|
||||
* .batchingStrategy(new TokenCountBatchingStrategy())
|
||||
* .build();
|
||||
* }</pre>
|
||||
*
|
||||
* <p>
|
||||
* Database Requirements:
|
||||
* </p>
|
||||
* <ul>
|
||||
* <li>Redis Stack with RediSearch and RedisJSON modules</li>
|
||||
* <li>Redis version 7.0 or higher</li>
|
||||
* <li>Sufficient memory for storing vectors and indexes</li>
|
||||
* </ul>
|
||||
*
|
||||
* <p>
|
||||
* Vector Algorithms:
|
||||
* </p>
|
||||
* <ul>
|
||||
* <li>HNSW: Default algorithm, provides better search performance with slightly higher
|
||||
* memory usage</li>
|
||||
* <li>FLAT: Brute force algorithm, provides exact results but slower for large
|
||||
* datasets</li>
|
||||
* </ul>
|
||||
*
|
||||
* <p>
|
||||
* Metadata Field Types:
|
||||
* </p>
|
||||
* <ul>
|
||||
* <li>TAG: For exact match filtering on categorical data</li>
|
||||
* <li>TEXT: For full-text search capabilities</li>
|
||||
* <li>NUMERIC: For range queries on numerical data</li>
|
||||
* </ul>
|
||||
*
|
||||
* @author Julien Ruaux
|
||||
* @author Christian Tzolov
|
||||
@@ -86,6 +182,7 @@ import org.springframework.util.CollectionUtils;
|
||||
* @see VectorStore
|
||||
* @see RedisVectorStoreConfig
|
||||
* @see EmbeddingModel
|
||||
* @since 1.0.0
|
||||
*/
|
||||
public class RedisVectorStore extends AbstractObservationVectorStore implements InitializingBean {
|
||||
|
||||
@@ -119,40 +216,67 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
|
||||
private static final String DEFAULT_DISTANCE_METRIC = "COSINE";
|
||||
|
||||
private final boolean initializeSchema;
|
||||
|
||||
private final JedisPooled jedis;
|
||||
|
||||
private final EmbeddingModel embeddingModel;
|
||||
private final boolean initializeSchema;
|
||||
|
||||
private final RedisVectorStoreConfig config;
|
||||
private final String indexName;
|
||||
|
||||
private final String prefix;
|
||||
|
||||
private final String contentFieldName;
|
||||
|
||||
private final String embeddingFieldName;
|
||||
|
||||
private final Algorithm vectorAlgorithm;
|
||||
|
||||
private final List<MetadataField> metadataFields;
|
||||
|
||||
private final BatchingStrategy batchingStrategy;
|
||||
|
||||
private FilterExpressionConverter filterExpressionConverter;
|
||||
private final FilterExpressionConverter filterExpressionConverter;
|
||||
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public RedisVectorStore(RedisVectorStoreConfig config, EmbeddingModel embeddingModel, JedisPooled jedis,
|
||||
boolean initializeSchema) {
|
||||
|
||||
this(config, embeddingModel, jedis, initializeSchema, ObservationRegistry.NOOP, null,
|
||||
new TokenCountBatchingStrategy());
|
||||
}
|
||||
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public RedisVectorStore(RedisVectorStoreConfig config, EmbeddingModel embeddingModel, JedisPooled jedis,
|
||||
boolean initializeSchema, ObservationRegistry observationRegistry,
|
||||
VectorStoreObservationConvention customObservationConvention, BatchingStrategy batchingStrategy) {
|
||||
|
||||
super(observationRegistry, customObservationConvention);
|
||||
this(builder().jedis(jedis)
|
||||
.embeddingModel(embeddingModel)
|
||||
.indexName(config.indexName)
|
||||
.prefix(config.prefix)
|
||||
.contentFieldName(config.contentFieldName)
|
||||
.embeddingFieldName(config.embeddingFieldName)
|
||||
.vectorAlgorithm(config.vectorAlgorithm)
|
||||
.metadataFields(config.metadataFields)
|
||||
.initializeSchema(initializeSchema)
|
||||
.observationRegistry(observationRegistry)
|
||||
.customObservationConvention(customObservationConvention)
|
||||
.batchingStrategy(batchingStrategy));
|
||||
}
|
||||
|
||||
Assert.notNull(config, "Config must not be null");
|
||||
Assert.notNull(embeddingModel, "Embedding model must not be null");
|
||||
this.initializeSchema = initializeSchema;
|
||||
protected RedisVectorStore(RedisBuilder builder) {
|
||||
super(builder);
|
||||
|
||||
this.jedis = jedis;
|
||||
this.embeddingModel = embeddingModel;
|
||||
this.config = config;
|
||||
this.filterExpressionConverter = new RedisFilterExpressionConverter(this.config.metadataFields);
|
||||
this.batchingStrategy = batchingStrategy;
|
||||
Assert.notNull(builder.jedis, "JedisPooled must not be null");
|
||||
|
||||
this.jedis = builder.jedis;
|
||||
this.indexName = builder.indexName;
|
||||
this.prefix = builder.prefix;
|
||||
this.contentFieldName = builder.contentFieldName;
|
||||
this.embeddingFieldName = builder.embeddingFieldName;
|
||||
this.vectorAlgorithm = builder.vectorAlgorithm;
|
||||
this.metadataFields = builder.metadataFields;
|
||||
this.initializeSchema = builder.initializeSchema;
|
||||
this.batchingStrategy = builder.batchingStrategy;
|
||||
this.filterExpressionConverter = new RedisFilterExpressionConverter(this.metadataFields);
|
||||
}
|
||||
|
||||
public JedisPooled getJedis() {
|
||||
@@ -168,8 +292,8 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
|
||||
for (Document document : documents) {
|
||||
var fields = new HashMap<String, Object>();
|
||||
fields.put(this.config.embeddingFieldName, embeddings.get(documents.indexOf(document)));
|
||||
fields.put(this.config.contentFieldName, document.getContent());
|
||||
fields.put(this.embeddingFieldName, embeddings.get(documents.indexOf(document)));
|
||||
fields.put(this.contentFieldName, document.getContent());
|
||||
fields.putAll(document.getMetadata());
|
||||
pipeline.jsonSetWithEscape(key(document.getId()), JSON_SET_PATH, fields);
|
||||
}
|
||||
@@ -186,7 +310,7 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
}
|
||||
|
||||
private String key(String id) {
|
||||
return this.config.prefix + id;
|
||||
return this.prefix + id;
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -216,13 +340,13 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
|
||||
String filter = nativeExpressionFilter(request);
|
||||
|
||||
String queryString = String.format(QUERY_FORMAT, filter, request.getTopK(), this.config.embeddingFieldName,
|
||||
String queryString = String.format(QUERY_FORMAT, filter, request.getTopK(), this.embeddingFieldName,
|
||||
EMBEDDING_PARAM_NAME, DISTANCE_FIELD_NAME);
|
||||
|
||||
List<String> returnFields = new ArrayList<>();
|
||||
this.config.metadataFields.stream().map(MetadataField::name).forEach(returnFields::add);
|
||||
returnFields.add(this.config.embeddingFieldName);
|
||||
returnFields.add(this.config.contentFieldName);
|
||||
this.metadataFields.stream().map(MetadataField::name).forEach(returnFields::add);
|
||||
returnFields.add(this.embeddingFieldName);
|
||||
returnFields.add(this.contentFieldName);
|
||||
returnFields.add(DISTANCE_FIELD_NAME);
|
||||
var embedding = this.embeddingModel.embed(request.getQuery());
|
||||
Query query = new Query(queryString).addParam(EMBEDDING_PARAM_NAME, RediSearchUtil.toByteArray(embedding))
|
||||
@@ -231,7 +355,7 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
.limit(0, request.getTopK())
|
||||
.dialect(2);
|
||||
|
||||
SearchResult result = this.jedis.ftSearch(this.config.indexName, query);
|
||||
SearchResult result = this.jedis.ftSearch(this.indexName, query);
|
||||
return result.getDocuments()
|
||||
.stream()
|
||||
.filter(d -> similarityScore(d) >= request.getSimilarityThreshold())
|
||||
@@ -240,14 +364,12 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
}
|
||||
|
||||
private Document toDocument(redis.clients.jedis.search.Document doc) {
|
||||
var id = doc.getId().substring(this.config.prefix.length());
|
||||
var content = doc.hasProperty(this.config.contentFieldName) ? doc.getString(this.config.contentFieldName) : "";
|
||||
Map<String, Object> metadata = this.config.metadataFields.stream()
|
||||
var id = doc.getId().substring(this.prefix.length());
|
||||
var content = doc.hasProperty(this.contentFieldName) ? doc.getString(this.contentFieldName) : "";
|
||||
Map<String, Object> metadata = this.metadataFields.stream()
|
||||
.map(MetadataField::name)
|
||||
.filter(doc::hasProperty)
|
||||
.collect(Collectors.toMap(Function.identity(), doc::getString));
|
||||
// TODO: this seems wrong. The key is named "vector_store", but the value is the
|
||||
// distance. Can we remove this after standardizing the metadata?
|
||||
metadata.put(DISTANCE_FIELD_NAME, 1 - similarityScore(doc));
|
||||
metadata.put(DocumentMetadata.DISTANCE.value(), 1 - similarityScore(doc));
|
||||
return Document.builder().id(id).text(content).metadata(metadata).score((double) similarityScore(doc)).build();
|
||||
@@ -272,12 +394,12 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
}
|
||||
|
||||
// If index already exists don't do anything
|
||||
if (this.jedis.ftList().contains(this.config.indexName)) {
|
||||
if (this.jedis.ftList().contains(this.indexName)) {
|
||||
return;
|
||||
}
|
||||
|
||||
String response = this.jedis.ftCreate(this.config.indexName,
|
||||
FTCreateParams.createParams().on(IndexDataType.JSON).addPrefix(this.config.prefix), schemaFields());
|
||||
String response = this.jedis.ftCreate(this.indexName,
|
||||
FTCreateParams.createParams().on(IndexDataType.JSON).addPrefix(this.prefix), schemaFields());
|
||||
if (!RESPONSE_OK.test(response)) {
|
||||
String message = MessageFormat.format("Could not create index: {0}", response);
|
||||
throw new RuntimeException(message);
|
||||
@@ -290,16 +412,16 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
vectorAttrs.put("DISTANCE_METRIC", DEFAULT_DISTANCE_METRIC);
|
||||
vectorAttrs.put("TYPE", VECTOR_TYPE_FLOAT32);
|
||||
List<SchemaField> fields = new ArrayList<>();
|
||||
fields.add(TextField.of(jsonPath(this.config.contentFieldName)).as(this.config.contentFieldName).weight(1.0));
|
||||
fields.add(TextField.of(jsonPath(this.contentFieldName)).as(this.contentFieldName).weight(1.0));
|
||||
fields.add(VectorField.builder()
|
||||
.fieldName(jsonPath(this.config.embeddingFieldName))
|
||||
.fieldName(jsonPath(this.embeddingFieldName))
|
||||
.algorithm(vectorAlgorithm())
|
||||
.attributes(vectorAttrs)
|
||||
.as(this.config.embeddingFieldName)
|
||||
.as(this.embeddingFieldName)
|
||||
.build());
|
||||
|
||||
if (!CollectionUtils.isEmpty(this.config.metadataFields)) {
|
||||
for (MetadataField field : this.config.metadataFields) {
|
||||
if (!CollectionUtils.isEmpty(this.metadataFields)) {
|
||||
for (MetadataField field : this.metadataFields) {
|
||||
fields.add(schemaField(field));
|
||||
}
|
||||
}
|
||||
@@ -318,7 +440,7 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
}
|
||||
|
||||
private VectorAlgorithm vectorAlgorithm() {
|
||||
if (this.config.vectorAlgorithm == Algorithm.HSNW) {
|
||||
if (this.vectorAlgorithm == Algorithm.HSNW) {
|
||||
return VectorAlgorithm.HNSW;
|
||||
}
|
||||
return VectorAlgorithm.FLAT;
|
||||
@@ -332,9 +454,9 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
public VectorStoreObservationContext.Builder createObservationContextBuilder(String operationName) {
|
||||
|
||||
return VectorStoreObservationContext.builder(VectorStoreProvider.REDIS.value(), operationName)
|
||||
.withCollectionName(this.config.indexName)
|
||||
.withCollectionName(this.indexName)
|
||||
.withDimensions(this.embeddingModel.dimensions())
|
||||
.withFieldName(this.config.embeddingFieldName)
|
||||
.withFieldName(this.embeddingFieldName)
|
||||
.withSimilarityMetric(VectorStoreSimilarityMetric.COSINE.value());
|
||||
|
||||
}
|
||||
@@ -361,9 +483,151 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
|
||||
}
|
||||
|
||||
public static RedisBuilder builder() {
|
||||
return new RedisBuilder();
|
||||
}
|
||||
|
||||
public static class RedisBuilder extends AbstractVectorStoreBuilder<RedisBuilder> {
|
||||
|
||||
private JedisPooled jedis;
|
||||
|
||||
private String indexName = DEFAULT_INDEX_NAME;
|
||||
|
||||
private String prefix = DEFAULT_PREFIX;
|
||||
|
||||
private String contentFieldName = DEFAULT_CONTENT_FIELD_NAME;
|
||||
|
||||
private String embeddingFieldName = DEFAULT_EMBEDDING_FIELD_NAME;
|
||||
|
||||
private Algorithm vectorAlgorithm = DEFAULT_VECTOR_ALGORITHM;
|
||||
|
||||
private List<MetadataField> metadataFields = new ArrayList<>();
|
||||
|
||||
private boolean initializeSchema = false;
|
||||
|
||||
private BatchingStrategy batchingStrategy = new TokenCountBatchingStrategy();
|
||||
|
||||
public RedisBuilder jedis(JedisPooled jedis) {
|
||||
Assert.notNull(jedis, "JedisPooled must not be null");
|
||||
this.jedis = jedis;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the Redis index name.
|
||||
* @param indexName the index name to use
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder indexName(String indexName) {
|
||||
if (StringUtils.hasText(indexName)) {
|
||||
this.indexName = indexName;
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the Redis key prefix (default: "embedding:").
|
||||
* @param prefix the prefix to use
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder prefix(String prefix) {
|
||||
if (StringUtils.hasText(prefix)) {
|
||||
this.prefix = prefix;
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the Redis content field name.
|
||||
* @param fieldName the content field name to use
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder contentFieldName(String fieldName) {
|
||||
if (StringUtils.hasText(fieldName)) {
|
||||
this.contentFieldName = fieldName;
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the Redis embedding field name.
|
||||
* @param fieldName the embedding field name to use
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder embeddingFieldName(String fieldName) {
|
||||
if (StringUtils.hasText(fieldName)) {
|
||||
this.embeddingFieldName = fieldName;
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the Redis vector algorithm.
|
||||
* @param algorithm the vector algorithm to use
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder vectorAlgorithm(Algorithm algorithm) {
|
||||
if (algorithm != null) {
|
||||
this.vectorAlgorithm = algorithm;
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the metadata fields.
|
||||
* @param fields the metadata fields to include
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder metadataFields(MetadataField... fields) {
|
||||
return metadataFields(Arrays.asList(fields));
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the metadata fields.
|
||||
* @param fields the list of metadata fields to include
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder metadataFields(List<MetadataField> fields) {
|
||||
if (fields != null && !fields.isEmpty()) {
|
||||
this.metadataFields = new ArrayList<>(fields);
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets whether to initialize the schema.
|
||||
* @param initializeSchema true to initialize schema, false otherwise
|
||||
* @return the builder instance
|
||||
*/
|
||||
public RedisBuilder initializeSchema(boolean initializeSchema) {
|
||||
this.initializeSchema = initializeSchema;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the batching strategy.
|
||||
* @param batchingStrategy the strategy to use
|
||||
* @return the builder instance
|
||||
* @throws IllegalArgumentException if batchingStrategy is null
|
||||
*/
|
||||
public RedisBuilder batchingStrategy(BatchingStrategy batchingStrategy) {
|
||||
Assert.notNull(batchingStrategy, "BatchingStrategy must not be null");
|
||||
this.batchingStrategy = batchingStrategy;
|
||||
return this;
|
||||
}
|
||||
|
||||
@Override
|
||||
public RedisVectorStore build() {
|
||||
validate();
|
||||
return new RedisVectorStore(this);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Configuration for the Redis vector store.
|
||||
*/
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public static final class RedisVectorStoreConfig {
|
||||
|
||||
private final String indexName;
|
||||
@@ -395,19 +659,20 @@ public class RedisVectorStore extends AbstractObservationVectorStore implements
|
||||
* Start building a new configuration.
|
||||
* @return The entry point for creating a new configuration.
|
||||
*/
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public static Builder builder() {
|
||||
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* {@return the default config}
|
||||
*/
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public static RedisVectorStoreConfig defaultConfig() {
|
||||
|
||||
return builder().build();
|
||||
}
|
||||
|
||||
@Deprecated(since = "1.0.0-M5", forRemoval = true)
|
||||
public static final class Builder {
|
||||
|
||||
private String indexName = DEFAULT_INDEX_NAME;
|
||||
@@ -14,14 +14,14 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.redis;
|
||||
|
||||
import java.util.Arrays;
|
||||
import java.util.List;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.redis.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.Expression;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.Group;
|
||||
import org.springframework.ai.vectorstore.filter.Filter.Key;
|
||||
@@ -49,7 +49,7 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void testEQ() {
|
||||
// country == "BG"
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.tag("country"))
|
||||
.convertExpression(new Expression(EQ, new Key("country"), new Value("BG")));
|
||||
assertThat(vectorExpr).isEqualTo("@country:{BG}");
|
||||
}
|
||||
@@ -57,8 +57,8 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void tesEqAndGte() {
|
||||
// genre == "drama" AND year >= 2020
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("genre"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.numeric("year"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.tag("genre"),
|
||||
RedisVectorStore.MetadataField.numeric("year"))
|
||||
.convertExpression(new Expression(AND, new Expression(EQ, new Key("genre"), new Value("drama")),
|
||||
new Expression(GTE, new Key("year"), new Value(2020))));
|
||||
assertThat(vectorExpr).isEqualTo("@genre:{drama} @year:[2020 inf]");
|
||||
@@ -67,18 +67,16 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void tesIn() {
|
||||
// genre in ["comedy", "documentary", "drama"]
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("genre"))
|
||||
.convertExpression(
|
||||
new Expression(IN, new Key("genre"), new Value(List.of("comedy", "documentary", "drama"))));
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.tag("genre")).convertExpression(
|
||||
new Expression(IN, new Key("genre"), new Value(List.of("comedy", "documentary", "drama"))));
|
||||
assertThat(vectorExpr).isEqualTo("@genre:{comedy | documentary | drama}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void testNe() {
|
||||
// year >= 2020 OR country == "BG" AND city != "Sofia"
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.numeric("year"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("city"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.numeric("year"),
|
||||
RedisVectorStore.MetadataField.tag("country"), RedisVectorStore.MetadataField.tag("city"))
|
||||
.convertExpression(new Expression(OR, new Expression(GTE, new Key("year"), new Value(2020)),
|
||||
new Group(new Expression(AND, new Expression(EQ, new Key("country"), new Value("BG")),
|
||||
new Expression(NE, new Key("city"), new Value("Sofia"))))));
|
||||
@@ -88,9 +86,8 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void testGroup() {
|
||||
// (year >= 2020 OR country == "BG") AND city NIN ["Sofia", "Plovdiv"]
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.numeric("year"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("city"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.numeric("year"),
|
||||
RedisVectorStore.MetadataField.tag("country"), RedisVectorStore.MetadataField.tag("city"))
|
||||
.convertExpression(new Expression(AND,
|
||||
new Group(new Expression(OR, new Expression(GTE, new Key("year"), new Value(2020)),
|
||||
new Expression(EQ, new Key("country"), new Value("BG")))),
|
||||
@@ -101,9 +98,8 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void tesBoolean() {
|
||||
// isOpen == true AND year >= 2020 AND country IN ["BG", "NL", "US"]
|
||||
String vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.numeric("year"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country"),
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("isOpen"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.numeric("year"),
|
||||
RedisVectorStore.MetadataField.tag("country"), RedisVectorStore.MetadataField.tag("isOpen"))
|
||||
.convertExpression(new Expression(AND,
|
||||
new Expression(AND, new Expression(EQ, new Key("isOpen"), new Value(true)),
|
||||
new Expression(GTE, new Key("year"), new Value(2020))),
|
||||
@@ -115,8 +111,7 @@ class RedisFilterExpressionConverterTests {
|
||||
@Test
|
||||
void testDecimal() {
|
||||
// temperature >= -15.6 && temperature <= +20.13
|
||||
String vectorExpr = converter(
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.numeric("temperature"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.numeric("temperature"))
|
||||
.convertExpression(new Expression(AND, new Expression(GTE, new Key("temperature"), new Value(-15.6)),
|
||||
new Expression(LTE, new Key("temperature"), new Value(20.13))));
|
||||
|
||||
@@ -125,12 +120,11 @@ class RedisFilterExpressionConverterTests {
|
||||
|
||||
@Test
|
||||
void testComplexIdentifiers() {
|
||||
String vectorExpr = converter(
|
||||
org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country 1 2 3"))
|
||||
String vectorExpr = converter(RedisVectorStore.MetadataField.tag("country 1 2 3"))
|
||||
.convertExpression(new Expression(EQ, new Key("\"country 1 2 3\""), new Value("BG")));
|
||||
assertThat(vectorExpr).isEqualTo("@\"country 1 2 3\":{BG}");
|
||||
|
||||
vectorExpr = converter(org.springframework.ai.vectorstore.RedisVectorStore.MetadataField.tag("country 1 2 3"))
|
||||
vectorExpr = converter(RedisVectorStore.MetadataField.tag("country 1 2 3"))
|
||||
.convertExpression(new Expression(EQ, new Key("'country 1 2 3'"), new Value("BG")));
|
||||
assertThat(vectorExpr).isEqualTo("@'country 1 2 3':{BG}");
|
||||
}
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.redis;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
@@ -34,8 +34,9 @@ import redis.clients.jedis.JedisPooled;
|
||||
import org.springframework.ai.document.Document;
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.RedisVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.redis.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.boot.SpringBootConfiguration;
|
||||
import org.springframework.boot.autoconfigure.AutoConfigurations;
|
||||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||
@@ -255,13 +256,13 @@ class RedisVectorStoreIT {
|
||||
@Bean
|
||||
public RedisVectorStore vectorStore(EmbeddingModel embeddingModel,
|
||||
JedisConnectionFactory jedisConnectionFactory) {
|
||||
return new RedisVectorStore(
|
||||
RedisVectorStoreConfig.builder()
|
||||
.withMetadataFields(MetadataField.tag("meta1"), MetadataField.tag("meta2"),
|
||||
MetadataField.tag("country"), MetadataField.numeric("year"))
|
||||
.build(),
|
||||
embeddingModel,
|
||||
new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()), true);
|
||||
return RedisVectorStore.builder()
|
||||
.jedis(new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()))
|
||||
.embeddingModel(embeddingModel)
|
||||
.metadataFields(MetadataField.tag("meta1"), MetadataField.tag("meta2"), MetadataField.tag("country"),
|
||||
MetadataField.numeric("year"))
|
||||
.initializeSchema(true)
|
||||
.build();
|
||||
}
|
||||
|
||||
@Bean
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.redis;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
@@ -38,8 +38,9 @@ import org.springframework.ai.observation.conventions.SpringAiKind;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreProvider;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreSimilarityMetric;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.RedisVectorStore.RedisVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.redis.RedisVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.ai.vectorstore.observation.DefaultVectorStoreObservationConvention;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationDocumentation.HighCardinalityKeyNames;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationDocumentation.LowCardinalityKeyNames;
|
||||
@@ -175,14 +176,16 @@ public class RedisVectorStoreObservationIT {
|
||||
@Bean
|
||||
public RedisVectorStore vectorStore(EmbeddingModel embeddingModel,
|
||||
JedisConnectionFactory jedisConnectionFactory, ObservationRegistry observationRegistry) {
|
||||
return new RedisVectorStore(
|
||||
RedisVectorStoreConfig.builder()
|
||||
.withMetadataFields(MetadataField.tag("meta1"), MetadataField.tag("meta2"),
|
||||
MetadataField.tag("country"), MetadataField.numeric("year"))
|
||||
.build(),
|
||||
embeddingModel,
|
||||
new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()), true,
|
||||
observationRegistry, null, new TokenCountBatchingStrategy());
|
||||
return RedisVectorStore.builder()
|
||||
.jedis(new JedisPooled(jedisConnectionFactory.getHostName(), jedisConnectionFactory.getPort()))
|
||||
.embeddingModel(embeddingModel)
|
||||
.observationRegistry(observationRegistry)
|
||||
.customObservationConvention(null)
|
||||
.initializeSchema(true)
|
||||
.batchingStrategy(new TokenCountBatchingStrategy())
|
||||
.metadataFields(MetadataField.tag("meta1"), MetadataField.tag("meta2"), MetadataField.tag("country"),
|
||||
MetadataField.numeric("year"))
|
||||
.build();
|
||||
}
|
||||
|
||||
@Bean
|
||||
Reference in New Issue
Block a user