Add builder pattern to WeaviateVectorStore and refactor package name
Introduces a builder pattern for configuring WeaviateVectorStore instances and moves the implementation to the org.springframework.ai.vectorstore.weaviate package. This change: - Makes configuration more flexible and type-safe through builder methods - Improves code organization by moving to a dedicated vector store package - Deprecates old constructors in favor of the builder pattern - Adds builder tests - Enables better IDE support through method chaining
This commit is contained in:
Soby Chacko
committed by
Mark Pollack
Mark Pollack
parent
25123a5364
commit
d77c950ea6
@@ -1,200 +1,20 @@
|
||||
= Weaviate
|
||||
|
||||
This section will walk you through setting up the Weaviate VectorStore to store document embeddings and perform similarity searches.
|
||||
This section walks you through setting up the Weaviate VectorStore to store document embeddings and perform similarity searches.
|
||||
|
||||
== What is Weaviate?
|
||||
|
||||
link:https://weaviate.io/[Weaviate] is an open-source vector database.
|
||||
It allows you to store data objects and vector embeddings from your favorite ML-models and scale seamlessly into billions of data objects.
|
||||
link:https://weaviate.io/[Weaviate] is an open-source vector database that allows you to store data objects and vector embeddings from your favorite ML-models and scale seamlessly into billions of data objects.
|
||||
It provides tools to store document embeddings, content, and metadata and to search through those embeddings, including metadata filtering.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
1. `EmbeddingModel` instance to compute the document embeddings. Several options are available:
|
||||
* A running Weaviate instance. The following options are available:
|
||||
** link:https://console.weaviate.cloud/[Weaviate Cloud Service] (requires account creation and API key)
|
||||
** link:https://weaviate.io/developers/weaviate/installation/docker[Docker container]
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `WeaviateVectorStore`.
|
||||
|
||||
- `Transformers Embedding` - computes the embedding in your local environment. Follow the ONNX Transformers Embedding instructions.
|
||||
- `OpenAI Embedding` - uses the OpenAI embedding endpoint. You need to create an account at link:https://platform.openai.com/signup[OpenAI Signup] and generate the api-key token at link:https://platform.openai.com/account/api-keys[API Keys].
|
||||
- You can also use the `Azure OpenAI Embedding` or the `PostgresML Embedding Model`.
|
||||
2. `Weaviate cluster`. You can set up a cluster locally in a Docker container or create a link:https://console.weaviate.cloud/[Weaviate Cloud Service]. For the latter, you need to create a Weaviate account, set up a cluster, and get your access API key from the link:https://console.weaviate.cloud/dashboard[dashboard details].
|
||||
== Dependencies
|
||||
|
||||
On startup, the `WeaviateVectorStore` creates the required `SpringAiWeaviate` object schema if it's not already provisioned.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
Then add the WeaviateVectorStore boot starter dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-weaviate-store-spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-weaviate-store-spring-boot-starter'
|
||||
}
|
||||
----
|
||||
|
||||
The vector store implementation can initialize the requisite schema for you, but you must opt-in by specifying the `initializeSchema` boolean in the appropriate constructor or by setting `...initialize-schema=true` in the `application.properties` file.
|
||||
|
||||
NOTE: this is a breaking change! In earlier versions of Spring AI, this schema initialization happened by default.
|
||||
|
||||
|
||||
The Vector Store, also requires an `EmbeddingModel` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingModel Implementations].
|
||||
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingModel] add the following dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-openai-spring-boot-starter'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
To connect to Weaviate and use the `WeaviateVectorStore`, you need to provide access details for your instance.
|
||||
A simple configuration can either be provided via Spring Boot's _application.properties_,
|
||||
|
||||
[source,properties]
|
||||
----
|
||||
spring.ai.vectorstore.weaviate.host=<host of your Weaviate instance>
|
||||
spring.ai.vectorstore.weaviate.api-key=<your api key>
|
||||
spring.ai.vectorstore.weaviate.scheme=http
|
||||
|
||||
# API key if needed, e.g. OpenAI
|
||||
spring.ai.openai.api.key=<api-key>
|
||||
----
|
||||
|
||||
TIP: Check the list of xref:#weaviate-vectorstore-properties[configuration parameters] to learn about the default values and configuration options.
|
||||
|
||||
Now you can Auto-wire the Weaviate Vector Store in your application and use it
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Autowired VectorStore vectorStore;
|
||||
|
||||
// ...
|
||||
|
||||
List <Document> documents = List.of(
|
||||
new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
|
||||
new Document("The World is Big and Salvation Lurks Around the Corner"),
|
||||
new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));
|
||||
|
||||
// Add the documents
|
||||
vectorStore.add(documents);
|
||||
|
||||
// Retrieve documents similar to a query
|
||||
List<Document> results = this.vectorStore.similaritySearch(SearchRequest.query("Spring").withTopK(5));
|
||||
----
|
||||
|
||||
[[weaviate-vectorstore-properties]]
|
||||
=== Configuration properties
|
||||
|
||||
You can use the following properties in your Spring Boot configuration to customize the weaviate vector store.
|
||||
|
||||
[cols="3,5,1",stripes=even]
|
||||
|===
|
||||
|Property| Description | Default value
|
||||
|
||||
|`spring.ai.vectorstore.weaviate.host`| The host of the Weaviate server. | localhost:8080
|
||||
|`spring.ai.vectorstore.weaviate.scheme`| Connection schema. | http
|
||||
|`spring.ai.vectorstore.weaviate.api-key`| The API key to use for authentication with the Weaviate server. | -
|
||||
|`spring.ai.vectorstore.weaviate.object-class`| | "SpringAiWeaviate"
|
||||
|`spring.ai.vectorstore.weaviate.consistency-level`| Desired tradeoff between consistency and speed | ConsistentLevel.ONE
|
||||
|`spring.ai.vectorstore.weaviate.filter-field`| spring.ai.vectorstore.weaviate.filter-field.<field-name>=<field-type> | -
|
||||
|`spring.ai.vectorstore.weaviate.headers`| | -
|
||||
|`spring.ai.vectorstore.weaviate.initialize-schema`| Whether to initialize the required schema | `false`
|
||||
|===
|
||||
|
||||
== Metadata filtering
|
||||
|
||||
You can leverage the generic, portable link:https://docs.spring.io/spring-ai/reference/api/vectordbs.html#_metadata_filters[metadata filters] with WeaviateVectorStore as well.
|
||||
|
||||
For example, you can use either the text expression language:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
vectorStore.similaritySearch(
|
||||
SearchRequest
|
||||
.query("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression("country in ['UK', 'NL'] && year >= 2020"));
|
||||
----
|
||||
|
||||
or programmatically using the expression DSL:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
FilterExpressionBuilder b = new FilterExpressionBuilder();
|
||||
|
||||
vectorStore.similaritySearch(
|
||||
SearchRequest
|
||||
.query("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression(b.and(
|
||||
b.in("country", "UK", "NL"),
|
||||
b.gte("year", 2020)).build()));
|
||||
----
|
||||
|
||||
The portable filter expressions get automatically converted into the proprietary Weaviate link:https://weaviate.io/developers/weaviate/api/graphql/filters[where filters].
|
||||
For example, the following portable filter expression:
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
country in ['UK', 'NL'] && year >= 2020
|
||||
----
|
||||
|
||||
is converted into Weaviate GraphQL link:https://weaviate.io/developers/weaviate/api/graphql/filters[where filter expression]:
|
||||
|
||||
[source,graphql]
|
||||
----
|
||||
operator:And
|
||||
operands:
|
||||
[{
|
||||
operator:Or
|
||||
operands:
|
||||
[{
|
||||
path:["meta_country"]
|
||||
operator:Equal
|
||||
valueText:"UK"
|
||||
},
|
||||
{
|
||||
path:["meta_country"]
|
||||
operator:Equal
|
||||
valueText:"NL"
|
||||
}]
|
||||
},
|
||||
{
|
||||
path:["meta_year"]
|
||||
operator:GreaterThanEqual
|
||||
valueNumber:2020
|
||||
}]
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
Instead of using the Spring Boot auto-configuration, you can manually configure the `WeaviateVectorStore`.
|
||||
For this you need to add the `spring-ai-weaviate-store` dependency to your project:
|
||||
Add the Weaviate Vector Store dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
@@ -213,46 +33,202 @@ dependencies {
|
||||
}
|
||||
----
|
||||
|
||||
To configure Weaviate in your application, you can create a WeaviateClient:
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
== Configuration
|
||||
|
||||
To connect to Weaviate and use the `WeaviateVectorStore`, you need to provide access details for your instance.
|
||||
A simple configuration can either be provided via Spring Boot's _application.properties_,
|
||||
|
||||
[source,properties]
|
||||
----
|
||||
spring.ai.vectorstore.weaviate.host=<host_of_your_weaviate_instance>
|
||||
spring.ai.vectorstore.weaviate.scheme=<http_or_https>
|
||||
spring.ai.vectorstore.weaviate.api-key=<your_api_key>
|
||||
# API key if needed, e.g. OpenAI
|
||||
spring.ai.openai.api-key=<api-key>
|
||||
----
|
||||
|
||||
environment variables,
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
export SPRING_AI_VECTORSTORE_WEAVIATE_HOST=<host_of_your_weaviate_instance>
|
||||
export SPRING_AI_VECTORSTORE_WEAVIATE_SCHEME=<http_or_https>
|
||||
export SPRING_AI_VECTORSTORE_WEAVIATE_API_KEY=<your_api_key>
|
||||
# API key if needed, e.g. OpenAI
|
||||
export SPRING_AI_OPENAI_API_KEY=<api-key>
|
||||
----
|
||||
|
||||
or can be a mix of those.
|
||||
|
||||
NOTE: If you choose to create a shell script for ease in future work, be sure to run it prior to starting your application by "sourcing" the file, i.e. `source <your_script_name>.sh`.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
Spring AI provides Spring Boot auto-configuration for the Weaviate Vector Store.
|
||||
To enable it, add the following dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-weaviate-store-spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-weaviate-store-spring-boot-starter'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Please have a look at the list of xref:#_weaviatevectorstore_properties[configuration parameters] for the vector store to learn about the default values and configuration options.
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other Embeddingmodel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
Now you can auto-wire the `WeaviateVectorStore` as a vector store in your application.
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
Instead of using Spring Boot auto-configuration, you can manually configure the `WeaviateVectorStore` using the builder pattern:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public WeaviateClient weaviateClient() {
|
||||
try {
|
||||
return WeaviateAuthClient.apiKey(
|
||||
new Config(<YOUR SCHEME>, <YOUR HOST>, <YOUR HEADERS>),
|
||||
<YOUR API KEY>);
|
||||
}
|
||||
catch (AuthException e) {
|
||||
throw new IllegalArgumentException("WeaviateClient could not be created.", e);
|
||||
}
|
||||
return new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
}
|
||||
|
||||
@Bean
|
||||
public VectorStore vectorStore(EmbeddingModel embeddingModel, WeaviateClient weaviateClient) {
|
||||
return WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.objectClass("CustomClass") // Optional: defaults to "SpringAiWeaviate"
|
||||
.consistencyLevel(ConsistentLevel.QUORUM) // Optional: defaults to ConsistentLevel.ONE
|
||||
.filterMetadataFields(List.of( // Optional: fields that can be used in filters
|
||||
MetadataField.text("country"),
|
||||
MetadataField.number("year")))
|
||||
.build();
|
||||
}
|
||||
----
|
||||
|
||||
Integrate with OpenAI's embeddings by adding the Spring Boot OpenAI starter to your project.
|
||||
This provides you with an implementation of the Embeddings client:
|
||||
== Metadata filtering
|
||||
|
||||
You can leverage the generic, portable xref:api/vectordbs.adoc#metadata-filters[metadata filters] with Weaviate store as well.
|
||||
|
||||
For example, you can use either the text expression language:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public WeaviateVectorStore vectorStore(EmbeddingModel embeddingModel, WeaviateClient weaviateClient) {
|
||||
|
||||
WeaviateVectorStoreConfig.Builder configBuilder = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()
|
||||
.withObjectClass(<YOUR OBJECT CLASS>)
|
||||
.withConsistencyLevel(<YOUR CONSISTENCY LEVEL>);
|
||||
|
||||
return new WeaviateVectorStore(configBuilder.build(), embeddingModel, weaviateClient);
|
||||
}
|
||||
vectorStore.similaritySearch(
|
||||
SearchRequest.defaults()
|
||||
.withQuery("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression("country in ['UK', 'NL'] && year >= 2020"));
|
||||
----
|
||||
|
||||
== Run Weaviate cluster in docker container
|
||||
or programmatically using the `Filter.Expression` DSL:
|
||||
|
||||
Start Weaviate in a docker container:
|
||||
[source,java]
|
||||
----
|
||||
FilterExpressionBuilder b = new FilterExpressionBuilder();
|
||||
|
||||
vectorStore.similaritySearch(SearchRequest.defaults()
|
||||
.withQuery("The World")
|
||||
.withTopK(TOP_K)
|
||||
.withSimilarityThreshold(SIMILARITY_THRESHOLD)
|
||||
.withFilterExpression(b.and(
|
||||
b.in("country", "UK", "NL"),
|
||||
b.gte("year", 2020)).build()));
|
||||
----
|
||||
|
||||
NOTE: Those (portable) filter expressions get automatically converted into the proprietary Weaviate link:https://weaviate.io/developers/weaviate/api/graphql/filters[where filters].
|
||||
|
||||
For example, this portable filter expression:
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
country in ['UK', 'NL'] && year >= 2020
|
||||
----
|
||||
|
||||
is converted into the proprietary Weaviate GraphQL filter format:
|
||||
|
||||
[source,graphql]
|
||||
----
|
||||
operator: And
|
||||
operands:
|
||||
[{
|
||||
operator: Or
|
||||
operands:
|
||||
[{
|
||||
path: ["meta_country"]
|
||||
operator: Equal
|
||||
valueText: "UK"
|
||||
},
|
||||
{
|
||||
path: ["meta_country"]
|
||||
operator: Equal
|
||||
valueText: "NL"
|
||||
}]
|
||||
},
|
||||
{
|
||||
path: ["meta_year"]
|
||||
operator: GreaterThanEqual
|
||||
valueNumber: 2020
|
||||
}]
|
||||
----
|
||||
|
||||
== Run Weaviate in Docker
|
||||
|
||||
To quickly get started with a local Weaviate instance, you can run it in Docker:
|
||||
|
||||
[source,bash]
|
||||
----
|
||||
docker run -it --rm --name weaviate -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true -e PERSISTENCE_DATA_PATH=/var/lib/weaviate -e QUERY_DEFAULTS_LIMIT=25 -e DEFAULT_VECTORIZER_MODULE=none -e CLUSTER_HOSTNAME=node1 -p 8080:8080 semitechnologies/weaviate:1.22.4
|
||||
docker run -it --rm --name weaviate \
|
||||
-e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
|
||||
-e PERSISTENCE_DATA_PATH=/var/lib/weaviate \
|
||||
-e QUERY_DEFAULTS_LIMIT=25 \
|
||||
-e DEFAULT_VECTORIZER_MODULE=none \
|
||||
-e CLUSTER_HOSTNAME=node1 \
|
||||
-p 8080:8080 \
|
||||
semitechnologies/weaviate:1.22.4
|
||||
----
|
||||
|
||||
Starts a Weaviate cluster at http://localhost:8080/v1 with scheme=http, host=localhost:8080, and apiKey="". Then follow the usage instructions.
|
||||
This starts a Weaviate instance accessible at http://localhost:8080.
|
||||
|
||||
== WeaviateVectorStore properties
|
||||
|
||||
You can use the following properties in your Spring Boot configuration to customize the Weaviate vector store.
|
||||
|
||||
[stripes=even]
|
||||
|===
|
||||
|Property|Description|Default value
|
||||
|
||||
|`spring.ai.vectorstore.weaviate.host`|The host of the Weaviate server|localhost:8080
|
||||
|`spring.ai.vectorstore.weaviate.scheme`|Connection schema|http
|
||||
|`spring.ai.vectorstore.weaviate.api-key`|The API key for authentication|
|
||||
|`spring.ai.vectorstore.weaviate.object-class`|The class name for storing documents|SpringAiWeaviate
|
||||
|`spring.ai.vectorstore.weaviate.consistency-level`|Desired tradeoff between consistency and speed|ConsistentLevel.ONE
|
||||
|`spring.ai.vectorstore.weaviate.filter-field`|Configures metadata fields that can be used in filters. Format: spring.ai.vectorstore.weaviate.filter-field.<field-name>=<field-type>|
|
||||
|===
|
||||
|
||||
@@ -25,9 +25,9 @@ import io.weaviate.client.v1.auth.exception.AuthException;
|
||||
import org.springframework.ai.embedding.BatchingStrategy;
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.embedding.TokenCountBatchingStrategy;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.WeaviateVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationConvention;
|
||||
import org.springframework.beans.factory.ObjectProvider;
|
||||
import org.springframework.boot.autoconfigure.AutoConfiguration;
|
||||
@@ -81,18 +81,20 @@ public class WeaviateVectorStoreAutoConfiguration {
|
||||
ObjectProvider<VectorStoreObservationConvention> customObservationConvention,
|
||||
BatchingStrategy batchingStrategy) {
|
||||
|
||||
WeaviateVectorStoreConfig.Builder configBuilder = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()
|
||||
.withObjectClass(properties.getObjectClass())
|
||||
.withFilterableMetadataFields(properties.getFilterField()
|
||||
return WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.objectClass(properties.getObjectClass())
|
||||
.filterMetadataFields(properties.getFilterField()
|
||||
.entrySet()
|
||||
.stream()
|
||||
.map(e -> new MetadataField(e.getKey(), e.getValue()))
|
||||
.map(e -> new WeaviateVectorStore.MetadataField(e.getKey(), e.getValue()))
|
||||
.toList())
|
||||
.withConsistencyLevel(properties.getConsistencyLevel());
|
||||
|
||||
return new WeaviateVectorStore(configBuilder.build(), embeddingModel, weaviateClient,
|
||||
observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP),
|
||||
customObservationConvention.getIfAvailable(() -> null), batchingStrategy);
|
||||
.consistencyLevel(WeaviateVectorStore.ConsistentLevel.valueOf(properties.getConsistencyLevel().name()))
|
||||
.observationRegistry(observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP))
|
||||
.customObservationConvention(customObservationConvention.getIfAvailable(() -> null))
|
||||
.batchingStrategy(batchingStrategy)
|
||||
.build();
|
||||
}
|
||||
|
||||
static class PropertiesWeaviateConnectionDetails implements WeaviateConnectionDetails {
|
||||
|
||||
@@ -18,9 +18,9 @@ package org.springframework.ai.autoconfigure.vectorstore.weaviate;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.ConsistentLevel;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.ConsistentLevel;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.MetadataField;
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
|
||||
/**
|
||||
@@ -41,7 +41,7 @@ public class WeaviateVectorStoreProperties {
|
||||
|
||||
private String objectClass = "SpringAiWeaviate";
|
||||
|
||||
private ConsistentLevel consistencyLevel = WeaviateVectorStoreConfig.ConsistentLevel.ONE;
|
||||
private ConsistentLevel consistencyLevel = WeaviateVectorStore.ConsistentLevel.ONE;
|
||||
|
||||
/**
|
||||
* spring.ai.vectorstore.weaviate.filter-field.<field-name>=<field-type>
|
||||
|
||||
@@ -32,7 +32,7 @@ import org.springframework.ai.observation.conventions.VectorStoreProvider;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.MetadataField;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationContext;
|
||||
import org.springframework.boot.autoconfigure.AutoConfigurations;
|
||||
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
|
||||
|
||||
@@ -32,7 +32,7 @@ import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
|
||||
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
@@ -52,8 +52,8 @@ import org.springframework.ai.embedding.EmbeddingOptionsBuilder;
|
||||
import org.springframework.ai.embedding.TokenCountBatchingStrategy;
|
||||
import org.springframework.ai.model.EmbeddingUtils;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreProvider;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.ConsistentLevel;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.AbstractVectorStoreBuilder;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.filter.Filter;
|
||||
import org.springframework.ai.vectorstore.observation.AbstractObservationVectorStore;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationContext;
|
||||
@@ -63,18 +63,36 @@ import org.springframework.util.CollectionUtils;
|
||||
import org.springframework.util.StringUtils;
|
||||
|
||||
/**
|
||||
* A VectorStore implementation backed by Weaviate vector database.
|
||||
* A vector store implementation that stores and retrieves vectors in a Weaviate database.
|
||||
*
|
||||
* Note: You can assign arbitrary metadata fields with your Documents. Later will be
|
||||
* persisted and managed as Document fields. But only the metadata keys listed in
|
||||
* {@link WeaviateVectorStore#filterMetadataFields} can be used for similarity search
|
||||
* expression filters.
|
||||
*
|
||||
* <p>
|
||||
* Example usage with builder:
|
||||
* </p>
|
||||
* <pre>{@code
|
||||
* // Create the vector store with builder
|
||||
* WeaviateVectorStore vectorStore = WeaviateVectorStore.builder()
|
||||
* .weaviateClient(weaviateClient) // Required: Configure Weaviate client
|
||||
* .embeddingModel(embeddingModel) // Required: Configure embedding model
|
||||
* .objectClass("CustomClass") // Optional: Custom class name (default: SpringAiWeaviate)
|
||||
* .consistencyLevel(ConsistentLevel.QUORUM) // Optional: Set consistency level (default: ONE)
|
||||
* .filterMetadataFields(List.of( // Optional: Configure filterable metadata fields
|
||||
* MetadataField.text("country"),
|
||||
* MetadataField.number("year")
|
||||
* ))
|
||||
* .build();
|
||||
* }</pre>
|
||||
*
|
||||
* @author Christian Tzolov
|
||||
* @author Eddú Meléndez
|
||||
* @author Josh Long
|
||||
* @author Soby Chacko
|
||||
* @author Thomas Vitale
|
||||
* @since 1.0.0
|
||||
*/
|
||||
public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
|
||||
@@ -92,8 +110,6 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
|
||||
private static final String ADDITIONAL_VECTOR_FIELD_NAME = "vector";
|
||||
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
private final WeaviateClient weaviateClient;
|
||||
|
||||
private final ConsistentLevel consistencyLevel;
|
||||
@@ -131,11 +147,16 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
/**
|
||||
* Constructs a new WeaviateVectorStore.
|
||||
* @param vectorStoreConfig The configuration for the store.
|
||||
* @param embeddingModel The client for embedding operations.
|
||||
* @param weaviateClient The client for Weaviate operations.
|
||||
* Constructs a new WeaviateVectorStore with default settings.
|
||||
* @param vectorStoreConfig The configuration for the store
|
||||
* @param embeddingModel The client for embedding operations
|
||||
* @param weaviateClient The client for Weaviate operations
|
||||
* @deprecated Use {@link #builder()} instead to create instances of
|
||||
* WeaviateVectorStore. This constructor will be removed in a future release.
|
||||
* @see #builder()
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public WeaviateVectorStore(WeaviateVectorStoreConfig vectorStoreConfig, EmbeddingModel embeddingModel,
|
||||
WeaviateClient weaviateClient) {
|
||||
this(vectorStoreConfig, embeddingModel, weaviateClient, ObservationRegistry.NOOP, null,
|
||||
@@ -143,31 +164,61 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Constructs a new WeaviateVectorStore.
|
||||
* @param vectorStoreConfig The configuration for the store.
|
||||
* @param embeddingModel The client for embedding operations.
|
||||
* @param weaviateClient The client for Weaviate operations.
|
||||
* @param observationRegistry The registry for observations.
|
||||
* @param customObservationConvention The custom observation convention.
|
||||
* Constructs a new WeaviateVectorStore with custom settings.
|
||||
* @param vectorStoreConfig The configuration for the store
|
||||
* @param embeddingModel The client for embedding operations
|
||||
* @param weaviateClient The client for Weaviate operations
|
||||
* @param observationRegistry The registry for observations
|
||||
* @param customObservationConvention The custom observation convention
|
||||
* @param batchingStrategy The strategy for batching operations
|
||||
* @deprecated Use {@link #builder()} instead to create instances of
|
||||
* WeaviateVectorStore. This constructor will be removed in a future release.
|
||||
* @see #builder()
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public WeaviateVectorStore(WeaviateVectorStoreConfig vectorStoreConfig, EmbeddingModel embeddingModel,
|
||||
WeaviateClient weaviateClient, ObservationRegistry observationRegistry,
|
||||
VectorStoreObservationConvention customObservationConvention, BatchingStrategy batchingStrategy) {
|
||||
|
||||
super(observationRegistry, customObservationConvention);
|
||||
this(builder().embeddingModel(embeddingModel)
|
||||
.weaviateClient(weaviateClient)
|
||||
.observationRegistry(observationRegistry)
|
||||
.customObservationConvention(customObservationConvention)
|
||||
.batchingStrategy(batchingStrategy));
|
||||
}
|
||||
|
||||
Assert.notNull(vectorStoreConfig, "WeaviateVectorStoreConfig must not be null");
|
||||
Assert.notNull(embeddingModel, "EmbeddingModel must not be null");
|
||||
/**
|
||||
* Protected constructor for creating a WeaviateVectorStore instance using the builder
|
||||
* pattern. This constructor initializes the vector store with the configured settings
|
||||
* from the builder and performs necessary validations.
|
||||
* @param builder the {@link WeaviateBuilder} containing all configuration settings
|
||||
* @throws IllegalArgumentException if the weaviateClient is null
|
||||
* @see WeaviateBuilder
|
||||
* @since 1.0.0
|
||||
*/
|
||||
protected WeaviateVectorStore(WeaviateBuilder builder) {
|
||||
super(builder);
|
||||
|
||||
this.embeddingModel = embeddingModel;
|
||||
this.consistencyLevel = vectorStoreConfig.consistencyLevel;
|
||||
this.weaviateObjectClass = vectorStoreConfig.weaviateObjectClass;
|
||||
this.filterMetadataFields = vectorStoreConfig.filterMetadataFields;
|
||||
Assert.notNull(builder.weaviateClient, "WeaviateClient must not be null");
|
||||
|
||||
this.weaviateClient = builder.weaviateClient;
|
||||
this.consistencyLevel = builder.consistencyLevel;
|
||||
this.weaviateObjectClass = builder.weaviateObjectClass;
|
||||
this.filterMetadataFields = builder.filterMetadataFields;
|
||||
this.batchingStrategy = builder.batchingStrategy;
|
||||
this.filterExpressionConverter = new WeaviateFilterExpressionConverter(
|
||||
this.filterMetadataFields.stream().map(MetadataField::name).toList());
|
||||
this.weaviateClient = weaviateClient;
|
||||
this.weaviateSimilaritySearchFields = buildWeaviateSimilaritySearchFields();
|
||||
this.batchingStrategy = batchingStrategy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a new WeaviateBuilder instance. This is the recommended way to instantiate
|
||||
* a WeaviateVectorStore.
|
||||
* @return a new WeaviateBuilder instance
|
||||
*/
|
||||
public static WeaviateBuilder builder() {
|
||||
return new WeaviateBuilder();
|
||||
}
|
||||
|
||||
private Field[] buildWeaviateSimilaritySearchFields() {
|
||||
@@ -402,8 +453,193 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Configuration class for the WeaviateVectorStore.
|
||||
* Defines the consistency levels for Weaviate operations.
|
||||
*
|
||||
* @see <a href=
|
||||
* "https://weaviate.io/developers/weaviate/concepts/replication-architecture/consistency#tunable-consistency-strategies">Weaviate
|
||||
* Consistency Strategies</a>
|
||||
*/
|
||||
public enum ConsistentLevel {
|
||||
|
||||
/**
|
||||
* Write must receive an acknowledgement from at least one replica node. This is
|
||||
* the fastest (most available), but least consistent option.
|
||||
*/
|
||||
ONE,
|
||||
|
||||
/**
|
||||
* Write must receive an acknowledgement from at least QUORUM replica nodes.
|
||||
* QUORUM is calculated as n / 2 + 1, where n is the number of replicas.
|
||||
*/
|
||||
QUORUM,
|
||||
|
||||
/**
|
||||
* Write must receive an acknowledgement from all replica nodes. This is the most
|
||||
* consistent, but 'slowest'.
|
||||
*/
|
||||
ALL
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents a metadata field configuration for Weaviate vector store.
|
||||
*
|
||||
* @param name the name of the metadata field
|
||||
* @param type the type of the metadata field
|
||||
*/
|
||||
public record MetadataField(String name, Type type) {
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type TEXT.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type TEXT
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
*/
|
||||
public static MetadataField text(String name) {
|
||||
Assert.hasText(name, "Text field must not be empty");
|
||||
return new MetadataField(name, Type.TEXT);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type NUMBER.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type NUMBER
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
*/
|
||||
public static MetadataField number(String name) {
|
||||
Assert.hasText(name, "Number field must not be empty");
|
||||
return new MetadataField(name, Type.NUMBER);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type BOOLEAN.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type BOOLEAN
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
*/
|
||||
public static MetadataField bool(String name) {
|
||||
Assert.hasText(name, "Boolean field name must not be empty");
|
||||
return new MetadataField(name, Type.BOOLEAN);
|
||||
}
|
||||
|
||||
/**
|
||||
* Defines the supported types for metadata fields.
|
||||
*/
|
||||
public enum Type {
|
||||
|
||||
TEXT, NUMBER, BOOLEAN
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
public static final class WeaviateBuilder extends AbstractVectorStoreBuilder<WeaviateBuilder> {
|
||||
|
||||
private String weaviateObjectClass = "SpringAiWeaviate";
|
||||
|
||||
private ConsistentLevel consistencyLevel = ConsistentLevel.ONE;
|
||||
|
||||
private List<MetadataField> filterMetadataFields = List.of();
|
||||
|
||||
private WeaviateClient weaviateClient;
|
||||
|
||||
private BatchingStrategy batchingStrategy = new TokenCountBatchingStrategy();
|
||||
|
||||
/**
|
||||
* Configures the Weaviate client.
|
||||
* @param weaviateClient the client for Weaviate operations
|
||||
* @return this builder instance
|
||||
* @throws IllegalArgumentException if weaviateClient is null
|
||||
*/
|
||||
public WeaviateBuilder weaviateClient(WeaviateClient weaviateClient) {
|
||||
Assert.notNull(weaviateClient, "weaviateClient must not be null");
|
||||
this.weaviateClient = weaviateClient;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Configures the Weaviate object class.
|
||||
* @param objectClass the object class to use
|
||||
* @return this builder instance
|
||||
* @throws IllegalArgumentException if objectClass is null or empty
|
||||
*/
|
||||
public WeaviateBuilder objectClass(String objectClass) {
|
||||
Assert.hasText(objectClass, "objectClass must not be empty");
|
||||
this.weaviateObjectClass = objectClass;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Configures the consistency level for Weaviate operations.
|
||||
* @param consistencyLevel the consistency level to use
|
||||
* @return this builder instance
|
||||
* @throws IllegalArgumentException if consistencyLevel is null
|
||||
*/
|
||||
public WeaviateBuilder consistencyLevel(ConsistentLevel consistencyLevel) {
|
||||
Assert.notNull(consistencyLevel, "consistencyLevel must not be null");
|
||||
this.consistencyLevel = consistencyLevel;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Configures the filterable metadata fields.
|
||||
* @param filterMetadataFields list of metadata fields that can be used in filters
|
||||
* @return this builder instance
|
||||
* @throws IllegalArgumentException if filterMetadataFields is null
|
||||
*/
|
||||
public WeaviateBuilder filterMetadataFields(List<MetadataField> filterMetadataFields) {
|
||||
Assert.notNull(filterMetadataFields, "filterMetadataFields must not be null");
|
||||
this.filterMetadataFields = filterMetadataFields;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Configures the batching strategy.
|
||||
* @param batchingStrategy the strategy for batching operations
|
||||
* @return this builder instance
|
||||
* @throws IllegalArgumentException if batchingStrategy is null
|
||||
*/
|
||||
public WeaviateBuilder batchingStrategy(BatchingStrategy batchingStrategy) {
|
||||
Assert.notNull(batchingStrategy, "batchingStrategy must not be null");
|
||||
this.batchingStrategy = batchingStrategy;
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds and returns a new WeaviateVectorStore instance with the configured
|
||||
* settings.
|
||||
* @return a new WeaviateVectorStore instance
|
||||
* @throws IllegalStateException if the builder configuration is invalid
|
||||
*/
|
||||
@Override
|
||||
public WeaviateVectorStore build() {
|
||||
validate();
|
||||
return new WeaviateVectorStore(this);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Configuration class for WeaviateVectorStore.
|
||||
*
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead to configure and
|
||||
* create instances of WeaviateVectorStore. This class will be removed in a future
|
||||
* release. Example migration: <pre>{@code
|
||||
* // Old approach:
|
||||
* WeaviateVectorStoreConfig config = WeaviateVectorStoreConfig.builder()
|
||||
* .withObjectClass("CustomClass")
|
||||
* .withConsistencyLevel(ConsistentLevel.QUORUM)
|
||||
* .build();
|
||||
*
|
||||
* // New approach:
|
||||
* WeaviateVectorStore store = WeaviateVectorStore.builder()
|
||||
* .objectClass("CustomClass")
|
||||
* .consistencyLevel(ConsistentLevel.QUORUM)
|
||||
* .build();
|
||||
* }</pre>
|
||||
* @see WeaviateVectorStore#builder()
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static final class WeaviateVectorStoreConfig {
|
||||
|
||||
private final String weaviateObjectClass;
|
||||
@@ -421,8 +657,10 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
|
||||
/**
|
||||
* Constructor using the builder.
|
||||
* @param builder The configuration builder.
|
||||
* @param builder The configuration builder
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public WeaviateVectorStoreConfig(Builder builder) {
|
||||
this.weaviateObjectClass = builder.objectClass;
|
||||
this.consistencyLevel = builder.consistencyLevel;
|
||||
@@ -432,22 +670,43 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
|
||||
/**
|
||||
* Start building a new configuration.
|
||||
* @return The entry point for creating a new configuration.
|
||||
* @return The entry point for creating a new configuration
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead to configure and
|
||||
* create instances of WeaviateVectorStore
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static Builder builder() {
|
||||
return new Builder();
|
||||
}
|
||||
|
||||
/**
|
||||
* {@return the default config}
|
||||
* Returns the default configuration.
|
||||
* @return the default configuration
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead to configure and
|
||||
* create instances of WeaviateVectorStore with default settings
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static WeaviateVectorStoreConfig defaultConfig() {
|
||||
return builder().build();
|
||||
}
|
||||
|
||||
/**
|
||||
* https://weaviate.io/developers/weaviate/concepts/replication-architecture/consistency#tunable-consistency-strategies
|
||||
* Defines the consistency levels for Weaviate operations.
|
||||
*
|
||||
* @see <a href=
|
||||
* "https://weaviate.io/developers/weaviate/concepts/replication-architecture/consistency#tunable-consistency-strategies">Weaviate
|
||||
* Consistency Strategies</a>
|
||||
* @deprecated Use {@link WeaviateVectorStore.ConsistentLevel} instead. This enum
|
||||
* will be removed in a future release. Example migration: <pre>{@code
|
||||
* // Old approach:
|
||||
* WeaviateVectorStoreConfig.ConsistentLevel level = WeaviateVectorStoreConfig.ConsistentLevel.QUORUM;
|
||||
*
|
||||
* // New approach:
|
||||
* WeaviateVectorStore.ConsistentLevel level = WeaviateVectorStore.ConsistentLevel.QUORUM;
|
||||
* }</pre>
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public enum ConsistentLevel {
|
||||
|
||||
/**
|
||||
@@ -470,33 +729,91 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents a metadata field configuration for Weaviate vector store.
|
||||
*
|
||||
* @param name the name of the metadata field
|
||||
* @param type the type of the metadata field
|
||||
* @deprecated Use {@link WeaviateVectorStore.MetadataField} instead. This record
|
||||
* will be removed in a future release. Example migration: <pre>{@code
|
||||
* // Old approach:
|
||||
* WeaviateVectorStoreConfig.MetadataField field = WeaviateVectorStoreConfig.MetadataField.text("field");
|
||||
*
|
||||
* // New approach:
|
||||
* WeaviateVectorStore.MetadataField field = WeaviateVectorStore.MetadataField.text("field");
|
||||
* }</pre>
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public record MetadataField(String name, Type type) {
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type TEXT.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type TEXT
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
* @deprecated Use {@link WeaviateVectorStore.MetadataField#text(String)}
|
||||
* instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static MetadataField text(String name) {
|
||||
return new MetadataField(name, Type.TEXT);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type NUMBER.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type NUMBER
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
* @deprecated Use {@link WeaviateVectorStore.MetadataField#number(String)}
|
||||
* instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static MetadataField number(String name) {
|
||||
return new MetadataField(name, Type.NUMBER);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a metadata field of type BOOLEAN.
|
||||
* @param name the name of the field
|
||||
* @return a new MetadataField instance of type BOOLEAN
|
||||
* @throws IllegalArgumentException if name is null or empty
|
||||
* @deprecated Use {@link WeaviateVectorStore.MetadataField#bool(String)}
|
||||
* instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static MetadataField bool(String name) {
|
||||
return new MetadataField(name, Type.BOOLEAN);
|
||||
}
|
||||
|
||||
/**
|
||||
* Defines the supported types for metadata fields.
|
||||
*
|
||||
* @deprecated Use {@link WeaviateVectorStore.MetadataField.Type} instead.
|
||||
* This enum will be removed in a future release.
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public enum Type {
|
||||
|
||||
TEXT, NUMBER, BOOLEAN
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* Builder for WeaviateVectorStoreConfig.
|
||||
*
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead to configure and
|
||||
* create instances of WeaviateVectorStore
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public static final class Builder {
|
||||
|
||||
private String objectClass = "SpringAiWeaviate";
|
||||
|
||||
private ConsistentLevel consistencyLevel = WeaviateVectorStoreConfig.ConsistentLevel.ONE;
|
||||
private ConsistentLevel consistencyLevel = ConsistentLevel.ONE;
|
||||
|
||||
private List<MetadataField> filterMetadataFields = List.of();
|
||||
|
||||
@@ -506,10 +823,15 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Weaviate known, filterable metadata fields.
|
||||
* @param filterMetadataFields known metadata fields to use.
|
||||
* @return this builder.
|
||||
* Configures the filterable metadata fields.
|
||||
* @param filterMetadataFields known metadata fields to use
|
||||
* @return this builder
|
||||
* @throws IllegalArgumentException if filterMetadataFields is null
|
||||
* @deprecated Use
|
||||
* {@link WeaviateVectorStore.WeaviateBuilder#filterMetadataFields(List)}
|
||||
* instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public Builder withFilterableMetadataFields(List<MetadataField> filterMetadataFields) {
|
||||
Assert.notNull(filterMetadataFields, "The filterMetadataFields can not be null.");
|
||||
this.filterMetadataFields = filterMetadataFields;
|
||||
@@ -517,10 +839,14 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Weaviate config headers.
|
||||
* @param headers config headers to use.
|
||||
* @return this builder.
|
||||
* Configures the Weaviate config headers.
|
||||
* @param headers config headers to use
|
||||
* @return this builder
|
||||
* @throws IllegalArgumentException if headers is null
|
||||
* @deprecated Use the new builder API in
|
||||
* {@link WeaviateVectorStore#builder()}
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public Builder withHeaders(Map<String, String> headers) {
|
||||
Assert.notNull(headers, "The headers can not be null.");
|
||||
this.headers = headers;
|
||||
@@ -528,10 +854,14 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Weaviate objectClass.
|
||||
* @param objectClass objectClass to use.
|
||||
* @return this builder.
|
||||
* Configures the Weaviate objectClass.
|
||||
* @param objectClass objectClass to use
|
||||
* @return this builder
|
||||
* @throws IllegalArgumentException if objectClass is empty or null
|
||||
* @deprecated Use
|
||||
* {@link WeaviateVectorStore.WeaviateBuilder#objectClass(String)} instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public Builder withObjectClass(String objectClass) {
|
||||
Assert.hasText(objectClass, "The objectClass can not be empty.");
|
||||
this.objectClass = objectClass;
|
||||
@@ -539,10 +869,15 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Weaviate consistencyLevel.
|
||||
* @param consistencyLevel consistencyLevel to use.
|
||||
* @return this builder.
|
||||
* Configures the Weaviate consistencyLevel.
|
||||
* @param consistencyLevel consistencyLevel to use
|
||||
* @return this builder
|
||||
* @throws IllegalArgumentException if consistencyLevel is null
|
||||
* @deprecated Use
|
||||
* {@link WeaviateVectorStore.WeaviateBuilder#consistencyLevel(WeaviateVectorStore.ConsistentLevel)}
|
||||
* instead
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public Builder withConsistencyLevel(ConsistentLevel consistencyLevel) {
|
||||
Assert.notNull(consistencyLevel, "The consistencyLevel can not be null.");
|
||||
this.consistencyLevel = consistencyLevel;
|
||||
@@ -550,8 +885,12 @@ public class WeaviateVectorStore extends AbstractObservationVectorStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* {@return the immutable configuration}
|
||||
* Builds and returns the immutable configuration.
|
||||
* @return the immutable configuration
|
||||
* @deprecated Use {@link WeaviateVectorStore#builder()} instead to configure
|
||||
* and create instances of WeaviateVectorStore
|
||||
*/
|
||||
@Deprecated(forRemoval = true, since = "1.0.0-M5")
|
||||
public WeaviateVectorStoreConfig build() {
|
||||
return new WeaviateVectorStoreConfig(this);
|
||||
}
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import org.testcontainers.utility.DockerImageName;
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
/*
|
||||
* Copyright 2023-2024 the original author or authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import io.weaviate.client.Config;
|
||||
import io.weaviate.client.WeaviateClient;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.extension.ExtendWith;
|
||||
import org.mockito.Mock;
|
||||
import org.mockito.junit.jupiter.MockitoExtension;
|
||||
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.ConsistentLevel;
|
||||
import org.springframework.ai.vectorstore.weaviate.WeaviateVectorStore.MetadataField;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/**
|
||||
* Tests for {@link WeaviateVectorStore.WeaviateBuilder}.
|
||||
*
|
||||
* @author Mark Pollack
|
||||
*/
|
||||
@ExtendWith(MockitoExtension.class)
|
||||
class WeaviateVectorStoreBuilderTests {
|
||||
|
||||
@Mock
|
||||
private EmbeddingModel embeddingModel;
|
||||
|
||||
@Test
|
||||
void shouldBuildWithMinimalConfiguration() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
WeaviateVectorStore vectorStore = WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.build();
|
||||
|
||||
assertThat(vectorStore).isNotNull();
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldBuildWithCustomConfiguration() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
WeaviateVectorStore vectorStore = WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.objectClass("CustomClass")
|
||||
.consistencyLevel(ConsistentLevel.QUORUM)
|
||||
.filterMetadataFields(List.of(MetadataField.text("country"), MetadataField.number("year")))
|
||||
.build();
|
||||
|
||||
assertThat(vectorStore).isNotNull();
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldFailWithoutWeaviateClient() {
|
||||
assertThatThrownBy(() -> WeaviateVectorStore.builder().embeddingModel(embeddingModel).build())
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessage("WeaviateClient must not be null");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldFailWithoutEmbeddingModel() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
assertThatThrownBy(() -> WeaviateVectorStore.builder().weaviateClient(weaviateClient).build())
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessage("EmbeddingModel must be configured");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldFailWithInvalidObjectClass() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
assertThatThrownBy(() -> WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.objectClass("")
|
||||
.build()).isInstanceOf(IllegalArgumentException.class).hasMessage("objectClass must not be empty");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldFailWithNullConsistencyLevel() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
assertThatThrownBy(() -> WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.consistencyLevel(null)
|
||||
.build()).isInstanceOf(IllegalArgumentException.class).hasMessage("consistencyLevel must not be null");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldFailWithNullFilterMetadataFields() {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(new Config("http", "localhost:8080"));
|
||||
|
||||
assertThatThrownBy(() -> WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.filterMetadataFields(null)
|
||||
.build()).isInstanceOf(IllegalArgumentException.class).hasMessage("filterMetadataFields must not be null");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldCreateMetadataFieldsWithValidation() {
|
||||
assertThatThrownBy(() -> MetadataField.text("")).isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessage("Text field must not be empty");
|
||||
|
||||
assertThatThrownBy(() -> MetadataField.number("")).isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessage("Number field must not be empty");
|
||||
|
||||
assertThatThrownBy(() -> MetadataField.bool("")).isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessage("Boolean field name must not be empty");
|
||||
|
||||
MetadataField textField = MetadataField.text("validName");
|
||||
assertThat(textField.name()).isEqualTo("validName");
|
||||
assertThat(textField.type()).isEqualTo(MetadataField.Type.TEXT);
|
||||
|
||||
MetadataField numberField = MetadataField.number("validName");
|
||||
assertThat(numberField.name()).isEqualTo("validName");
|
||||
assertThat(numberField.type()).isEqualTo(MetadataField.Type.NUMBER);
|
||||
|
||||
MetadataField boolField = MetadataField.bool("validName");
|
||||
assertThat(boolField.name()).isEqualTo("validName");
|
||||
assertThat(boolField.type()).isEqualTo(MetadataField.Type.BOOLEAN);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
@@ -35,8 +35,8 @@ import org.testcontainers.weaviate.WeaviateContainer;
|
||||
import org.springframework.ai.document.Document;
|
||||
import org.springframework.ai.embedding.EmbeddingModel;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig.MetadataField;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.boot.SpringBootConfiguration;
|
||||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
|
||||
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
|
||||
@@ -252,13 +252,13 @@ public class WeaviateVectorStoreIT {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(
|
||||
new Config("http", weaviateContainer.getHttpHostAddress()));
|
||||
|
||||
WeaviateVectorStoreConfig config = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()
|
||||
.withFilterableMetadataFields(List.of(MetadataField.text("country"), MetadataField.number("year")))
|
||||
.withConsistencyLevel(WeaviateVectorStoreConfig.ConsistentLevel.ONE)
|
||||
return WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.filterMetadataFields(List.of(WeaviateVectorStore.MetadataField.text("country"),
|
||||
WeaviateVectorStore.MetadataField.number("year")))
|
||||
.consistencyLevel(WeaviateVectorStore.ConsistentLevel.ONE)
|
||||
.build();
|
||||
|
||||
return new WeaviateVectorStore(config, embeddingModel, weaviateClient);
|
||||
|
||||
}
|
||||
|
||||
@Bean
|
||||
@@ -14,7 +14,7 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package org.springframework.ai.vectorstore;
|
||||
package org.springframework.ai.vectorstore.weaviate;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
@@ -37,7 +37,8 @@ import org.springframework.ai.embedding.TokenCountBatchingStrategy;
|
||||
import org.springframework.ai.observation.conventions.SpringAiKind;
|
||||
import org.springframework.ai.observation.conventions.VectorStoreProvider;
|
||||
import org.springframework.ai.transformers.TransformersEmbeddingModel;
|
||||
import org.springframework.ai.vectorstore.WeaviateVectorStore.WeaviateVectorStoreConfig;
|
||||
import org.springframework.ai.vectorstore.SearchRequest;
|
||||
import org.springframework.ai.vectorstore.VectorStore;
|
||||
import org.springframework.ai.vectorstore.observation.DefaultVectorStoreObservationConvention;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationDocumentation.HighCardinalityKeyNames;
|
||||
import org.springframework.ai.vectorstore.observation.VectorStoreObservationDocumentation.LowCardinalityKeyNames;
|
||||
@@ -165,13 +166,13 @@ public class WeaviateVectorStoreObservationIT {
|
||||
WeaviateClient weaviateClient = new WeaviateClient(
|
||||
new io.weaviate.client.Config("http", weaviateContainer.getHttpHostAddress()));
|
||||
|
||||
WeaviateVectorStoreConfig config = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()
|
||||
.withConsistencyLevel(WeaviateVectorStoreConfig.ConsistentLevel.ONE)
|
||||
return WeaviateVectorStore.builder()
|
||||
.weaviateClient(weaviateClient)
|
||||
.embeddingModel(embeddingModel)
|
||||
.consistencyLevel(WeaviateVectorStore.ConsistentLevel.ONE)
|
||||
.observationRegistry(observationRegistry)
|
||||
.batchingStrategy(new TokenCountBatchingStrategy())
|
||||
.build();
|
||||
|
||||
return new WeaviateVectorStore(config, embeddingModel, weaviateClient, observationRegistry, null,
|
||||
new TokenCountBatchingStrategy());
|
||||
|
||||
}
|
||||
|
||||
@Bean
|
||||
Reference in New Issue
Block a user