Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

BREAKING CHANGE - Change vector store initialize-schema to false

Browse Source 
* Change default schema initialization of vector stores from `true` to `false.`
  Users need to explicitly opt-in for schema initialization by setting the
  `initialize-schema` property on the corresponding vector store.
* Update integration tests
* Update docs

Fixes #907
This commit is contained in:
Soby Chacko
2024-06-21 12:37:36 -04:00
committed by Mark Pollack
parent edf943ec97
commit 50d34b8a48
57 changed files with 329 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/apache-cassandra.adoc
View File

@@ -78,6 +78,24 @@ Add these dependencies to your project:
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
== Configuration Properties
You can use the following properties in your Spring Boot configuration to customize the Apache Cassandra vector store.
|===
|Property|Default value
|`spring.ai.vectorstore.cassandra.keyspace`|springframework
|`spring.ai.vectorstore.cassandra.table`|ai_vector_store
|`spring.ai.vectorstore.cassandra.initialze-schema`|false
|`spring.ai.vectorstore.cassandra.index-name`|
|`spring.ai.vectorstore.cassandra.content-column-name`|content
|`spring.ai.vectorstore.cassandra.embedding-column-name`|embedding
|`spring.ai.vectorstore.cassandra.return-embeddings`|false
|`spring.ai.vectorstore.cassandra.fixed-thread-pool-executor-size`|16
|===
== Usage

20
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/azure.adoc
View File

@@ -12,7 +12,7 @@ link:https://azure.microsoft.com/en-us/products/ai-services/ai-search/[Azure AI
== Configuration
On startup, the `AzureVectorStore` can attempt to create a new index within your AI Search service instance if you've opted in by setting the relevant `initializeSchema` `boolean` property to `true` in the constructor or, if using Spring Boot, setting `...initialize-schema=true` in your `application.properties` file.
On startup, the `AzureVectorStore` can attempt to create a new index within your AI Search service instance if you've opted in by setting the relevant `initialize-schema` `boolean` property to `true` in the constructor or, if using Spring Boot, setting `...initialize-schema=true` in your `application.properties` file.
NOTE: this is a breaking change! In earlier versions of Spring AI, this schema initialization happened by default.
@@ -88,6 +88,24 @@ Add these dependencies to your project:
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
== Configuration Properties
You can use the following properties in your Spring Boot configuration to customize the Azure vector store.
|===
|Property|Default value
|`spring.ai.vectorstore.azure.url`|
|`spring.ai.vectorstore.azure.api-key`|
|`spring.ai.vectorstore.azure.initialze-schema`|false
|`spring.ai.vectorstore.azure.index-name`|spring_ai_azure_vector_store
|`spring.ai.vectorstore.azure.default-top-k`|4
|`spring.ai.vectorstore.azure.default-similarity-threshold`|0.0
|`spring.ai.vectorstore.azure.embedding-property`|embedding
|`spring.ai.vectorstore.azure.index-name`|spring-ai-document-index
|===
== Sample Code
To configure an Azure `SearchIndexClient` in your application, you can use the following code:

3
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/chroma.adoc
View File

@@ -66,7 +66,6 @@ A simple configuration can either be provided via Spring Boot's _application.pro
[source,properties]
----
# Chroma Vector Store connection properties
spring.ai.vectorstore.chroma.client.initialize-schema=<true or false>
spring.ai.vectorstore.chroma.client.host=<your Chroma instance host>
spring.ai.vectorstore.chroma.client.port=<your Chroma instance port>
spring.ai.vectorstore.chroma.client.key-token=<your access token (if configure)>
@@ -74,6 +73,7 @@ spring.ai.vectorstore.chroma.client.username=<your username (if configure)>
spring.ai.vectorstore.chroma.client.password=<your password (if configure)>
# Chroma Vector Store collection properties
spring.ai.vectorstore.chroma.initialize-schema=<true or false>
spring.ai.vectorstore.chroma.collection-name=<your collection name>
# Chroma Vector Store configuration properties
@@ -117,6 +117,7 @@ You can use the following properties in your Spring Boot configuration to custom
|`spring.ai.vectorstore.chroma.client.username`| Access username (if configured) | -
|`spring.ai.vectorstore.chroma.client.password`| Access password (if configured) | -
|`spring.ai.vectorstore.chroma.collection-name`| Collection name | `SpringAiCollection`
|`spring.ai.vectorstore.chroma.initialize-schema`| Whether to initialize the required schema | `false`
|===
[NOTE]

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/elasticsearch.adoc
View File

@@ -140,6 +140,7 @@ Properties starting with the `spring.ai.vectorstore.elasticsearch.*` prefix are
|===
|Property | Description | Default Value
|`spring.ai.vectorstore.elasticsearch.initialize-schema`| Whether to initialize the required schema | `false`
|`spring.ai.vectorstore.elasticsearch.index-name` | The name of the index to store the vectors. | spring-ai-document-index
|`spring.ai.vectorstore.elasticsearch.dimensions` | The number of dimensions in the vector. | 1536
|`spring.ai.vectorstore.elasticsearch.similarity` | The similarity function to use. | `cosine`

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/gemfire.adoc
View File

@@ -44,6 +44,7 @@ You can use the following properties in your Spring Boot configuration to furthe
|`spring.ai.vectorstore.gemfire.host`|localhost
|`spring.ai.vectorstore.gemfire.port`|8080
|`spring.ai.vectorstore.gemfire.initialize-schema`| `false`
|`spring.ai.vectorstore.gemfire.index-name`|spring-ai-gemfire-store
|`spring.ai.vectorstore.gemfire.beam-width`|100
|`spring.ai.vectorstore.gemfire.max-connections`|16

10
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/opensearch.adoc
View File

@@ -120,11 +120,11 @@ You can use the following properties in your Spring Boot configuration to custom
|===
|Property| Description | Default value
|`spring.opensearch.uris`| URIs of the OpenSearch cluster endpoints. | -
|`spring.opensearch.username`| Username for accessing the OpenSearch cluster. | -
|`spring.opensearch.password`| Password for the specified username. | -
|`spring.opensearch.indexName`| Name of the default index to be used within the OpenSearch cluster. | `spring-ai-document-index`
|`spring.opensearch.mappingJson`| JSON string defining the mapping for the index; specifies how documents and their
|`spring.ai.vectorstore.opensearch.uris`| URIs of the OpenSearch cluster endpoints. | -
|`spring.ai.vectorstore.opensearch.username`| Username for accessing the OpenSearch cluster. | -
|`spring.ai.vectorstore.opensearch.password`| Password for the specified username. | -
|`spring.ai.vectorstore.opensearch.indexName`| Name of the default index to be used within the OpenSearch cluster. | `spring-ai-document-index`
|`spring.ai.vectorstore.opensearch.mappingJson`| JSON string defining the mapping for the index; specifies how documents and their
fields are stored and indexed. |
{
"properties":{

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/redis.adoc
View File

@@ -108,7 +108,7 @@ You can use the following properties in your Spring Boot configuration to custom
|`spring.ai.vectorstore.redis.uri`| Server connection URI | `redis://localhost:6379`
|`spring.ai.vectorstore.redis.index`| Index name | `default-index`
|`spring.ai.vectorstore.redis.initialize-schema`| whether to initialize the required schema | `false`
|`spring.ai.vectorstore.redis.initialize-schema`| Whether to initialize the required schema | `false`
|`spring.ai.vectorstore.redis.prefix`| Prefix | `default:`
|===

5
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/typesense.adoc
View File

@@ -103,8 +103,9 @@ You can use the following properties in your Spring Boot configuration to custom
|`spring.ai.vectorstore.typesense.client.host`| Hostname | `localhost`
|`spring.ai.vectorstore.typesense.client.port`| Port | `8108`
|`spring.ai.vectorstore.typesense.client.apiKey`| ApiKey | `xyz`
|`spring.ai.vectorstore.typesense.collectionName`| Collection Name | `vector_store`
|`spring.ai.vectorstore.typesense.embeddingDimension`| Embedding Dimension | `1536`
|`spring.ai.vectorstore.typesense.initialize-schema`| Whether to initialize the required schema | `false`
|`spring.ai.vectorstore.typesense.collection-name`| Collection Name | `vector_store`
|`spring.ai.vectorstore.typesense.embedding-dimension`| Embedding Dimension | `1536`
|===

10
spring-ai-docs/src/main/antora/modules/ROOT/pages/upgrade-notes.adoc
View File

@@ -5,6 +5,16 @@
* The configuration prefix for the Chroma Vector Store has been changes from `spring.ai.vectorstore.chroma.store` to `spring.ai.vectorstore.chroma` in order to align with the naming conventions of other vector stores.
* The default value of the `initialize-schema` property on vector stores capable of initializing a schema is now set to `false`.
This implies that the applications now need to explicitly opt-in for schema initialization on supported vector stores, if the schema is expected to be created at application startup.
Not all vector stores support this property.
See the corresponding vector store documentation for more details.
The following are the vector stores that currently don't support the `initialize-schema` property.
1. Hana
2. Pinecone
3. Weaviate
== Upgrading to 1.0.0.M1
On our march to release 1.0.0 M1 we have made several breaking changes. Apologies, it is for the best!

24
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/CommonVectorStoreProperties.java
View File

@@ -1,11 +1,33 @@
/*
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.autoconfigure.vectorstore;
/**
* @author Josh Long
* @author Soby Chacko
*/
public class CommonVectorStoreProperties {
private boolean initializeSchema = true;
/**
* Vector stores do not initialize schema by default on application startup. The
* applications explicitly need to opt-in for initializing the schema on startup. The
* recommended way to initialize the schema on startup is to set the initialize-schema
* property on the vector store. See {@link #setInitializeSchema(boolean)}.
*/
private boolean initializeSchema = false;
public boolean isInitializeSchema() {
return initializeSchema;

2
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/gemfire/GemFireVectorStoreAutoConfiguration.java
View File

@@ -57,7 +57,7 @@ public class GemFireVectorStoreAutoConfiguration {
.setVectorSimilarityFunction(properties.getVectorSimilarityFunction())
.setFields(properties.getFields())
.setSslEnabled(properties.isSslEnabled());
return new GemFireVectorStore(config, embeddingModel);
return new GemFireVectorStore(config, embeddingModel, properties.isInitializeSchema());
}
private static class PropertiesGemFireConnectionDetails implements GemFireConnectionDetails {

3
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/gemfire/GemFireVectorStoreProperties.java
View File

@@ -16,6 +16,7 @@
package org.springframework.ai.autoconfigure.vectorstore.gemfire;
import org.springframework.ai.autoconfigure.vectorstore.CommonVectorStoreProperties;
import org.springframework.ai.vectorstore.GemFireVectorStoreConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;
@@ -23,7 +24,7 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
* @author Geet Rawat
*/
@ConfigurationProperties(GemFireVectorStoreProperties.CONFIG_PREFIX)
public class GemFireVectorStoreProperties {
public class GemFireVectorStoreProperties extends CommonVectorStoreProperties {
/**
* Configuration prefix for Spring AI VectorStore GemFire.

3
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/opensearch/OpenSearchVectorStoreAutoConfiguration.java
View File

@@ -61,7 +61,8 @@ public class OpenSearchVectorStoreAutoConfiguration {
var indexName = Optional.ofNullable(properties.getIndexName()).orElse(OpenSearchVectorStore.DEFAULT_INDEX_NAME);
var mappingJson = Optional.ofNullable(properties.getMappingJson())
.orElse(OpenSearchVectorStore.DEFAULT_MAPPING_EMBEDDING_TYPE_KNN_VECTOR_DIMENSION_1536);
return new OpenSearchVectorStore(indexName, openSearchClient, embeddingModel, mappingJson);
return new OpenSearchVectorStore(indexName, openSearchClient, embeddingModel, mappingJson,
properties.isInitializeSchema());
}
@Configuration(proxyBeanMethods = false)

3
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/opensearch/OpenSearchVectorStoreProperties.java
View File

@@ -15,12 +15,13 @@
*/
package org.springframework.ai.autoconfigure.vectorstore.opensearch;
import org.springframework.ai.autoconfigure.vectorstore.CommonVectorStoreProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.List;
@ConfigurationProperties(prefix = OpenSearchVectorStoreProperties.CONFIG_PREFIX)
public class OpenSearchVectorStoreProperties {
public class OpenSearchVectorStoreProperties extends CommonVectorStoreProperties {
public static final String CONFIG_PREFIX = "spring.ai.vectorstore.opensearch";

16
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/typesense/TypesenseServiceClientProperties.java
View File

@@ -1,3 +1,19 @@
/*
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.autoconfigure.vectorstore.typesense;
import org.springframework.boot.context.properties.ConfigurationProperties;

2
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/typesense/TypesenseVectorStoreAutoConfiguration.java
View File

@@ -57,7 +57,7 @@ public class TypesenseVectorStoreAutoConfiguration {
.withEmbeddingDimension(properties.getEmbeddingDimension())
.build();
return new TypesenseVectorStore(typesenseClient, embeddingModel, config);
return new TypesenseVectorStore(typesenseClient, embeddingModel, config, properties.isInitializeSchema());
}
@Bean

20
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/typesense/TypesenseVectorStoreProperties.java
View File

@@ -1,13 +1,31 @@
/*
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.autoconfigure.vectorstore.typesense;
import org.springframework.ai.autoconfigure.vectorstore.CommonVectorStoreProperties;
import org.springframework.ai.vectorstore.TypesenseVectorStore;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author Pablo Sanchidrian Herrera
* @author Soby Chacko
*/
@ConfigurationProperties(TypesenseVectorStoreProperties.CONFIG_PREFIX)
public class TypesenseVectorStoreProperties {
public class TypesenseVectorStoreProperties extends CommonVectorStoreProperties {
public static final String CONFIG_PREFIX = "spring.ai.vectorstore.typesense";

6
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/weaviate/WeaviateVectorStoreAutoConfiguration.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -32,6 +32,7 @@ import org.springframework.context.annotation.Bean;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
*/
@AutoConfiguration
@ConditionalOnClass({ EmbeddingModel.class, WeaviateVectorStore.class })
@@ -72,8 +73,7 @@ public class WeaviateVectorStoreAutoConfiguration {
.toList())
.withConsistencyLevel(properties.getConsistencyLevel());
return new WeaviateVectorStore(configBuilder.build(), embeddingModel, weaviateClient,
properties.isInitializeSchema());
return new WeaviateVectorStore(configBuilder.build(), embeddingModel, weaviateClient);
}
static class PropertiesWeaviateConnectionDetails implements WeaviateConnectionDetails {

2
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/vectorstore/weaviate/WeaviateVectorStoreProperties.java
View File

@@ -27,7 +27,7 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
* @author Christian Tzolov
*/
@ConfigurationProperties(WeaviateVectorStoreProperties.CONFIG_PREFIX)
public class WeaviateVectorStoreProperties extends CommonVectorStoreProperties {
public class WeaviateVectorStoreProperties {
public static final String CONFIG_PREFIX = "spring.ai.vectorstore.weaviate";

10
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/azure/AzureVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -44,6 +44,7 @@ import static org.hamcrest.Matchers.hasSize;
/**
* @author Christian Tzolov
* @author Soby Chacko
*/
@EnabledIfEnvironmentVariable(named = "AZURE_AI_SEARCH_API_KEY", matches = ".+")
@EnabledIfEnvironmentVariable(named = "AZURE_AI_SEARCH_ENDPOINT", matches = ".+")
@@ -68,7 +69,8 @@ public class AzureVectorStoreAutoConfigurationIT {
.withConfiguration(AutoConfigurations.of(AzureVectorStoreAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.azure.apiKey=" + System.getenv("AZURE_AI_SEARCH_API_KEY"),
"spring.ai.vectorstore.azure.url=" + System.getenv("AZURE_AI_SEARCH_ENDPOINT"));
"spring.ai.vectorstore.azure.url=" + System.getenv("AZURE_AI_SEARCH_ENDPOINT"))
.withPropertyValues("spring.ai.vectorstore.azure.initialize-schema=true");
@BeforeAll
public static void beforeAll() {
@@ -81,8 +83,8 @@ public class AzureVectorStoreAutoConfigurationIT {
public void addAndSearchTest() {
contextRunner
.withPropertyValues("spring.ai.vectorstore.azure.indexName=my_test_index",
"spring.ai.vectorstore.azure.defaultTopK=6",
.withPropertyValues("spring.ai.vectorstore.azure.initializeSchema=true",
"spring.ai.vectorstore.azure.indexName=my_test_index", "spring.ai.vectorstore.azure.defaultTopK=6",
"spring.ai.vectorstore.azure.defaultSimilarityThreshold=0.75")
.run(context -> {

1
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/cassandra/CassandraVectorStoreAutoConfigurationIT.java
View File

@@ -59,6 +59,7 @@ class CassandraVectorStoreAutoConfigurationIT {
.withConfiguration(
AutoConfigurations.of(CassandraVectorStoreAutoConfiguration.class, CassandraAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.cassandra.initialize-schema=true")
.withPropertyValues("spring.ai.vectorstore.cassandra.keyspace=test_autoconfigure")
.withPropertyValues("spring.ai.vectorstore.cassandra.contentColumnName=doc_chunk");

2
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/cassandra/CassandraVectorStorePropertiesTests.java
View File

@@ -35,7 +35,7 @@ class CassandraVectorStorePropertiesTests {
assertThat(props.getContentColumnName()).isEqualTo(CassandraVectorStoreConfig.DEFAULT_CONTENT_COLUMN_NAME);
assertThat(props.getEmbeddingColumnName()).isEqualTo(CassandraVectorStoreConfig.DEFAULT_EMBEDDING_COLUMN_NAME);
assertThat(props.getIndexName()).isNull();
assertThat(props.getDisallowSchemaCreation()).isFalse();
assertThat(props.getDisallowSchemaCreation()).isTrue();
assertThat(props.getFixedThreadPoolExecutorSize())
.isEqualTo(CassandraVectorStoreConfig.DEFAULT_ADD_CONCURRENCY);
}

2
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/chroma/ChromaVectorStoreAutoConfigurationIT.java
View File

@@ -38,6 +38,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class ChromaVectorStoreAutoConfigurationIT {
@@ -50,6 +51,7 @@ public class ChromaVectorStoreAutoConfigurationIT {
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.chroma.client.host=http://" + chroma.getHost(),
"spring.ai.vectorstore.chroma.client.port=" + chroma.getMappedPort(8000),
"spring.ai.vectorstore.chroma.initializeSchema=true",
"spring.ai.vectorstore.chroma.collectionName=TestCollection");
@Test

1
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/elasticsearch/ElasticsearchVectorStoreAutoConfigurationIT.java
View File

@@ -60,6 +60,7 @@ class ElasticsearchVectorStoreAutoConfigurationIT {
ElasticsearchVectorStoreAutoConfiguration.class, RestClientAutoConfiguration.class,
SpringAiRetryAutoConfiguration.class, OpenAiAutoConfiguration.class))
.withPropertyValues("spring.elasticsearch.uris=" + elasticsearchContainer.getHttpHostAddress(),
"spring.ai.vectorstore.elasticsearch.initializeSchema=true",
"spring.ai.openai.api-key=" + System.getenv("OPENAI_API_KEY"));
// No parametrized test based on similarity function,

3
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/gemfire/GemFireVectorStoreAutoConfigurationIT.java
View File

@@ -92,7 +92,8 @@ class GemFireVectorStoreAutoConfigurationIT {
.withPropertyValues("spring.ai.vectorstore.gemfire.buckets=" + BUCKET_COUNT)
.withPropertyValues("spring.ai.vectorstore.gemfire.fields=someField1,someField2")
.withPropertyValues("spring.ai.vectorstore.gemfire.host=localhost")
.withPropertyValues("spring.ai.vectorstore.gemfire.port=" + HTTP_SERVICE_PORT);
.withPropertyValues("spring.ai.vectorstore.gemfire.port=" + HTTP_SERVICE_PORT)
.withPropertyValues("spring.ai.vectorstore.gemfire.initialize-schema=true");
@BeforeAll
public static void startGemFireCluster() {

5
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/milvus/MilvusVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -39,6 +39,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class MilvusVectorStoreAutoConfigurationIT {
@@ -62,7 +63,7 @@ public class MilvusVectorStoreAutoConfigurationIT {
"spring.ai.vectorstore.milvus.indexType=IVF_FLAT",
"spring.ai.vectorstore.milvus.embeddingDimension=384",
"spring.ai.vectorstore.milvus.collectionName=myTestCollection",
"spring.ai.vectorstore.milvus.initializeSchema=true",
"spring.ai.vectorstore.milvus.client.host=" + milvus.getHost(),
"spring.ai.vectorstore.milvus.client.port=" + milvus.getMappedPort(19530))
.run(context -> {

1
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/mongo/MongoDBAtlasVectorStoreAutoConfigurationIT.java
View File

@@ -65,6 +65,7 @@ class MongoDBAtlasVectorStoreAutoConfigurationIT {
MongoDBAtlasVectorStoreAutoConfiguration.class, RestClientAutoConfiguration.class,
SpringAiRetryAutoConfiguration.class, OpenAiAutoConfiguration.class))
.withPropertyValues("spring.data.mongodb.database=springaisample",
"spring.ai.vectorstore.mongodb.initialize-schema=true",
"spring.ai.vectorstore.mongodb.collection-name=test_collection",
// "spring.ai.vectorstore.mongodb.path-name=testembedding",
"spring.ai.vectorstore.mongodb.index-name=text_index",

5
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/neo4j/Neo4jVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -40,6 +40,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Jingzhou Ou
* @author Soby Chacko
*/
@Testcontainers
public class Neo4jVectorStoreAutoConfigurationIT {
@@ -59,7 +60,7 @@ public class Neo4jVectorStoreAutoConfigurationIT {
.withConfiguration(AutoConfigurations.of(Neo4jAutoConfiguration.class, Neo4jVectorStoreAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.neo4j.uri=" + neo4jContainer.getBoltUrl(),
"spring.neo4j.authentication.username=" + "neo4j",
"spring.ai.vectorstore.neo4j.initialize-schema=true", "spring.neo4j.authentication.username=" + "neo4j",
"spring.neo4j.authentication.password=" + neo4jContainer.getAdminPassword());
@Test

3
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/opensearch/AwsOpenSearchVectorStoreAutoConfigurationIT.java
View File

@@ -47,7 +47,7 @@ import static org.awaitility.Awaitility.await;
import static org.hamcrest.Matchers.hasSize;
@Testcontainers
class AwsOpenSearchVectorStoreAutoConfigurationIT {
class a {
@Container
private static final LocalStackContainer localstack = new LocalStackContainer(
@@ -65,6 +65,7 @@ class AwsOpenSearchVectorStoreAutoConfigurationIT {
.withConfiguration(AutoConfigurations.of(OpenSearchVectorStoreAutoConfiguration.class,
SpringAiRetryAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.opensearch.initialize-schema=true")
.withPropertyValues(
OpenSearchVectorStoreProperties.CONFIG_PREFIX + ".aws.host="
+ String.format("testcontainers-domain.%s.opensearch.localhost.localstack.cloud:%s",

4
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/opensearch/OpenSearchVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -63,7 +63,7 @@ class OpenSearchVectorStoreAutoConfigurationIT {
SpringAiRetryAutoConfiguration.class))
.withClassLoader(new FilteredClassLoader(Region.class, ApacheHttpClient.class))
.withUserConfiguration(Config.class)
.withPropertyValues(
.withPropertyValues("spring.ai.vectorstore.opensearch.initialize-schema=true",
OpenSearchVectorStoreProperties.CONFIG_PREFIX + ".uris=" + opensearchContainer.getHttpHostAddress(),
OpenSearchVectorStoreProperties.CONFIG_PREFIX + ".indexName=" + DOCUMENT_INDEX,
OpenSearchVectorStoreProperties.CONFIG_PREFIX + ".mappingJson=" + """

1
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/oracle/OracleVectorStoreAutoConfigurationIT.java
View File

@@ -63,6 +63,7 @@ public class OracleVectorStoreAutoConfigurationIT {
JdbcTemplateAutoConfiguration.class, DataSourceAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("test.spring.ai.vectorstore.oracle.distanceType=COSINE",
"spring.ai.vectorstore.oracle.initialize-schema=true",
"test.spring.ai.vectorstore.oracle.dimensions=384",
// JdbcTemplate configuration
String.format("spring.datasource.url=%s", oracle23aiContainer.getJdbcUrl()),

4
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/pgvector/PgVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -46,6 +46,7 @@ import org.testcontainers.junit.jupiter.Testcontainers;
/**
* @author Christian Tzolov
* @author Muthukumaran Navaneethakrishnan
* @author Soby Chacko
*/
@Testcontainers
public class PgVectorStoreAutoConfigurationIT {
@@ -75,6 +76,7 @@ public class PgVectorStoreAutoConfigurationIT {
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.pgvector.distanceType=COSINE_DISTANCE",
"spring.ai.vectorstore.pgvector.initialize-schema=true",
// JdbcTemplate configuration
String.format("spring.datasource.url=jdbc:postgresql://%s:%d/%s", postgresContainer.getHost(),
postgresContainer.getMappedPort(5432), postgresContainer.getDatabaseName()),
"spring.datasource.username=" + postgresContainer.getUsername(),

1
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/pinecone/PineconeVectorStoreAutoConfigurationIT.java
View File

@@ -42,6 +42,7 @@ import org.springframework.core.io.DefaultResourceLoader;
/**
* @author Christian Tzolov
* @author Soby Chacko
*/
@EnabledIfEnvironmentVariable(named = "PINECONE_API_KEY", matches = ".+")
public class PineconeVectorStoreAutoConfigurationIT {

4
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/qdrant/QdrantVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -41,6 +41,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
* @since 0.8.1
*/
@Testcontainers
@@ -58,6 +59,7 @@ public class QdrantVectorStoreAutoConfigurationIT {
.withConfiguration(AutoConfigurations.of(QdrantVectorStoreAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.ai.vectorstore.qdrant.port=" + qdrantContainer.getGrpcPort(),
"spring.ai.vectorstore.qdrant.initialize-schema=true",
"spring.ai.vectorstore.qdrant.host=" + qdrantContainer.getHost());
@Test

5
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/qdrant/QdrantVectorStoreCloudAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -46,6 +46,7 @@ import static org.assertj.core.api.Assertions.assertThat;
* Test using a free tier Qdrant Cloud instance: https://cloud.qdrant.io
*
* @author Christian Tzolov
* @author Soby Chacko
* @since 0.8.1
*/
// NOTE: The free Qdrant Cluster and the QDRANT_API_KEY expire after 4 weeks of
@@ -98,7 +99,7 @@ public class QdrantVectorStoreCloudAutoConfigurationIT {
"spring.ai.vectorstore.qdrant.host=" + CLOUD_HOST,
"spring.ai.vectorstore.qdrant.api-key=" + CLOUD_API_KEY,
"spring.ai.vectorstore.qdrant.collection-name=" + COLLECTION_NAME,
"spring.ai.vectorstore.qdrant.use-tls=true");
"spring.ai.vectorstore.qdrant.initializeSchema=true", "spring.ai.vectorstore.qdrant.use-tls=true");
@Test
public void addAndSearch() {

9
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/redis/RedisVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -40,6 +40,7 @@ import com.redis.testcontainers.RedisStackContainer;
/**
* @author Julien Ruaux
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
class RedisVectorStoreAutoConfigurationIT {
@@ -56,8 +57,10 @@ class RedisVectorStoreAutoConfigurationIT {
private final ApplicationContextRunner contextRunner = new ApplicationContextRunner()
.withConfiguration(AutoConfigurations.of(RedisAutoConfiguration.class, RedisVectorStoreAutoConfiguration.class))
.withUserConfiguration(Config.class)
.withPropertyValues("spring.data.redis.url=" + redisContainer.getRedisURI(),
"spring.ai.vectorstore.redis.index=myIdx", "spring.ai.vectorstore.redis.prefix=doc:");
.withPropertyValues("spring.data.redis.url=" + redisContainer.getRedisURI())
.withPropertyValues("spring.ai.vectorstore.redis.initialize-schema=true")
.withPropertyValues("spring.ai.vectorstore.redis.index=myIdx")
.withPropertyValues("spring.ai.vectorstore.redis.prefix=doc:");
@Test
void addAndSearch() {

4
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/typesense/TypesenseVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -39,6 +39,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Pablo Sanchidrian Herrera
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class TypesenseVectorStoreAutoConfigurationIT {
@@ -63,6 +64,7 @@ public class TypesenseVectorStoreAutoConfigurationIT {
contextRunner
.withPropertyValues("spring.ai.vectorstore.typesense.embeddingDimension=384",
"spring.ai.vectorstore.typesense.collectionName=myTestCollection",
"spring.ai.vectorstore.typesense.initialize-schema=true",
"spring.ai.vectorstore.typesense.client.apiKey=xyz",
"spring.ai.vectorstore.typesense.client.protocol=http",
"spring.ai.vectorstore.typesense.client.host=" + typesenseContainer.getHost(),

7
spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/weaviate/WeaviateVectorStoreAutoConfigurationTests.java → spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/vectorstore/weaviate/WeaviateVectorStoreAutoConfigurationIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -40,16 +40,15 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class WeaviateVectorStoreAutoConfigurationTests {
public class WeaviateVectorStoreAutoConfigurationIT {
@Container
static WeaviateContainer weaviate = new WeaviateContainer("semitechnologies/weaviate:1.25.4")
.waitingFor(Wait.forHttp("/v1/.well-known/ready").forPort(8080));
;
private final ApplicationContextRunner contextRunner = new ApplicationContextRunner()
.withConfiguration(AutoConfigurations.of(WeaviateVectorStoreAutoConfiguration.class))
.withUserConfiguration(Config.class)

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/chroma/ChromaContainerConnectionDetailsFactoryTest.java
View File

@@ -40,7 +40,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@Testcontainers
@TestPropertySource(properties = "spring.ai.vectorstore.chroma.store.collectionName=TestCollection")
@TestPropertySource(properties = { "spring.ai.vectorstore.chroma.store.collectionName=TestCollection",
"spring.ai.vectorstore.chroma.initialize-schema=true" })
class ChromaContainerConnectionDetailsFactoryTest {
@Container

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/chroma/ChromaWithToken2ContainerConnectionDetailsFactoryTest.java
View File

@@ -40,7 +40,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@Testcontainers
@TestPropertySource(properties = "spring.ai.vectorstore.chroma.store.collectionName=TestCollection")
@TestPropertySource(properties = { "spring.ai.vectorstore.chroma.store.collectionName=TestCollection",
"spring.ai.vectorstore.chroma.initialize-schema=true" })
class ChromaWithToken2ContainerConnectionDetailsFactoryTest {
@Container

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/chroma/ChromaWithTokenContainerConnectionDetailsFactoryTest.java
View File

@@ -40,7 +40,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@Testcontainers
@TestPropertySource(properties = "spring.ai.vectorstore.chroma.store.collectionName=TestCollection")
@TestPropertySource(properties = { "spring.ai.vectorstore.chroma.store.collectionName=TestCollection",
"spring.ai.vectorstore.chroma.initialize-schema=true" })
class ChromaWithTokenContainerConnectionDetailsFactoryTest {
@Container

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/milvus/MilvusContainerConnectionDetailsFactoryTest.java
View File

@@ -45,7 +45,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@Testcontainers
@TestPropertySource(properties = { "spring.ai.vectorstore.milvus.metricType=COSINE",
"spring.ai.vectorstore.milvus.indexType=IVF_FLAT", "spring.ai.vectorstore.milvus.embeddingDimension=384",
"spring.ai.vectorstore.milvus.collectionName=myTestCollection" })
"spring.ai.vectorstore.milvus.collectionName=myTestCollection",
"spring.ai.vectorstore.milvus.initialize-schema=true" })
class MilvusContainerConnectionDetailsFactoryTest {
@Container

1
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/opensearch/OpenSearchContainerConnectionDetailsFactoryTest.java
View File

@@ -44,6 +44,7 @@ import org.testcontainers.junit.jupiter.Testcontainers;
@SpringBootTest(properties = {
"spring.ai.vectorstore.opensearch.index-name=" + OpenSearchContainerConnectionDetailsFactoryTest.DOCUMENT_INDEX,
"spring.ai.vectorstore.opensearch.initialize-schema=true",
"spring.ai.vectorstore.opensearch.mapping-json="
+ OpenSearchContainerConnectionDetailsFactoryTest.MAPPING_JSON })
@Testcontainers

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/qdrant/QdrantContainerConnectionDetailsFactoryTest.java
View File

@@ -43,7 +43,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@Testcontainers
@TestPropertySource(properties = "spring.ai.vectorstore.qdrant.collectionName=test_collection")
@TestPropertySource(properties = { "spring.ai.vectorstore.qdrant.collectionName=test_collection",
"spring.ai.vectorstore.qdrant.initialize-schema=true" })
public class QdrantContainerConnectionDetailsFactoryTest {
@Container

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/qdrant/QdrantContainerWithApiKeyConnectionDetailsFactoryTest.java
View File

@@ -43,7 +43,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@Testcontainers
@TestPropertySource(properties = "spring.ai.vectorstore.qdrant.collectionName=test_collection")
@TestPropertySource(properties = { "spring.ai.vectorstore.qdrant.collectionName=test_collection",
"spring.ai.vectorstore.qdrant.initialize-schema=true" })
public class QdrantContainerWithApiKeyConnectionDetailsFactoryTest {
@Container

1
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/typesense/TypesenseContainerConnectionDetailsFactoryTest.java
View File

@@ -29,6 +29,7 @@ import static org.assertj.core.api.Assertions.assertThat;
@SpringJUnitConfig
@TestPropertySource(properties = { "spring.ai.vectorstore.typesense.embeddingDimension=384",
"spring.ai.vectorstore.typesense.initialize-schema=true",
"spring.ai.vectorstore.typesense.collectionName=myTestCollection" })
@Testcontainers
class TypesenseContainerConnectionDetailsFactoryTest {

3
spring-ai-spring-boot-testcontainers/src/test/java/org/springframework/ai/testcontainers/service/connection/weaviate/WeaviateContainerConnectionDetailsFactoryTest.java
View File

@@ -46,7 +46,8 @@ import static org.assertj.core.api.Assertions.assertThat;
@TestPropertySource(properties = { "spring.ai.vectorstore.weaviate.filter-field.country=TEXT",
"spring.ai.vectorstore.weaviate.filter-field.year=NUMBER",
"spring.ai.vectorstore.weaviate.filter-field.active=BOOLEAN",
"spring.ai.vectorstore.weaviate.filter-field.price=NUMBER" })
"spring.ai.vectorstore.weaviate.filter-field.price=NUMBER",
"spring.ai.vectorstore.weaviate.initialize-schema=true" })
class WeaviateContainerConnectionDetailsFactoryTest {
@Container

31
vector-stores/spring-ai-elasticsearch-store/src/main/java/org/springframework/ai/vectorstore/ElasticsearchVectorStore.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -21,7 +21,6 @@ import co.elastic.clients.elasticsearch.core.BulkResponse;
import co.elastic.clients.elasticsearch.core.SearchResponse;
import co.elastic.clients.elasticsearch.core.bulk.BulkResponseItem;
import co.elastic.clients.elasticsearch.core.search.Hit;
import co.elastic.clients.elasticsearch.indices.CreateIndexResponse;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import com.fasterxml.jackson.databind.DeserializationFeature;
@@ -57,6 +56,7 @@ import static org.springframework.ai.vectorstore.SimilarityFunction.l2_norm;
* @author Jemin Huh
* @author Wei Jiang
* @author Laura Trotta
* @author Soby Chacko
* @since 1.0.0
*/
public class ElasticsearchVectorStore implements VectorStore, InitializingBean {
@@ -98,12 +98,15 @@ public class ElasticsearchVectorStore implements VectorStore, InitializingBean {
logger.debug("Calling EmbeddingModel for document id = " + document.getId());
document.setEmbedding(this.embeddingModel.embed(document));
}
bulkRequestBuilder.operations(op -> op
.index(idx -> idx.index(this.options.getIndexName()).id(document.getId()).document(document)));
// We call operations on BulkRequest.Builder only if the index exists.
// For the index to be present, either it must be pre-created or set the
// initializeSchema to true.
if (indexExists()) {
bulkRequestBuilder.operations(op -> op
.index(idx -> idx.index(this.options.getIndexName()).id(document.getId()).document(document)));
}
}
BulkResponse bulkRequest = bulkRequest(bulkRequestBuilder.build());
if (bulkRequest.errors()) {
List<BulkResponseItem> bulkResponseItems = bulkRequest.items();
for (BulkResponseItem bulkResponseItem : bulkResponseItems) {
@@ -117,8 +120,14 @@ public class ElasticsearchVectorStore implements VectorStore, InitializingBean {
@Override
public Optional<Boolean> delete(List<String> idList) {
BulkRequest.Builder bulkRequestBuilder = new BulkRequest.Builder();
for (String id : idList)
bulkRequestBuilder.operations(op -> op.delete(idx -> idx.index(this.options.getIndexName()).id(id)));
// We call operations on BulkRequest.Builder only if the index exists.
// For the index to be present, either it must be pre-created or set the
// initializeSchema to true.
if (indexExists()) {
for (String id : idList) {
bulkRequestBuilder.operations(op -> op.delete(idx -> idx.index(this.options.getIndexName()).id(id)));
}
}
return Optional.of(bulkRequest(bulkRequestBuilder.build()).errors());
}
@@ -201,9 +210,9 @@ public class ElasticsearchVectorStore implements VectorStore, InitializingBean {
}
}
private CreateIndexResponse createIndexMapping() {
private void createIndexMapping() {
try {
return this.elasticsearchClient.indices()
this.elasticsearchClient.indices()
.create(cr -> cr.index(options.getIndexName())
.mappings(map -> map.properties("embedding", p -> p.denseVector(
dv -> dv.similarity(options.getSimilarity().toString()).dims(options.getDimensions())))));
@@ -215,11 +224,9 @@ public class ElasticsearchVectorStore implements VectorStore, InitializingBean {
@Override
public void afterPropertiesSet() {
if (!this.initializeSchema) {
return;
}
if (!indexExists()) {
createIndexMapping();
}

61
vector-stores/spring-ai-gemfire-store/src/main/java/org/springframework/ai/vectorstore/GemFireVectorStore.java
View File

@@ -63,6 +63,34 @@ public class GemFireVectorStore implements VectorStore, InitializingBean {
private static final String DOCUMENT_FIELD = "document";
private final boolean initializeSchema;
/**
* Configures and initializes a GemFireVectorStore instance based on the provided
* configuration.
* @param config the configuration for the GemFireVectorStore
* @param embeddingModel the embedding client used for generating embeddings
*/
public GemFireVectorStore(GemFireVectorStoreConfig config, EmbeddingModel embeddingModel,
boolean initializeSchema) {
Assert.notNull(config, "GemFireVectorStoreConfig must not be null");
Assert.notNull(embeddingModel, "EmbeddingModel must not be null");
this.initializeSchema = initializeSchema;
this.indexName = config.indexName;
this.embeddingModel = embeddingModel;
this.beamWidth = config.beamWidth;
this.maxConnections = config.maxConnections;
this.buckets = config.buckets;
this.vectorSimilarityFunction = config.vectorSimilarityFunction;
this.fields = config.fields;
String base = UriComponentsBuilder.fromUriString(DEFAULT_URI)
.build(config.sslEnabled ? "s" : "", config.host, config.port)
.toString();
this.client = WebClient.create(base);
}
// Create Index Parameters
private String indexName;
@@ -113,11 +141,12 @@ public class GemFireVectorStore implements VectorStore, InitializingBean {
*/
@Override
public void afterPropertiesSet() throws Exception {
if (indexExists()) {
deleteIndex();
if (!this.initializeSchema) {
return;
}
if (!indexExists()) {
createIndex();
}
createIndex();
}
/**
@@ -133,30 +162,6 @@ public class GemFireVectorStore implements VectorStore, InitializingBean {
return client.get().uri("/" + indexName).retrieve().bodyToMono(String.class).onErrorReturn("").block();
}
/**
* Configures and initializes a GemFireVectorStore instance based on the provided
* configuration.
* @param config the configuration for the GemFireVectorStore
* @param embeddingModel the embedding client used for generating embeddings
*/
public GemFireVectorStore(GemFireVectorStoreConfig config, EmbeddingModel embeddingModel) {
Assert.notNull(config, "GemFireVectorStoreConfig must not be null");
Assert.notNull(embeddingModel, "EmbeddingModel must not be null");
this.indexName = config.indexName;
this.embeddingModel = embeddingModel;
this.beamWidth = config.beamWidth;
this.maxConnections = config.maxConnections;
this.buckets = config.buckets;
this.vectorSimilarityFunction = config.vectorSimilarityFunction;
this.fields = config.fields;
String base = UriComponentsBuilder.fromUriString(DEFAULT_URI)
.build(config.sslEnabled ? "s" : "", config.host, config.port)
.toString();
this.client = WebClient.create(base);
}
public static class CreateRequest {
@JsonProperty("name")

2
vector-stores/spring-ai-gemfire-store/src/test/java/org/springframework/ai/vectorstore/GemFireVectorStoreIT.java
View File

@@ -216,7 +216,7 @@ public class GemFireVectorStoreIT {
@Bean
public GemFireVectorStore vectorStore(GemFireVectorStoreConfig config, EmbeddingModel embeddingModel) {
return new GemFireVectorStore(config, embeddingModel);
return new GemFireVectorStore(config, embeddingModel, true);
}
@Bean

23
vector-stores/spring-ai-opensearch-store/src/main/java/org/springframework/ai/vectorstore/OpenSearchVectorStore.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -45,6 +45,7 @@ import java.util.stream.Collectors;
/**
* @author Jemin Huh
* @author Soby Chacko
* @since 1.0.0
*/
public class OpenSearchVectorStore implements VectorStore, InitializingBean {
@@ -78,16 +79,21 @@ public class OpenSearchVectorStore implements VectorStore, InitializingBean {
private String similarityFunction;
public OpenSearchVectorStore(OpenSearchClient openSearchClient, EmbeddingModel embeddingModel) {
this(openSearchClient, embeddingModel, DEFAULT_MAPPING_EMBEDDING_TYPE_KNN_VECTOR_DIMENSION_1536);
private final boolean initializeSchema;
public OpenSearchVectorStore(OpenSearchClient openSearchClient, EmbeddingModel embeddingModel,
boolean initializeSchema) {
this(openSearchClient, embeddingModel, DEFAULT_MAPPING_EMBEDDING_TYPE_KNN_VECTOR_DIMENSION_1536,
initializeSchema);
}
public OpenSearchVectorStore(OpenSearchClient openSearchClient, EmbeddingModel embeddingModel, String mappingJson) {
this(DEFAULT_INDEX_NAME, openSearchClient, embeddingModel, mappingJson);
public OpenSearchVectorStore(OpenSearchClient openSearchClient, EmbeddingModel embeddingModel, String mappingJson,
boolean initializeSchema) {
this(DEFAULT_INDEX_NAME, openSearchClient, embeddingModel, mappingJson, initializeSchema);
}
public OpenSearchVectorStore(String index, OpenSearchClient openSearchClient, EmbeddingModel embeddingModel,
String mappingJson) {
String mappingJson, boolean initializeSchema) {
Objects.requireNonNull(embeddingModel, "RestClient must not be null");
Objects.requireNonNull(embeddingModel, "EmbeddingModel must not be null");
this.openSearchClient = openSearchClient;
@@ -98,6 +104,7 @@ public class OpenSearchVectorStore implements VectorStore, InitializingBean {
// the potential functions for vector fields at
// https://opensearch.org/docs/latest/search-plugins/knn/approximate-knn/#spaces
this.similarityFunction = COSINE_SIMILARITY_FUNCTION;
this.initializeSchema = initializeSchema;
}
public OpenSearchVectorStore withSimilarityFunction(String similarityFunction) {
@@ -228,8 +235,8 @@ public class OpenSearchVectorStore implements VectorStore, InitializingBean {
@Override
public void afterPropertiesSet() {
if (!exists(this.index)) {
createIndexMapping(this.index, mappingJson);
if (this.initializeSchema && !exists(this.index)) {
createIndexMapping(this.index, this.mappingJson);
}
}

9
vector-stores/spring-ai-opensearch-store/src/test/java/org/springframework/ai/vectorstore/OpenSearchVectorStoreIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -54,6 +54,11 @@ import static org.assertj.core.api.Assertions.assertThat;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
/**
* @author Jemin Huh
* @author Soby Chacko
* @since 1.0.0
*/
@Testcontainers
@EnabledIfEnvironmentVariable(named = "OPENAI_API_KEY", matches = ".+")
class OpenSearchVectorStoreIT {
@@ -346,7 +351,7 @@ class OpenSearchVectorStoreIT {
try {
return new OpenSearchVectorStore(new OpenSearchClient(ApacheHttpClient5TransportBuilder
.builder(HttpHost.create(opensearchContainer.getHttpHostAddress()))
.build()), embeddingModel);
.build()), embeddingModel, true);
}
catch (URISyntaxException e) {
throw new RuntimeException(e);

1
vector-stores/spring-ai-pinecone-store/src/main/java/org/springframework/ai/vectorstore/PineconeVectorStore.java
View File

@@ -13,6 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.vectorstore;
import java.time.Duration;

31
vector-stores/spring-ai-typesense-store/src/main/java/org/springframework/ai/vectorstore/TypesenseVectorStore.java
View File

@@ -1,3 +1,19 @@
/*
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.vectorstore;
import java.util.HashMap;
@@ -25,6 +41,7 @@ import org.typesense.model.MultiSearchSearchesParameter;
/**
* @author Pablo Sanchidrian Herrera
* @author Soby Chacko
*/
public class TypesenseVectorStore implements VectorStore, InitializingBean {
@@ -56,6 +73,8 @@ public class TypesenseVectorStore implements VectorStore, InitializingBean {
public final FilterExpressionConverter filterExpressionConverter = new TypesenseFilterExpressionConverter();
private final boolean initializeSchema;
public static class TypesenseVectorStoreConfig {
private final String collectionName;
@@ -127,16 +146,18 @@ public class TypesenseVectorStore implements VectorStore, InitializingBean {
}
public TypesenseVectorStore(Client client, EmbeddingModel embeddingModel) {
this(client, embeddingModel, TypesenseVectorStoreConfig.defaultConfig());
this(client, embeddingModel, TypesenseVectorStoreConfig.defaultConfig(), false);
}
public TypesenseVectorStore(Client client, EmbeddingModel embeddingModel, TypesenseVectorStoreConfig config) {
public TypesenseVectorStore(Client client, EmbeddingModel embeddingModel, TypesenseVectorStoreConfig config,
boolean initializeSchema) {
Assert.notNull(client, "Typesense must not be null");
Assert.notNull(embeddingModel, "EmbeddingModel must not be null");
this.client = client;
this.embeddingModel = embeddingModel;
this.config = config;
this.initializeSchema = initializeSchema;
}
@Override
@@ -265,8 +286,10 @@ public class TypesenseVectorStore implements VectorStore, InitializingBean {
// Initialization
// ---------------------------------------------------------------------------------
@Override
public void afterPropertiesSet() throws Exception {
this.createCollection();
public void afterPropertiesSet() {
if (this.initializeSchema) {
this.createCollection();
}
}
private boolean hasCollection() {

46
vector-stores/spring-ai-typesense-store/src/test/java/org/springframework/ai/vectorstore/TypesenseVectorStoreIT.java
View File

@@ -1,3 +1,19 @@
/*
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.ai.vectorstore;
import org.junit.jupiter.api.Test;
@@ -32,6 +48,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/**
* @author Pablo Sanchidrian Herrera
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class TypesenseVectorStoreIT {
@@ -59,22 +76,12 @@ public class TypesenseVectorStoreIT {
}
}
private void resetCollection(VectorStore vectorStore) {
((TypesenseVectorStore) vectorStore).dropCollection();
((TypesenseVectorStore) vectorStore).createCollection();
}
@Test
void documentUpdate() {
contextRunner.run(context -> {
VectorStore vectorStore = context.getBean(VectorStore.class);
resetCollection(vectorStore);
Document document = new Document(UUID.randomUUID().toString(), "Spring AI rocks!!",
Collections.singletonMap("meta1", "meta1"));
vectorStore.add(List.of(document));
Map<String, Object> info = ((TypesenseVectorStore) vectorStore).getCollectionInfo();
@@ -112,17 +119,16 @@ public class TypesenseVectorStoreIT {
info = ((TypesenseVectorStore) vectorStore).getCollectionInfo();
assertThat(info.get("num_documents")).isEqualTo(0L);
((TypesenseVectorStore) vectorStore).dropCollection();
});
}
@Test
void addAndSearch() {
contextRunner.run(context -> {
VectorStore vectorStore = context.getBean(VectorStore.class);
resetCollection(vectorStore);
vectorStore.add(documents);
Map<String, Object> info = ((TypesenseVectorStore) vectorStore).getCollectionInfo();
@@ -132,17 +138,16 @@ public class TypesenseVectorStoreIT {
List<Document> results = vectorStore.similaritySearch(SearchRequest.query("Spring"));
assertThat(results).hasSize(3);
((TypesenseVectorStore) vectorStore).dropCollection();
});
}
@Test
void searchWithFilters() {
contextRunner.run(context -> {
VectorStore vectorStore = context.getBean(VectorStore.class);
resetCollection(vectorStore);
var bgDocument = new Document("The World is Big and Salvation Lurks Around the Corner",
Map.of("country", "BG", "year", 2020));
var nlDocument = new Document("The World is Big and Salvation Lurks Around the Corner",
@@ -188,18 +193,17 @@ public class TypesenseVectorStoreIT {
assertThat(results.get(0).getId()).isIn(nlDocument.getId(), bgDocument2.getId());
assertThat(results.get(1).getId()).isIn(nlDocument.getId(), bgDocument2.getId());
((TypesenseVectorStore) vectorStore).dropCollection();
});
}
@Test
void searchWithThreshold() {
contextRunner.run(context -> {
VectorStore vectorStore = context.getBean(VectorStore.class);
resetCollection(vectorStore);
vectorStore.add(documents);
List<Document> fullResult = vectorStore
@@ -221,6 +225,8 @@ public class TypesenseVectorStoreIT {
"Spring AI provides abstractions that serve as the foundation for developing AI applications.");
assertThat(resultDoc.getMetadata()).containsKeys("meta1", "distance");
((TypesenseVectorStore) vectorStore).dropCollection();
});
}
@@ -236,7 +242,7 @@ public class TypesenseVectorStoreIT {
.withEmbeddingDimension(embeddingModel.dimensions())
.build();
return new TypesenseVectorStore(client, embeddingModel, config);
return new TypesenseVectorStore(client, embeddingModel, config, true);
}
@Bean

41
vector-stores/spring-ai-weaviate-store/src/main/java/org/springframework/ai/vectorstore/WeaviateVectorStore.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -62,8 +62,9 @@ import org.springframework.util.StringUtils;
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Josh Long
* @author Soby Chacko
*/
public class WeaviateVectorStore implements VectorStore, InitializingBean {
public class WeaviateVectorStore implements VectorStore {
public static final String DOCUMENT_METADATA_DISTANCE_KEY_NAME = "distance";
@@ -281,11 +282,10 @@ public class WeaviateVectorStore implements VectorStore, InitializingBean {
* @param embeddingModel The client for embedding operations.
*/
public WeaviateVectorStore(WeaviateVectorStoreConfig vectorStoreConfig, EmbeddingModel embeddingModel,
WeaviateClient weaviateClient, boolean initializeSchema) {
WeaviateClient weaviateClient) {
Assert.notNull(vectorStoreConfig, "WeaviateVectorStoreConfig must not be null");
Assert.notNull(embeddingModel, "EmbeddingModel must not be null");
this.initializeSchema = initializeSchema;
this.embeddingModel = embeddingModel;
this.consistencyLevel = vectorStoreConfig.consistencyLevel;
this.weaviateObjectClass = vectorStoreConfig.weaviateObjectClass;
@@ -526,37 +526,4 @@ public class WeaviateVectorStore implements VectorStore, InitializingBean {
return doubleList.stream().map(Number::floatValue).toList().toArray(new Float[0]);
}
private final boolean initializeSchema;
@Override
public void afterPropertiesSet() throws Exception {
if (!this.initializeSchema) {
return;
}
Map<String, Object> metadata = new HashMap<>();
if (!CollectionUtils.isEmpty(this.filterMetadataFields)) {
for (MetadataField mf : this.filterMetadataFields) {
switch (mf.type()) {
case TEXT:
metadata.put(mf.name(), "Hello");
break;
case NUMBER:
metadata.put(mf.name(), 3.14);
break;
case BOOLEAN:
metadata.put(mf.name(), true);
break;
default:
break;
}
}
}
var document = new Document("Hello world", metadata);
this.add(List.of(document));
this.delete(List.of(document.getId()));
}
}

5
vector-stores/spring-ai-weaviate-store/src/test/java/org/springframework/ai/vectorstore/WeaviateVectorStoreIT.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2023 - 2024 the original author or authors.
* Copyright 2023-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -46,6 +46,7 @@ import io.weaviate.client.WeaviateClient;
/**
* @author Christian Tzolov
* @author Eddú Meléndez
* @author Soby Chacko
*/
@Testcontainers
public class WeaviateVectorStoreIT {
@@ -256,7 +257,7 @@ public class WeaviateVectorStoreIT {
.withConsistencyLevel(WeaviateVectorStoreConfig.ConsistentLevel.ONE)
.build();
return new WeaviateVectorStore(config, embeddingModel, weaviateClient, true);
return new WeaviateVectorStore(config, embeddingModel, weaviateClient);
}