Commit Graph

156 Commits

Author SHA1 Message Date
Soby Chacko
50d34b8a48 BREAKING CHANGE - Change vector store initialize-schema to false
* Change default schema initialization of vector stores from `true` to `false.`
  Users need to explicitly opt-in for schema initialization by setting the
  `initialize-schema` property on the corresponding vector store.
* Update integration tests
* Update docs

Fixes #907
2024-07-18 14:28:00 -04:00
Michael Simons
41d2cb5827 Upgrade to latest version of Cypher-DSL.
Also use the bom, so if anyone combines it with Spring Data, things settle at some common version.
2024-07-18 15:04:49 +02:00
Eddú Meléndez
0a07f65d6a Use RedisAutoConfiguration in RedisVectorStoreAutoConfiguration
Currently, `RedisVectorStoreAutoConfiguration` creates its own
configuration to connect with Redis. This commit reuse
`RedisAutoConfiguration` from spring boot project. It's limited
to Jedis.
2024-07-18 14:24:10 +02:00
mck
4aacab020b Implement CassandraChatMemory
- provides a flexible schema, can be combined with a vector store, and supports time-to-live rows.
 - fix initialize-schema docs and so that it actually works.
 - move CommonVectorStoreProperties to .vectorstore. package
 - add CassandraAutoConfiguration to the AutoConfiguration.imports
2024-07-17 11:42:09 +02:00
Eddú Meléndez
0a42bf01f3 Fix assertions 2024-07-17 11:01:37 +02:00
Francisco Javier Torres
867bd173e5 Set embeddings on added Document objects for Milvus and PgVector stores. 2024-07-05 16:37:38 -04:00
jiwoo
6931148377 Fix MilvusVectorStore max allowed dimension size to 32768 2024-07-05 16:32:59 -04:00
Ross Lawley
24da2efc86 Fix IndexAlreadyExists exception for spring-ai-mongodb-atlas-store
In testing the docker mongodb/atlas container doesn't throw an exception when
calling `mongoTemplate.executeCommand({"createSearchIndexes": ...});`. However,
when using the Atlas service it does and throws a `IndexAlreadyExists` exception.

Added error handling for error code 68 or error code name `IndexAlreadyExists`.
See: https://www.mongodb.com/docs/manual/reference/error-codes/

Fixes #910
2024-07-05 14:38:38 -04:00
Christian Tzolov
c09286eedf Fix javadoc formatting 2024-06-28 11:31:57 +02:00
dongfeng3692
df52c9fbb5 Minor javadoc improvements 2024-06-28 11:20:07 +02:00
Mark Pollack
a1c4a29073 AzureVectorStore Fix - when not initializing schema, create searchClient from index name 2024-06-26 19:22:53 -04:00
Eddú Meléndez
feecca77c3 Use Weaviate's readiness endpoint
Currently, Testcontainers Weaviate implementation relies on port availability.
This commit switch to Weaviate's readiness endpoint.
2024-06-24 17:48:04 +02:00
Mark Pollack
84737ecc01 update some javadoc warnings 2024-06-21 16:29:15 -04:00
Muthukumaran Navaneethakrishnan
f0ca61252b Add support for customizable PgVectorStore schema, table, and index names (#747)
This change allows users to specify custom names, facilitating management of multiple vector databases within a single database instance.

 Key changes:
 - Implement configurable schema, table, and index names for PgVectorStore
 - Add properties to set custom schema and table names
 - Introduce optional schema/table & field validation for custom configurations
 - Include additional tests for new configurations and existing deployments

 Additional improvements:
 - Rename properties to schemaName, tableName, and schemaValidation
 - Update pgvector documentation with new properties
 - Add schema/table name tests to PgVectorStoreAutoConfigurationIT and PgVectorStorePropertiesTests
 - Create standalone PgVectorSchemaValidator class for schema/table validation
 - Add missing 'CREATE SCHEMA IF NOT EXISTS' when initializeSchema=true
 - Remove redundant code and classes

 Resolves #747

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
2024-06-21 11:02:22 +02:00
geetrawat
067a33dbe2 Improved GemFire support
- Adds spring boot auto-configuration support for GemFireVectorStore
- Adds integration test GemFireVectorStoreAutoConfigurationIT
- Includes gemfire-testcontainers in integration tests
- Adds unit test GemFireVectorStorePropertiesTests
- Refactors GemFireVectorStore.java extracting GemFireVectorStoreConfig.java
- Renames spring-ai-gemfire to spring-ai-gemfire-store
- Adds GemFireConnectionDetails
- Adds GemFireVectorStoreProperties with default values
- Remove gemfire-release-repo maven repository

Co-authored-by: Louis Jacome <louis.jacome@broadcom.com>
Co-authored-by: Jason Huyn <jason.huynh@broadcom.com>
2024-06-20 17:17:13 -04:00
Eddú Meléndez
5844f9bf39 Polish Qdrant tests
Use helper methods provided by Testcontainers.
2024-06-20 07:16:03 +02:00
Eddú Meléndez
e4607a6c29 Support ChromaDB's CHROMA_SERVER_AUTHN_CREDENTIALS env var in Docker Compose and Testcontainers
Read `CHROMA_SERVER_AUTHN_CREDENTIALS` env var from compose file and
Testcontainers definition to configure the connection details for
ChromaDB.
2024-06-19 16:08:57 +02:00
Eddú Meléndez
6ce998f9d3 Add support for Typesense Service Connection
Docker Compose and Testcontainers Service Connection support for
Typesense.
2024-06-19 15:59:49 +02:00
Christian Tzolov
b7475ed060 Update Weaviate docker version. remove import wildcard definitions 2024-06-18 09:33:30 +02:00
Christian Tzolov
2c7dc3d6f1 Pinecone: Add configurable content and distance metadata fields (#882)
- Add `withContentFieldName` and `withDistanceMetadataFieldName` to Pinecone Config Builder
 - Add `spring.ai.vectorstore.pinecone.contentFieldName` and `spring.ai.vectorstore.pinecone.distanceMetadataFieldName`
   properties to the auto-config
 - Default content field name: "document_content", distance field: "distance"
 - Update tests and docs

 Resolves #882
2024-06-18 07:57:15 +02:00
Lee-ChungMu
dfabcc7f61 Use List.of instead of Arrays.asList In the SEARCH_OUTPUT_FIELDS field.
The Arrays.asList method can change elements. SEARCH_OUTPUT_FIELDS is an immutable field, so it has been improved to util List.of.
2024-06-17 15:51:26 +02:00
Laura Trotta
78f3797f8d Update Elasticsearch Store: Use KNN insettad of script_score
- knn instead of script_score, removed initialization
 - only using normalized similarities, adjusted unit test
 - making l2norm's distances consistent with others
 - update dependency version and docs
 - upate autoconfigure ITs
2024-06-17 15:17:48 +02:00
Christian Tzolov
958549ecce Rename all Embedding Client doc and variables occurrences into Embedding Model 2024-06-16 21:24:03 +02:00
Eddú Meléndez
704dec20d9 Chroma Vector Store: replace RestTemplate by RestClient
- replace RestTemplate by  RestClient but default to SimpleClientHttpRequestFactory as Chroma seems to have issues with HTTP2.
 - update the documentation.
 - update the ghcr.io/chroma-core/chroma version to 0.5.0. Fix the withBasicAuthCredentials.

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
2024-06-16 21:07:03 +02:00
Jemin Huh
46e47849ce Add OpenSearch vector store integration
- implement OpensSearchVectorStore
 - add opensearch auto-configuration and boot starter
 - add documentation for OpenSearch VectorStore
 - add bom dependecies
 - align with to new Spirng AI API
2024-06-16 12:43:09 +02:00
Christian Tzolov
476164b6e3 Move the vector expression converters from core to their projects 2024-06-15 20:24:29 +02:00
PabloSanchi
3e2ed8b9bf Add Typesense vector store integration
- autoconfigure setup
 - add post bean initialization and create method
 - add embedding field
 - create collection add nested field options
 - add typesense tests
 - use embedding variable instead of word vec
 - check in runtime the number of documents in the collection
 - add typesense expression converter
 - add filter tests. add update document test and search with threshold test
 - distance threshold and add distance key into metadata
 - add typesesne boot starter
 - add typesense docs
 - add client properties in autoconfigure
 - add embedding dimension method
 - add typesense vector store autoconfiguration tests
 - add docs to nav.adoc and vectorsdb.adoc.
 - fix module name.
 - move the expression converter to the typesense project.
2024-06-15 19:55:05 +02:00
Christian Tzolov
1d479e3aab Minor PgVector ITs improvements 2024-06-15 17:49:21 +02:00
Christian Tzolov
baadf4cb53 Fix Oracle Vector Store dependencies
- force override the outdated SpringBoot Oracle version.
 - add auto-config tests.
2024-06-15 12:21:56 +02:00
Christian Tzolov
a5d92ec30f Adjust Oracle Vector store class names 2024-06-14 06:55:54 +02:00
LLEFEVRE
41cf693bc0 Add Support for Oracle 23ai as vector database
- add OracleVectorStore with metadata filter expression support.
 - add ITs using oracle-free-slim testcontainers.
 - add auto-configuration and boot starter.
 - add adoc documentation.
 - Adjust javadoc references.

 Resolves #703

 Co-authored-by: Eddú Meléndez Gonzales <eddu.melendez@gmail.com>
2024-06-14 06:46:01 +02:00
yinpeng
20c6a1adcf fix: typo 2024-06-03 15:00:19 +02:00
Mark Pollack
ac91302eed Next development version 2024-05-28 13:53:04 -04:00
Mark Pollack
0670575f3e Release version 1.0.0-M1 2024-05-28 13:49:11 -04:00
Josh Long
2d43e40024 Add property to initialize schema for vector stores
* Default is fale
* Update docs
2024-05-26 00:31:19 -04:00
skewgod
593e5fd747 docs(qdrant): added additional info to setup api key for qdrant (#736) 2024-05-24 11:36:26 -04:00
Mark Pollack
4a2ad60059 Package refactoring
* Move ChatClient and related classes into the chat.client package
* Move ChatModel and related class into the chat.model package
* Smaller refactorings to remove DSM cycles
* Update README.md
2024-05-23 17:52:27 -04:00
Josh Long
fbfc87e814 Refactoring of ChatClient to add fluent API and introduce Model as dependent object
* Rename the ModelClient class hierarchy into Model:
  - Rename ModelClient into Model. Update all code and doc references.
  - Rename ChatClient to ChatModel. Update all ChatClient suffixes and chatClient fields and variables in code and doc.
  - Rename EmbeddingClient into EmbeddingModel. Update the XxxEmbeddingClient class and variable suffixes and embeddingClient variables and fields in code and docs.
  - Rename ImageClient into ImageModel.
  - Rename SpeechClient into SpeechModel.
  - Rename TranscriptionClient into TranscriptionModel.
  - Update all javadocs and antora pages. Update the related diagrams.

* Create fluent API in ChatClient interface that now includes streaming support
* Add OpenAI FunctionCallbackWrapper2IT auto-config tests.
* Add ChatClientTest mockito testing.
* Add ChatModel#getDefaultOptions(), and remove @FunctionalInterface

* ChatModel enums extend the new ModelDescription interface.
* Implement fromOptions copy method in every ChatOptions implementation.
* Extend ChatClient to use the model default options if not provided explicitly.

* Update readme to provide guidance on how to adapt to breaking changes.

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
Co-authored-by: Mark Pollack <mpollack@vmware.com>
2024-05-22 16:07:12 -04:00
Dimitrios Begnis
bce45c2d2f Use <> for inequality operator in Neo4j filter expression implementation 2024-05-21 17:17:32 -04:00
Christian Tzolov
3475f17e98 Unify the vector store module and pom names
spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-cassandra -> spring-ai-cassandra-store
 spring-ai-pinecone -> spring-ai-pinecone-store
 spring-ai-redis -> spring-ai-redis-store
 spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-gemfire -> spring-ai-gemfire-store
 spring-ai-azure-vector-store-spring-boot-starter -> spring-ai-azure-store-spring-boot-starter
 spring-ai-redis-spring-boot-starter -> spring-ai-redis-store-spring-boot-starter
2024-05-17 17:05:09 +02:00
Christian Tzolov
1985824fa9 clean code imports 2024-05-17 15:32:46 +02:00
Christian Tzolov
3ee04ed8da fix code style and missing dependency renaming 2024-05-17 15:31:58 +02:00
Josh Long
04d854cc49 make project names consistent 2024-05-17 15:31:58 +02:00
Josh Long
c5b7dd9c46 i ran the spring javaformat maven plugin to clean up any source code incompatabilities. 2024-05-17 15:31:57 +02:00
Josh Long
8c758617f3 moving to ApplicationListener<ApplicationReadyEvent> to avoid slowing down the application on startup. 2024-05-17 15:31:53 +02:00
Eddú Meléndez
d610dd6f1d Add default collection name to Qdrant
Add a default collection name similar to other vector store
implementations. Currently, when using starters, qdrant requires
a collection name. Otherwise, it fails.
2024-05-15 12:02:05 +02:00
Christian Tzolov
b0799babf4 Update Qdrant client version from 1.7.1 to 1.9.1
- Also update the qdrant/qdrant docker image version from v1.7.4 to v1.9.2
2024-05-15 06:19:46 +02:00
Eddú Meléndez
cdb80e83e9 Fix ChromaDB basic authentication
Currently, `username` is added as a password in `BasicAuthenticationInterceptor`.
This commit fixes the issue and also make sure the integration test
setup is correct.
2024-05-11 14:51:28 +02:00
Eddú Meléndez
5beef21a4e Expose QdrantClient and WeaviateClient as beans
Currently, `QdrantClient` and `WeaviateClient` are not exposed as
  beans. Having access to those would benefit to perform operations
  with an already configured client.

  - Deprecate  QdrantVectorStoreConfig.
  - Update Qdrant manual config adoc.
  - Improve Qdrant adoc.
  - Update Weaviate docs.
2024-05-10 15:15:14 +03:00
Christian Tzolov
6154fa663f Disable MondoDB IT due to #698 2024-05-09 13:27:57 +03:00