- Avoid overlapping package names
Changed in spring-ai-commons package from org.sf.ai.model to org.sf.ai.content
Refactor advisor module name to be spring-ai-advisors-vector-store
Moved advisors into org.springframework.ai.chat.client.advisor.vectorstore
- Created top level memory directory
- Create new module spring-ai-model-chat-memory-neo4j and moved neo4j memory classes out of the vectorstore module
Updated neo4j autoconfiguation
- Remove batch strategy usage as it is taken care by the vector store parent builder
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Fixes: #938
Issue link: https://github.com/spring-projects/spring-ai/issues/938
This commit integrates Couchbase as a vector store option in Spring AI, providing:
- CouchbaseSearchVectorStore implementation with vector similarity search capabilities
- Support for metadata filtering with SQL++ expression conversion
- Spring Boot auto-configuration and starter module for easy integration
- Comprehensive documentation covering setup, configuration, and usage examples
- Integration tests using TestContainers with Couchbase 7.6
The implementation supports configuring dimensions, similarity functions (dot_product/l2_norm),
and optimization strategies (recall/latency). Schema initialization is now opt-in via
the initializeSchema property. Documentation includes both auto-configuration and
manual configuration instructions, along with property configuration details.
Signed-off-by: Abhiraj <abhiraj.official15@gmail.com>
co-authored-by: Laurent Doguin <laurent.doguin@gmail.com>
- Remove deprecations from models, vector stores and usage
- Deprecations from FunctionCallback and ObservationContext/Convention will be in a separate PR
Models updates
- Remove AbstractToolCallSupport from the models which use ToolCallingManager
- Remove deprecated constructors and their usage
- Remove FunctionCallbackResolver and FunctionCallbacks usage in the models
- Add back deprecations for VectorStoreChatMemoryAdvisor until builder is fixed
- Update OpenAiPaymentTransactionIT to use ToolCallbackResolver in config
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Updated `SearchRequest.java` to make the class non-final.
- Added `MilvusSearchRequest` with specific Milvus parameters such as `nativeExpression` and `searchParamsJson`.
- Modified `doSimilaritySearch` method in `MilvusVectorStore` to handle these new fields from `MilvusSearchRequest`.
Add unit tests for MilvusVectorStore and MilvusSearchRequest
Introduce comprehensive unit tests to validate the functionality of MilvusVectorStore and MilvusSearchRequest, including scenarios for native and filter expressions. Refactor MilvusVectorStore to improve filter expression handling by introducing a helper method for converted expressions.
Add detailed documentation for MilvusSearchRequest usage
Introduced sections explaining MilvusSearchRequest's parameters, `nativeExpression`, and `searchParamsJson`, with examples for enhanced clarity. This update provides guidance on leveraging Milvus-specific features for precise filtering and optimal search performance.
Signed-off-by: waileong <wai_leong1015@hotmail.com>
- Upgrade the java client from 0.8.0 to 4.0.1
- Use io.pinecone.clients.Pinecone to setup the client configuration with the API key
- Remove projectId, environment and other deprecated client configurations
- Update upsert, delete, search operations with the new client
- Remove the projectID and Environment configurations from the builder
- Updated the Pinecone vectorstore autoconfiguration
- Update tests
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Disable autoconfig IT until the auto-configuration is modularised
- Upgrade Milvus java sdk to 2.5.4
- Remove use of fastjson
- Use com.google.gson.JsonObject as Milvus SDK replaces fastjson with gson
- Change SearchResult RowRecord similarity metric name to score
- Milvus SDK 2.5.4 uses "score" instead of "distance"
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Extract common vector store delete tests to base class
This commit extracts shared delete operation tests into a reusable BaseVectorStoreTests class.
This reduces code duplication and provides a consistent test suite for delete operations across
different vector store implementations. The base class includes tests for:
Deleting by ID
Deleting by filter expressions
Deleting by string filter expressions
Most of the vector store implementation now extends this base class and inherits these
common tests while maintaining the ability to add vector store specific tests.
Adding javadoc
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
- Change VectorStore.delete() and related implementations to return void instead of Optional<Boolean>
- Remove unnecessary boolean return values and success status checks across all vector store implementations
- Clean up tests by removing redundant assertions
- Implementations continue using runtime exceptions for error signaling
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
- Spring AI's dependencies management to derive from Spring Boot 3.4.2
- Remove explicit versioning of dependencies
- Update upgrade notes for Spring Boot 3.4.2
- Fix ElasticSearch client changes with the latest dependency derived from Spring Boot 3.4.2
- Add handling for UUID, TEXT, INTEGER, SERIAL, BIGSERIAL formats in `convertIdToPgType` function.
- Implemented type conversion logic based on the IdType value (UUID, TEXT, INTEGER, SERIAL, BIGSERIAL).
- Add unit tests to validate correct conversion for UUID and non-UUID IdType formats.
- `testToPgTypeWithUuidIdType`: Validates UUID handling.
- `testToPgTypeWithNonUuidIdType`: Validates handling for non-UUID IdTypes.
Signed-off-by: jitokim <pigberger70@gmail.com>
Adds getNativeClient API to VectorStore interface allowing access to the underlying native client implementation.
This change:
- Adds getNativeClient() default method to VectorStore interface returning Optional<T>
- Implements getNativeClient() in all vector store implementations exposing their respective native clients
- Adds integration tests verifying native client access for all implementations
Fixes: #2137
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Replace the existing builder implementation with a type-safe Step Builder pattern
to enforce required field initialization in a specific order. This change:
- Introduces a new builder() method that guides users through required fields
- Deprecates the old builder method with all parameters
- Makes field configuration more explicit and type-safe
- Updates all usage points to use the new builder pattern
The required fields must now be provided in sequence:
1. embeddingModel
2. apiKey
3. projectId
4. environment
5. indexName
This change improves API usability by preventing parameter confusion and ensuring
all required fields are set before optional configuration.
Implement filter-based deletion for WeaviateVectorStore by leveraging the
existing ID-based deletion mechanism.
Key changes:
- Add doDelete(Filter.Expression) implementation that delegates to the existing
ID-based deletion method
- Use similarity search with filter to find matching documents first
- Delete matched documents by their IDs
- Add integration tests for filter deletion scenarios:
* Deleting by simple equality filter
* Deleting by string-based filter expression
This maintains consistency with other vector store implementations while
working around Weaviate's limitations in direct filtered deletion.
Implement filter-based deletion for CassandraVectorStore to support more
flexible document removal based on metadata filters.
Key changes:
- Add doDelete(Filter.Expression) implementation
- Implement workaround for Cassandra's limitations in direct filtered deletion
- Fetch matching documents first, then delete by IDs
- Add integration tests for various filter deletion scenarios
- Support simple and complex filter expressions
- Includes tests for:
* Deleting by simple equality filter
* Deleting by string-based filter expression
* Deleting by complex AND filter expression
Provides consistent deletion capabilities across vector store implementations
while addressing Cassandra-specific constraints.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Pinecone vector store and improve test organization through refactoring.
Key changes:
- Add delete(Filter.Expression) implementation using two-step process:
* Search for documents matching filter
* Delete matching documents by ID
- Leverage existing search functionality to maintain filter consistency
- Extract common test patterns into helper methods
- Create reusable document factory methods for test data
- Add comprehensive integration tests for filter deletion
- Standardize test cleanup and verification patterns
This maintains consistency with other vector store implementations while
working within Pinecone's API limitations. The test refactoring improves
maintainability and makes the test patterns clearer and more consistent.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Typesense vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Typesense filter expressions
- Leverage existing TypesenseFilterExpressionConverter for filter translation
- Use Typesense's DeleteDocumentsParameters for filtered deletion
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Typesense's native filtering capabilities for efficient metadata-based
deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Redis vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Redis FT.SEARCH and JSON.DEL
- Configure metadata fields properly to support numeric and tag operations
- Support both simple and complex filter expressions
- Handle Redis-specific JSON string responses in tests
- Add comprehensive integration tests for filter deletion cases
This maintains consistency with other vector store implementations while
utilizing Redis Search capabilities for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Qdrant vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Qdrant's filter API
- Leverage existing QdrantFilterExpressionConverter for filter translation
- Use Qdrant's native deleteAsync with filter capabilities
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Qdrant's native filtering capabilities for efficient metadata-based
deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Oracle vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Oracle JSON_EXISTS
- Leverage existing SqlJsonPathFilterExpressionConverter for JSON path expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
- Handle Oracle-specific JSON types in test assertions
This maintains consistency with other vector store implementations while
utilizing Oracle's JSON path capabilities for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Neo4j and OpenSearch vector stores, providing consistent deletion capabilities
with other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for Neo4j store using Cypher queries
- Add delete(Filter.Expression) implementation for OpenSearch store using query_string
- Leverage existing filter expression converters for both stores
- Use Neo4j's transaction batching for efficient large-scale deletions
- Use OpenSearch's delete_by_query API for metadata-based deletion
- Add comprehensive integration tests for both stores covering:
* Simple equality filters
* String-based filter expressions
* Complex filter expressions with multiple conditions
This maintains consistency with other vector store implementations while utilizing
store-specific features for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MongoDB Atlas vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for MongoDB Atlas store
- Leverage existing MongoDBAtlasFilterExpressionConverter for filter translation
- Use MongoTemplate's native query capabilities for deletion
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing MongoDB-specific query capabilities for efficient metadata-based deletion.
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Milvus vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for Milvus store
- Leverage existing MilvusFilterExpressionConverter for filter translation
- Use Milvus client's native delete API with filter expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Milvus-specific APIs for efficient metadata-based deletion.
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MariaDB vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for MariaDB store
- Integrate with existing MariaDBFilterExpressionConverter
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations and
enables flexible document deletion based on metadata filters.
Testcontainers 1.20.4 provides a new module for typesense with
TypesenseContainer implementation.
Signed-off-by: Eddú Meléndez <eddu.melendez@gmail.com>
