Adds getNativeClient API to VectorStore interface allowing access to the underlying native client implementation.
This change:
- Adds getNativeClient() default method to VectorStore interface returning Optional<T>
- Implements getNativeClient() in all vector store implementations exposing their respective native clients
- Adds integration tests verifying native client access for all implementations
Fixes: #2137
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Replace the existing builder implementation with a type-safe Step Builder pattern
to enforce required field initialization in a specific order. This change:
- Introduces a new builder() method that guides users through required fields
- Deprecates the old builder method with all parameters
- Makes field configuration more explicit and type-safe
- Updates all usage points to use the new builder pattern
The required fields must now be provided in sequence:
1. embeddingModel
2. apiKey
3. projectId
4. environment
5. indexName
This change improves API usability by preventing parameter confusion and ensuring
all required fields are set before optional configuration.
Implement filter-based deletion for WeaviateVectorStore by leveraging the
existing ID-based deletion mechanism.
Key changes:
- Add doDelete(Filter.Expression) implementation that delegates to the existing
ID-based deletion method
- Use similarity search with filter to find matching documents first
- Delete matched documents by their IDs
- Add integration tests for filter deletion scenarios:
* Deleting by simple equality filter
* Deleting by string-based filter expression
This maintains consistency with other vector store implementations while
working around Weaviate's limitations in direct filtered deletion.
Implement filter-based deletion for CassandraVectorStore to support more
flexible document removal based on metadata filters.
Key changes:
- Add doDelete(Filter.Expression) implementation
- Implement workaround for Cassandra's limitations in direct filtered deletion
- Fetch matching documents first, then delete by IDs
- Add integration tests for various filter deletion scenarios
- Support simple and complex filter expressions
- Includes tests for:
* Deleting by simple equality filter
* Deleting by string-based filter expression
* Deleting by complex AND filter expression
Provides consistent deletion capabilities across vector store implementations
while addressing Cassandra-specific constraints.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Pinecone vector store and improve test organization through refactoring.
Key changes:
- Add delete(Filter.Expression) implementation using two-step process:
* Search for documents matching filter
* Delete matching documents by ID
- Leverage existing search functionality to maintain filter consistency
- Extract common test patterns into helper methods
- Create reusable document factory methods for test data
- Add comprehensive integration tests for filter deletion
- Standardize test cleanup and verification patterns
This maintains consistency with other vector store implementations while
working within Pinecone's API limitations. The test refactoring improves
maintainability and makes the test patterns clearer and more consistent.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Typesense vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Typesense filter expressions
- Leverage existing TypesenseFilterExpressionConverter for filter translation
- Use Typesense's DeleteDocumentsParameters for filtered deletion
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Typesense's native filtering capabilities for efficient metadata-based
deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Redis vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Redis FT.SEARCH and JSON.DEL
- Configure metadata fields properly to support numeric and tag operations
- Support both simple and complex filter expressions
- Handle Redis-specific JSON string responses in tests
- Add comprehensive integration tests for filter deletion cases
This maintains consistency with other vector store implementations while
utilizing Redis Search capabilities for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Qdrant vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Qdrant's filter API
- Leverage existing QdrantFilterExpressionConverter for filter translation
- Use Qdrant's native deleteAsync with filter capabilities
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Qdrant's native filtering capabilities for efficient metadata-based
deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Oracle vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation using Oracle JSON_EXISTS
- Leverage existing SqlJsonPathFilterExpressionConverter for JSON path expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
- Handle Oracle-specific JSON types in test assertions
This maintains consistency with other vector store implementations while
utilizing Oracle's JSON path capabilities for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Neo4j and OpenSearch vector stores, providing consistent deletion capabilities
with other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for Neo4j store using Cypher queries
- Add delete(Filter.Expression) implementation for OpenSearch store using query_string
- Leverage existing filter expression converters for both stores
- Use Neo4j's transaction batching for efficient large-scale deletions
- Use OpenSearch's delete_by_query API for metadata-based deletion
- Add comprehensive integration tests for both stores covering:
* Simple equality filters
* String-based filter expressions
* Complex filter expressions with multiple conditions
This maintains consistency with other vector store implementations while utilizing
store-specific features for efficient metadata-based deletion.
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MongoDB Atlas vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for MongoDB Atlas store
- Leverage existing MongoDBAtlasFilterExpressionConverter for filter translation
- Use MongoTemplate's native query capabilities for deletion
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing MongoDB-specific query capabilities for efficient metadata-based deletion.
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Milvus vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for Milvus store
- Leverage existing MilvusFilterExpressionConverter for filter translation
- Use Milvus client's native delete API with filter expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Milvus-specific APIs for efficient metadata-based deletion.
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MariaDB vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for MariaDB store
- Integrate with existing MariaDBFilterExpressionConverter
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations and
enables flexible document deletion based on metadata filters.
Testcontainers 1.20.4 provides a new module for typesense with
TypesenseContainer implementation.
Signed-off-by: Eddú Meléndez <eddu.melendez@gmail.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for vector stores. This provides a more convenient API for simple filter cases
while maintaining the full flexibility of expression-based filtering.
Key changes:
- Add delete(Filter.Expression) and delete(String) methods to VectorStore
- Add default string filter implementation in VectorStore interface
- Implement filter deletion in Chroma/Elasticsearch/PgVector stores
- Add integration tests for both filter APIs across implementations
This extends vector store deletion capabilities while maintaining consistent
behavior across implementations.
Moving BatchingStrategy configuration from individual vector store implementations
to the base AbstractVectorStoreBuilder to reduce code duplication and provide consistent
batching behavior across all vector stores.
The default TokenCountBatchingStrategy is now set in the base builder class.
* Remove use of Document.getContext method from spring-ai-core, use getText
* Remove deprecated ChatOptionsBuilder class
* Remove deprecated FunctionCallingOptionsBuilder class
Updated the manual configuration examples in the following docs to show
the correct usage of the inner builder class:
- azure.adoc: Show builder(searchIndexClient, embeddingModel) with all available options
- chroma.adoc: Show builder(chromaApi, embeddingModel) with collection config
- oracle.adoc: Show builder(jdbcTemplate, embeddingModel) with database options
The examples now reflect the current implementation where the builder takes
both the client and embedding model as constructor arguments.
- Rename all specific builder inner classes (PineconeBuilder, MongoDBBuilder, etc.)
to simply Builder for consistency across vector store implementations
- Update code references to use the new standardized Builder class names
The change establishes a consistent naming convention for builder classes
across the vector store implementations, improving code uniformity.
- Replace Document.getContent() with getText() across all vector store implementations
- Fix incorrect package declarations in package-info.java files
- Make builder constructors private and implement proper builder patterns
- Add @Nullable annotations for better null safety
- Simplify PineconeVectorStore builder API by requiring essential parameters in factory method
- Make static Map fields final
- Clean up code and improve JavaDoc documentation
The changes focus on making the vector store APIs more consistent,
type-safe, and maintainable while following best practices for
builder patterns and null safety.
This commit refactors the builder pattern implementation across all VectorStore
implementations to make the EmbeddingModel a required constructor parameter
rather than an optional builder method. Key changes include:
- Move embeddingModel from being a builder method to a required constructor parameter
- Make embeddingModel final in AbstractVectorStoreBuilder
- Remove redundant validate() methods since EmbeddingModel validation now happens
in constructor
- Update all VectorStore builder instantiations to pass EmbeddingModel in builder
creation
- Add @Nullable annotations to appropriate methods in VectorStore interface
This change improves the API design by:
1. Enforcing that EmbeddingModel is provided at builder creation time
2. Removing the possibility of forgotten EmbeddingModel configuration
3. Simplifying the builder implementation by moving validation to construction
4. Making the dependency on EmbeddingModel more explicit in the API
Breaking Changes:
- VectorStore builders must now be created with an EmbeddingModel parameter
- The embeddingModel() builder method has been removed from all implementations
