Commit Graph

324 Commits

Author SHA1 Message Date
Ilayaperumal Gopinathan
78a58a15e8 Update commons logging for the latest changes 2025-01-29 10:21:20 +00:00
Soby Chacko
bca65de6ae Add filter-based deletion to Redis vector store
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Redis vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation using Redis FT.SEARCH and JSON.DEL
- Configure metadata fields properly to support numeric and tag operations
- Support both simple and complex filter expressions
- Handle Redis-specific JSON string responses in tests
- Add comprehensive integration tests for filter deletion cases

This maintains consistency with other vector store implementations while
utilizing Redis Search capabilities for efficient metadata-based deletion.

Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
2025-01-28 12:29:42 -05:00
Soby Chacko
81d5618b3a Add filter-based deletion to Qdrant vector store
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Qdrant vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation using Qdrant's filter API
- Leverage existing QdrantFilterExpressionConverter for filter translation
- Use Qdrant's native deleteAsync with filter capabilities
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions

This maintains consistency with other vector store implementations while
utilizing Qdrant's native filtering capabilities for efficient metadata-based
deletion.

Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
2025-01-28 12:28:27 -05:00
Soby Chacko
b30713e8e3 Add filter-based deletion to Oracle vector store
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Oracle vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation using Oracle JSON_EXISTS
- Leverage existing SqlJsonPathFilterExpressionConverter for JSON path expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
- Handle Oracle-specific JSON types in test assertions

This maintains consistency with other vector store implementations while
utilizing Oracle's JSON path capabilities for efficient metadata-based deletion.

Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
2025-01-28 12:21:34 -05:00
Soby Chacko
07b905896a Add filter-based deletion to Neo4j and OpenSearch vector stores
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Neo4j and OpenSearch vector stores, providing consistent deletion capabilities
with other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation for Neo4j store using Cypher queries
- Add delete(Filter.Expression) implementation for OpenSearch store using query_string
- Leverage existing filter expression converters for both stores
- Use Neo4j's transaction batching for efficient large-scale deletions
- Use OpenSearch's delete_by_query API for metadata-based deletion
- Add comprehensive integration tests for both stores covering:
  * Simple equality filters
  * String-based filter expressions
  * Complex filter expressions with multiple conditions

This maintains consistency with other vector store implementations while utilizing
store-specific features for efficient metadata-based deletion.

Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
2025-01-28 11:55:48 -05:00
Soby Chacko
ef0bade409 Add filter-based deletion to MongoDB Atlas vector store
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MongoDB Atlas vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation for MongoDB Atlas store
- Leverage existing MongoDBAtlasFilterExpressionConverter for filter translation
- Use MongoTemplate's native query capabilities for deletion
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions

This maintains consistency with other vector store implementations while
utilizing MongoDB-specific query capabilities for efficient metadata-based deletion.
2025-01-28 11:39:14 -05:00
Ilayaperumal Gopinathan
c5dd7683b6 Fix LogAccessor formatting 2025-01-28 15:34:07 +00:00
Soby Chacko
946657fd77 Add filter-based deletion to Milvus vector store (#2127)
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Milvus vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation for Milvus store
- Leverage existing MilvusFilterExpressionConverter for filter translation
- Use Milvus client's native delete API with filter expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions

This maintains consistency with other vector store implementations while
utilizing Milvus-specific APIs for efficient metadata-based deletion.
2025-01-28 09:32:32 -05:00
Soby Chacko
fc1e90cf49 Add filter-based deletion to MariaDB vector store (#2125)
Add string-based filter deletion alongside the Filter.Expression-based deletion
for MariaDB vector store, providing consistent deletion capabilities with
other vector store implementations.

Key changes:
- Add delete(Filter.Expression) implementation for MariaDB store
- Integrate with existing MariaDBFilterExpressionConverter
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions

This maintains consistency with other vector store implementations and
enables flexible document deletion based on metadata filters.
2025-01-28 09:22:39 -05:00
Ilayaperumal Gopinathan
8303a52611 Use Apache Commons Logging
- Remove existing spring-boot-starter-logging
 - Update to use Springframework's LogAccessor to use commons logging

Resolves #2095
2025-01-28 11:00:05 +00:00
Eddú Meléndez
840304955e Use TypesenseContainer for Service Connection
Testcontainers 1.20.4 provides a new module for typesense with
TypesenseContainer implementation.

Signed-off-by: Eddú Meléndez <eddu.melendez@gmail.com>
2025-01-27 13:57:22 -05:00
Soby Chacko
f588716b41 Add filter-based deletion and string filter API to VectorStore
Add string-based filter deletion alongside the Filter.Expression-based deletion
for vector stores. This provides a more convenient API for simple filter cases
while maintaining the full flexibility of expression-based filtering.

Key changes:
- Add delete(Filter.Expression) and delete(String) methods to VectorStore
- Add default string filter implementation in VectorStore interface
- Implement filter deletion in Chroma/Elasticsearch/PgVector stores
- Add integration tests for both filter APIs across implementations

This extends vector store deletion capabilities while maintaining consistent
behavior across implementations.
2025-01-24 14:15:46 -05:00
jitokim
822576bf91 Fix typo in PgDistanceType enum class
Signed-off-by: jitokim <pigberger70@gmail.com>
2025-01-24 16:59:00 +00:00
Ilayaperumal Gopinathan
224191aea7 Revert "Remove UUID Enforcement for ID Column in PGVectorStore"
This reverts commit bac507c62a.
2025-01-24 12:40:35 +00:00
jitokim
bac507c62a Remove UUID Enforcement for ID Column in PGVectorStore
Signed-off-by: jitokim <pigberger70@gmail.com>
2025-01-24 11:48:09 +00:00
Gerrit Meier
387de95ac5 Neo4j module: remove unnecessary property removal.
Closes #1932

Signed-off-by: Gerrit Meier <meistermeier@gmail.com>
2025-01-15 19:26:45 -05:00
dafriz
95675a85f8 Set ElasticSearch size to match requested topK used in KNN search 2025-01-15 12:16:43 +00:00
Soby Chacko
329e6c0219 Remove superfluous batchingStrategy implementation in CosmosDBVectorStore 2025-01-09 11:16:11 -05:00
Ilayaperumal Gopinathan
7634342ed6 Make vector store builders extensible (#2058)
- The VectorStore builders shouldn't be final as they are meant to be extensible
 - Suppress the `FinalClass` checkstyle errors on VectorStore classes
2025-01-09 11:03:38 -05:00
Ilayaperumal Gopinathan
bd687958c7 Remove deprecated HanaCloudVectorStoreConfig 2025-01-09 15:07:16 +00:00
Soby Chacko
9844a18983 Move batching strategy to base vector store builder
Moving BatchingStrategy configuration from individual vector store implementations
to the base AbstractVectorStoreBuilder to reduce code duplication and provide consistent
batching behavior across all vector stores.

The default TokenCountBatchingStrategy is now set in the base builder class.
2025-01-09 12:30:59 +00:00
Soby Chacko
6a5326816c Addressing vector store releated checkstyle issues 2025-01-08 16:55:19 -05:00
Soby Chacko
ebe30acba2 Remove deprecations in Coherence and Qdrant vector stores 2025-01-08 16:16:12 -05:00
Ilayaperumal Gopinathan
825de11f9e Fix javadoc error
- Remove unused type from RedisVectorStore
2025-01-08 12:45:00 +00:00
Soby Chacko
eff7a80b07 Remove vector store related deprecations introduced in 1.0.0-M5 2025-01-08 09:34:17 +00:00
Ilayaperumal Gopinathan
977500f7a1 Remove deprecated classes and methods in spring-ai-core
* Remove use of Document.getContext method from spring-ai-core, use getText
* Remove deprecated ChatOptionsBuilder class
* Remove deprecated FunctionCallingOptionsBuilder class
2025-01-06 16:57:55 -05:00
Ilayaperumal Gopinathan
7fe3b389c4 Fix checkstyle errors 2025-01-02 13:43:58 +00:00
Mark Pollack
d7fe07b0f1 Next development version 2024-12-23 14:25:21 -05:00
Mark Pollack
ab022fa956 Release version 1.0.0-M5 2024-12-23 14:24:55 -05:00
Mark Pollack
4b4d48603b Change milvus package from org.sf.ai.milvus.vectorstore to org.sf.ai.vectorstore.milvus 2024-12-23 12:40:25 -05:00
Mark Pollack
3bb49c3fd6 Update class level javadoc in some vector stores to match builder signatures 2024-12-23 10:41:01 -05:00
Ilayaperumal Gopinathan
fe3db2a64d Fix the usage of deprecated methods
- Fix GemFireVectorStorePropertiesTests to remove deprecated config
   - Replace the deprecated methods in CoherenceVectorStore
2024-12-22 11:11:01 +00:00
Mark Pollack
86b155d6ff Update VectorStore docs to use inner builder class
Updated the manual configuration examples in the following docs to show
the correct usage of the inner builder class:

- azure.adoc: Show builder(searchIndexClient, embeddingModel) with all available options
- chroma.adoc: Show builder(chromaApi, embeddingModel) with collection config
- oracle.adoc: Show builder(jdbcTemplate, embeddingModel) with database options

The examples now reflect the current implementation where the builder takes
both the client and embedding model as constructor arguments.
2024-12-21 20:22:59 -05:00
Soby Chacko
f9d741dd85 Standardize builder class names in vector stores
- Rename all specific builder inner classes (PineconeBuilder, MongoDBBuilder, etc.)
  to simply Builder for consistency across vector store implementations
- Update code references to use the new standardized Builder class names

The change establishes a consistent naming convention for builder classes
across the vector store implementations, improving code uniformity.
2024-12-21 11:28:39 -05:00
Ilayaperumal Gopinathan
1dd9de422d Fix javadoc errors 2024-12-21 00:13:06 +00:00
Ilayaperumal Gopinathan
80317e2de6 Fix CassandraVectorStore
- Remove non-null session check from CassandraBuilder constructor
2024-12-20 22:34:16 +00:00
Ilayaperumal Gopinathan
a0ef1179c4 Fix PgVectorStoreWithChatMemoryAdvisorIT
- Use the PgVectorStoreBuilder to create the instance of PgVectorStore
2024-12-20 20:38:41 +00:00
Ilayaperumal Gopinathan
2804604933 Refactor SearchRequest builder methods
- Add a new Builder inner class to move all the builder methods and deprecate the existing builder methods
 - Update docs and references
2024-12-20 19:35:17 +00:00
Thomas Vitale
97e58ccfc4 Fix Qdrant failing test
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
2024-12-20 13:35:00 -05:00
Soby Chacko
1a6e79cbb9 Enhance vector stores with consistent APIs and null safety
- Replace Document.getContent() with getText() across all vector store implementations
- Fix incorrect package declarations in package-info.java files
- Make builder constructors private and implement proper builder patterns
- Add @Nullable annotations for better null safety
- Simplify PineconeVectorStore builder API by requiring essential parameters in factory method
- Make static Map fields final
- Clean up code and improve JavaDoc documentation

The changes focus on making the vector store APIs more consistent,
type-safe, and maintainable while following best practices for
builder patterns and null safety.
2024-12-20 12:19:12 -05:00
Soby Chacko
ee8bf37359 Require essential dependencies in vector store builder constructors
This commit refactors the builder pattern implementation across all VectorStore
implementations to make the EmbeddingModel a required constructor parameter
rather than an optional builder method. Key changes include:

- Move embeddingModel from being a builder method to a required constructor parameter
- Make embeddingModel final in AbstractVectorStoreBuilder
- Remove redundant validate() methods since EmbeddingModel validation now happens
  in constructor
- Update all VectorStore builder instantiations to pass EmbeddingModel in builder
  creation
- Add @Nullable annotations to appropriate methods in VectorStore interface

This change improves the API design by:
1. Enforcing that EmbeddingModel is provided at builder creation time
2. Removing the possibility of forgotten EmbeddingModel configuration
3. Simplifying the builder implementation by moving validation to construction
4. Making the dependency on EmbeddingModel more explicit in the API

Breaking Changes:
- VectorStore builders must now be created with an EmbeddingModel parameter
- The embeddingModel() builder method has been removed from all implementations
2024-12-20 12:15:58 -05:00
Mark Pollack
a4acd8e4fa Add builder pattern to PineconeVectorStore and refactor package name 2024-12-19 13:50:21 -05:00
Mark Pollack
d6d246096b reformatting 2024-12-19 13:44:38 -05:00
Soby Chacko
351dfc2f85 Pinecone vector store builder refactoring 2024-12-19 13:44:38 -05:00
Soby Chacko
d4a6e1e619 CosmosDB vector store builder refactoring 2024-12-19 13:40:18 -05:00
Soby Chacko
138e11a3e5 Add builder pattern to AzureVectorStore and refactor package name 2024-12-19 13:38:22 -05:00
Soby Chacko
8c28c6dc88 Gemfire vector store builder refactoring 2024-12-19 13:22:54 -05:00
Soby Chacko
71584e5238 Add builder pattern to HanaCloudVectorStore and refactor package name 2024-12-19 13:21:29 -05:00
Soby Chacko
7a7a914dc0 Add builder pattern to CoherenceVectorStore and refactor package name 2024-12-19 13:17:59 -05:00
Soby Chacko
08bf66cbd5 Oracle vector store builder refactoring 2024-12-19 13:12:37 -05:00