- Remove deprecations from models, vector stores and usage
- Deprecations from FunctionCallback and ObservationContext/Convention will be in a separate PR
Models updates
- Remove AbstractToolCallSupport from the models which use ToolCallingManager
- Remove deprecated constructors and their usage
- Remove FunctionCallbackResolver and FunctionCallbacks usage in the models
- Add back deprecations for VectorStoreChatMemoryAdvisor until builder is fixed
- Update OpenAiPaymentTransactionIT to use ToolCallbackResolver in config
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Updated `SearchRequest.java` to make the class non-final.
- Added `MilvusSearchRequest` with specific Milvus parameters such as `nativeExpression` and `searchParamsJson`.
- Modified `doSimilaritySearch` method in `MilvusVectorStore` to handle these new fields from `MilvusSearchRequest`.
Add unit tests for MilvusVectorStore and MilvusSearchRequest
Introduce comprehensive unit tests to validate the functionality of MilvusVectorStore and MilvusSearchRequest, including scenarios for native and filter expressions. Refactor MilvusVectorStore to improve filter expression handling by introducing a helper method for converted expressions.
Add detailed documentation for MilvusSearchRequest usage
Introduced sections explaining MilvusSearchRequest's parameters, `nativeExpression`, and `searchParamsJson`, with examples for enhanced clarity. This update provides guidance on leveraging Milvus-specific features for precise filtering and optimal search performance.
Signed-off-by: waileong <wai_leong1015@hotmail.com>
- Upgrade Milvus java sdk to 2.5.4
- Remove use of fastjson
- Use com.google.gson.JsonObject as Milvus SDK replaces fastjson with gson
- Change SearchResult RowRecord similarity metric name to score
- Milvus SDK 2.5.4 uses "score" instead of "distance"
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Extract common vector store delete tests to base class
This commit extracts shared delete operation tests into a reusable BaseVectorStoreTests class.
This reduces code duplication and provides a consistent test suite for delete operations across
different vector store implementations. The base class includes tests for:
Deleting by ID
Deleting by filter expressions
Deleting by string filter expressions
Most of the vector store implementation now extends this base class and inherits these
common tests while maintaining the ability to add vector store specific tests.
Adding javadoc
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
- Change VectorStore.delete() and related implementations to return void instead of Optional<Boolean>
- Remove unnecessary boolean return values and success status checks across all vector store implementations
- Clean up tests by removing redundant assertions
- Implementations continue using runtime exceptions for error signaling
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Adds getNativeClient API to VectorStore interface allowing access to the underlying native client implementation.
This change:
- Adds getNativeClient() default method to VectorStore interface returning Optional<T>
- Implements getNativeClient() in all vector store implementations exposing their respective native clients
- Adds integration tests verifying native client access for all implementations
Fixes: #2137
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for Milvus vector store, providing consistent deletion capabilities with
other vector store implementations.
Key changes:
- Add delete(Filter.Expression) implementation for Milvus store
- Leverage existing MilvusFilterExpressionConverter for filter translation
- Use Milvus client's native delete API with filter expressions
- Add comprehensive integration tests for filter deletion
- Support both simple and complex filter expressions
This maintains consistency with other vector store implementations while
utilizing Milvus-specific APIs for efficient metadata-based deletion.
Moving BatchingStrategy configuration from individual vector store implementations
to the base AbstractVectorStoreBuilder to reduce code duplication and provide consistent
batching behavior across all vector stores.
The default TokenCountBatchingStrategy is now set in the base builder class.
* Remove use of Document.getContext method from spring-ai-core, use getText
* Remove deprecated ChatOptionsBuilder class
* Remove deprecated FunctionCallingOptionsBuilder class
- Rename all specific builder inner classes (PineconeBuilder, MongoDBBuilder, etc.)
to simply Builder for consistency across vector store implementations
- Update code references to use the new standardized Builder class names
The change establishes a consistent naming convention for builder classes
across the vector store implementations, improving code uniformity.
- Replace Document.getContent() with getText() across all vector store implementations
- Fix incorrect package declarations in package-info.java files
- Make builder constructors private and implement proper builder patterns
- Add @Nullable annotations for better null safety
- Simplify PineconeVectorStore builder API by requiring essential parameters in factory method
- Make static Map fields final
- Clean up code and improve JavaDoc documentation
The changes focus on making the vector store APIs more consistent,
type-safe, and maintainable while following best practices for
builder patterns and null safety.
This commit refactors the builder pattern implementation across all VectorStore
implementations to make the EmbeddingModel a required constructor parameter
rather than an optional builder method. Key changes include:
- Move embeddingModel from being a builder method to a required constructor parameter
- Make embeddingModel final in AbstractVectorStoreBuilder
- Remove redundant validate() methods since EmbeddingModel validation now happens
in constructor
- Update all VectorStore builder instantiations to pass EmbeddingModel in builder
creation
- Add @Nullable annotations to appropriate methods in VectorStore interface
This change improves the API design by:
1. Enforcing that EmbeddingModel is provided at builder creation time
2. Removing the possibility of forgotten EmbeddingModel configuration
3. Simplifying the builder implementation by moving validation to construction
4. Making the dependency on EmbeddingModel more explicit in the API
Breaking Changes:
- VectorStore builders must now be created with an EmbeddingModel parameter
- The embeddingModel() builder method has been removed from all implementations
This commit deprecates builder methods with the "with" prefix in the VectorStoreObservationContext class
and introduces new methods without the prefix for a cleaner API. For example:
withCollectionName() → collectionName()
withDimensions() → dimensions()
withNamespace() → namespace()
The old methods are marked as deprecated for maintaining backward compatibility.
All vector store implementations have been updated to use the new method names.
Introduces a fluent builder API to improve configuration readability and
type safety when creating MilvusVectorStore instances. This replaces the
existing configuration object approach which was less intuitive and harder
to maintain.
The builder pattern provides better encapsulation of configuration logic
and validation, while maintaining backward compatibility through a
deprecated config class. This change makes the codebase more maintainable
and the API more discoverable for users.
Key changes:
- Replace configuration object with fluent builder pattern
- Move Milvus-related classes to dedicated milvus package
- Deprecate MilvusVectorStoreConfig in favor of builder
- Update constructor to use builder internally
- Maintain backward compatibility with deprecated config
- Add comprehensive builder methods with validation
The Document class previously allowed multiple media entries while also having a
text field, leading to ambiguity in content handling. This change enforces a
clear separation between text and media documents to prevent content type
confusion and simplify document processing.
A Document now must contain either text content or a single media entry, but
never both. This aligns with the class's primary use in ETL pipelines where
clear content type boundaries are essential for proper embedding generation and
vector database storage.
Additional architectural changes:
- Document now implements a cleaner API by removing deprecated methods
- Removed MediaContent interface implementation from Document class
- Document.getMedia() now returns a single Media object instead of Collection
- Removed EMPTY_TEXT constant in favor of proper null handling
- Constructor signatures simplified and streamlined
- Builder pattern improved to enforce single content type constraint
The breaking changes include:
- Media is now a single entry instead of a collection
- Content field renamed to text for clarity
- Removed support for mixed content types
- Simplified builder API to prevent ambiguous construction
Prefer using text-related methods over deprecated content methods to
better reflect the actual content type being handled and improve API clarity.
- Since the Document object's reference to the `embedding` is deprecated and will be removed, the VectorStore implementations require a way to store the embedding of the corresponding Document objects
- One way to fix this is, to have the EmbeddingModel#embed to return the embeddings in the same order as that of the Documents passed to it.
- Since both the Document and embedding collections use the List object, their iteration operation will make sure to keep them in line with the same order.
- A fix is required to preserve the order when batching strategy is applied.
- Updated the Javadoc for BatchingStrategy
- Fixed the Document List order in TokenCountBatchingStrategy
- Refactored the vector store implementations to update this change
Resolves #GH-1826
Document
* Introduced “score” attribute in Document API. It stores the similarity score.
* Consolidate “distance” metadata for Documents. It stores the distance measurement.
* Adopted prefix-less naming convention in Document.Builder and deprecated old methods.
* Deprecated the many overloaded Document constructors in favour of Document.Builder.
Vector Stores
* Every vector store implementation now configures a “score” attribute with the similarity score of the Document embedding. It also includes the “distance” metadata with the distance measurement.
* Fixed error in Elasticsearch where distance and similarity were mixed up.
* Added missing integration tests for SimpleVectorStore.
* The Azure Vector Store and HanaDB Vector Store do not include those measurements because the product documentation do not include information about how the similarity score is returned, and without access to the cloud products I could not verify that via debugging.
* Improved tests to actually assert the result of the similarity search based on the returned score.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Add new Maven profile 'ci-fast-integration-tests' for running selective ITs
- Remove redundant vector store skip flags from properties section
- Update maven-failsafe-plugin to version 3.5.2
- Configure test exclusions for various components:
- Most model integration tests (Anthropic and OpenAI)
- Most vector store tests (except PgVector and Chroma)
- Most auto-configuration tests
- All test containers and docker compose tests
- AI evaluation tests
- Convert the docker-compose tests into ITs
- Convert the testcontainers tests into ITs
- Updated README.md
- Explain the new profile and also the new integration tests repo
- Describe ways to run integration tests for specific modules
- Add badge for https://github.com/spring-projects/spring-ai-integration-tests
- Add maven properties for all vector stores such as
skip.vectorstore.azure-cosmos-db to control IT test execution
- Chroma and PGVector IT tests are enabled by default
- Docker Compose and Testcontainers module ITs are skipped by default
- Add parallel job to run docker-compose and testcontainers ITs
Replace traditional switch statements with Java 14 switch expressions across
vector store filter converters and related components. This change improves
code quality in our filter expression handling for Azure, Milvus, Redis,
Typesense and Weaviate implementations.
The switch expressions eliminate fall-through behavior, enforce exhaustive
pattern matching at compile time, and provide a more direct way to return
values. This makes the filter conversion logic more robust and maintainable.
- Milvus requires "not in" operator syntax instead of "nin" for filtering.
- See https://milvus.io/docs/boolean.md
- Updated test and added additional test coverage
- Disable project-wide Checkstyle checks to unblock development
- Add documentation for enabling Checkstyle locally
- Fix remaining checkstyle violations in current codebase
Fixes#1669
- The doSimilaritySearch method did not pass the databaseName parameter in milvus 2.3.4 and before, resulting in the use of the default database; The current milvus upgrade to 2.3.5 supports passing the databaseName parameter
- Update doSimilaritySearch to set the database name from the Milvus configuration
When using a non-default database, MilvusVectorStore's doDelete method wasn't
passing the specified databaseName in DeleteParam, causing searches to fall back
to the default database where collections couldn't be found. Added explicit
databaseName parameter to fix this issue.
- Add databaseName to DeleteParam builder
- Upgrade milvus-sdk from 2.3.4 to 2.3.5
* GH-513 fix: Support custom field names for Milvus VectorStore collection
- Add configuration properties to override the default field names for doc_id, content, metadata and embedding
- Add support to allow auto-id when enabled
- Add tests
Resolves#513
* Fix package modifier
Adds @EnabledIfEnvironmentVariable annotation to integration tests that
use OpenAI embeddings. Tests will be skipped if OPENAI_API_KEY is not set,
making the build process more reliable for contributors who don't have
access to OpenAI services.
