- Fix parameter access syntax in setNodeVectorProperty DDL statement
- Define DEFAULT_TRANSACTION_SIZE for the hardcoded batch size in
the delete query
Fixes#1623
Signed-off-by: jitokim <pigberger70@gmail.com>
Integrates Oracle Coherence 24.09+ as a vector store backend for Spring AI. The
implementation provides:
- Support for vector similarity search with configurable distance metrics (Cosine,
Inner Product, L2)
- Multiple indexing options including HNSW and Binary Quantization indexes
- Filter expression evaluation for metadata-based filtering
- Vector normalization capabilities
- Comprehensive test coverage including integration tests
The implementation includes:
- CoherenceVectorStore main implementation
- CoherenceFilterExpressionConverter for Spring AI to Coherence filter conversion
- Auto-configuration support via spring-boot-starter
- Documentation and usage examples
Requires Oracle Coherence 24.09 or later.
Replace traditional switch statements with Java 14 switch expressions across
vector store filter converters and related components. This change improves
code quality in our filter expression handling for Azure, Milvus, Redis,
Typesense and Weaviate implementations.
The switch expressions eliminate fall-through behavior, enforce exhaustive
pattern matching at compile time, and provide a more direct way to return
values. This makes the filter conversion logic more robust and maintainable.
- Milvus requires "not in" operator syntax instead of "nin" for filtering.
- See https://milvus.io/docs/boolean.md
- Updated test and added additional test coverage
- Disable project-wide Checkstyle checks to unblock development
- Add documentation for enabling Checkstyle locally
- Fix remaining checkstyle violations in current codebase
Fixes#1669
- The doSimilaritySearch method did not pass the databaseName parameter in milvus 2.3.4 and before, resulting in the use of the default database; The current milvus upgrade to 2.3.5 supports passing the databaseName parameter
- Update doSimilaritySearch to set the database name from the Milvus configuration
This change uses `session.executeWrite` and `session.executeRead` to interact with the database.
Those methods use build-in retries for several Neo4j specific error states.
Doing will improve the overall user experience in this case as the no other external retry mechanism should be used.
Doing so is fine, as the Neo4j vector store is not subject to Springs general transaction management.
The system text advise prompt is only joined with the value of the system text if it's non-null and non-empty.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
When using a non-default database, MilvusVectorStore's doDelete method wasn't
passing the specified databaseName in DeleteParam, causing searches to fall back
to the default database where collections couldn't be found. Added explicit
databaseName parameter to fix this issue.
- Add databaseName to DeleteParam builder
- Upgrade milvus-sdk from 2.3.4 to 2.3.5
- Define the version for Azure Cosmos DB
- Add dependencies for the azure-cosmos-db-store module and boot starter
- Include the artifacts in spring-ai-bom
Signed-off-by: jitokim <pigberger70@gmail.com>
* GH-513 fix: Support custom field names for Milvus VectorStore collection
- Add configuration properties to override the default field names for doc_id, content, metadata and embedding
- Add support to allow auto-id when enabled
- Add tests
Resolves#513
* Fix package modifier
Adds @EnabledIfEnvironmentVariable annotation to integration tests that
use OpenAI embeddings. Tests will be skipped if OPENAI_API_KEY is not set,
making the build process more reliable for contributors who don't have
access to OpenAI services.
ObjectMapper instantiation is costly, so unless its usage
is one-shot, it is better to create a reusable instance
for upcoming usage.
Also, before this commit, serialization of most Kotlin
classes was not supported due to the lack of proper
Jackson KotlinModule detection.
This commit:
- Avoids per invocation ObjectMapper instantiation when
relevant
- Automatically detects and enables well-known Jackson
modules including the Kotlin one
- Removes org.springframework.ai.vectorstore.JsonUtils
which looks not needed anymore
More optimizations are possible like reusing more
ObjectMapper instances, but this could introduce more breaking
changes so this commit intends to be a good first step.
Kotlin tests will be provided in a follow-up commit.
Additional changes:
- Update ModelOptionsUtils to use JacksonUtils.instantiateAvailableModules()
- Add missing license headers
- Add missing author Javadoc comments
- Implement core vector store module for CosmosDB integration
- Add Spring Boot auto-configuration capabilities
- Integrate batch processing strategy for optimized operations
- Include comprehensive tests for core and auto-config modules
- Add reference docs for the CosmosDB vector store support
- Remove unnecessary spring-web dependencies
- Update third-party library versions
- Refactor API classes to use consistent header handling
- Remove ApiUtils class and inline its functionality
- Adjust RestClient and WebClient builder usage in autoconfiguration
- Replace direct RestClient.Builder injections with ObjectProvider<RestClient.Builder>
and WebClient.Builder injections with ObjectProvider<WebClient.Builder>
- Update ChromaVectorStoreAutoConfiguration to use ObjectProvider
- Rename MongoDbAtlasLocalContainerConnectionDetailsFactoryTest to IT
- Switch spring-ai-chroma-store dependency from spring-web to spring-webflux
- Simplify ChromaApi constructor by using method reference for default headers
- Adjust import order
Resolves#1066Resolves#524
- Improve ChromaVectorStore to throw exception on missing collection with disabled schema
- Add test case to verify exception is thrown when collection doesn't exist and schema init is false
- Correctly set Chroma collection id when initializeSchema set to false
Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
- Resolved issue where index name was not being sent during
similaritySearch
- Updated similaritySearch method to include index in the SearchRequest
- Implemented test to verify documents can be added and retrieved from
two different indices using separate OpenSearchVectorStore instances
- Ensured similarity search results are correctly returned for the
respective indices
Fixes#885
* Unify image definition for vector stores in vector-store modules
* Unify image definition for vector stores in spring-ai-testcontainers module
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Vector store observations support several key-value pairs, coming from the Spring AI abstractions. Currently, whenever a value is not available (either because not configured by the user or not supported by the vector store provider), span/metrics attributes are generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and non-compliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for vector store observations to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Testcontainers 1.20.2 offers `MongoDBAtlasLocalContainer`. Previous
image used during tests was pulling images in every execution. The
new image is cached locally and can be executed offline.
This commit introduces a major overhaul of the advisor system in Spring AI,
improving modularity, type safety, and consistency
Core Changes:
- Replace RequestAdvisor and ResponseAdvisor with CallAroundAdvisor and StreamAroundAdvisor
- Introduce AdvisedRequest and AdvisedResponse classes for better encapsulation
- Deprecate RequestResponseAdvisor in favor of new advisor types
- Remove AdvisorObservableHelper class
Advisor Implementation Updates:
- Update AbstractChatMemoryAdvisor, MessageChatMemoryAdvisor, PromptChatMemoryAdvisor,
QuestionAnswerAdvisor, SafeGuardAroundAdvisor, SimpleLoggerAdvisor, and
VectorStoreChatMemoryAdvisor to implement new advisor interfaces
- Remove CacheAroundAdvisor (functionality likely moved elsewhere)
- Make CallAroundAdvisor and StreamAroundAdvisor extend Ordered interface
Client and Chain Management:
- Modify DefaultChatClient to use new advisor chain approach
- Refactor DefaultAroundAdvisorChain for better ordering and observation
- Implement builder pattern for advisor chain construction in DefaultChatClient
- Separate call and stream advisors in DefaultAroundAdvisorChain
Observation and Context Handling:
- Update observation conventions and context handling in advisors
- Add order field to AdvisorObservationContext
- Modify DefaultAdvisorObservationConvention to include order in high cardinality key values
Testing and Integration:
- Refactor ChatClientAdvisorTests and add new AdvisorsTests
- Update integration tests to reflect new advisor structure
- Enhance AdvisorsTests to verify correct advisor execution order
New Features:
- Generalize the Protect From Blocking functionality across all advisors
- Add (experimental) Re2 advisor to enhance reasoning capabilities of LLMs
- Add disabled Re2 test in OpenAiChatClientIT
Documentation:
- Add Advisors documentation
- Enhance advisors documentation with order explanation and Re2 example
Advisor Ordering:
- Introduce Advisor constants for precedence ordering
- Update AbstractChatMemoryAdvisor to use new precedence constant
- Improve advisor ordering and management in DefaultAroundAdvisorChain.Builder
- Remove redundant reordering logic from DefaultAroundAdvisorChain
These changes aim to provide a more flexible and powerful advisor system,
allowing for easier implementation of complex AI-driven interactions
Co-authored-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
Resolves https://github.com/spring-projects/spring-ai/issues/1199
- Implement configurable maxDocumentBatchSize to prevent insert timeouts
when adding large numbers of documents
- Update PgVectorStore to process document inserts in controlled batches
- Add maxDocumentBatchSize property to PgVectorStoreProperties
- Update PgVectorStoreAutoConfiguration to use the new batching property
- Add tests to verify batching behavior and performance
This change addresses the issue of PgVectorStore inserts timing out due to
large document volumes. By introducing configurable batching, users can now
control the insert process to avoid timeouts while maintaining performance
and reducing memory overhead for large-scale document additions.
Apply batching when adding Documents to the following vector stores:
- Azure vector store
- Cassandra
- MongoDB Atlas
- OpenSearch
- Oracle
- Pinecone
This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.
Related to #1261
Apply batching when adding Documents to the following vector stores:
- Chroma
- ElasticSearch
- Neo4j
- Qdrant
- Redis
- Typesense
- Weaviate
This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.
Related to #1261
