- Fix the embedding dimension configuration for opensearch client indices mapping
The dimension config is obtained by the underlying embedding model's dimension
- Updated OpenSearch mapping JSON to include dynamic embedding dimension.
- Changed default mapping for OpenSearch vector store to use generic dimension placeholder.
- Fixed OpenSearchVectorStoreAutoConfiguration to fallback to new default mapping if no custom mapping is provided.
- Added test for verifying mapping configuration with OpenSearch vector store.
- Added test dependencies for Ollama integration.
- Refactored OpenSearch integration tests to use new mapping and dimension logic.
Add tests
- Verify the mappingJson field is correctly set - verify the override works fine
- Add integration tests with Ollama embedding model
Resolves#1589
- Add maven properties for all vector stores such as
skip.vectorstore.azure-cosmos-db to control IT test execution
- Chroma and PGVector IT tests are enabled by default
- Docker Compose and Testcontainers module ITs are skipped by default
- Add parallel job to run docker-compose and testcontainers ITs
- Disable project-wide Checkstyle checks to unblock development
- Add documentation for enabling Checkstyle locally
- Fix remaining checkstyle violations in current codebase
Fixes#1669
- Resolved issue where index name was not being sent during
similaritySearch
- Updated similaritySearch method to include index in the SearchRequest
- Implemented test to verify documents can be added and retrieved from
two different indices using separate OpenSearchVectorStore instances
- Ensured similarity search results are correctly returned for the
respective indices
Fixes#885
* Unify image definition for vector stores in vector-store modules
* Unify image definition for vector stores in spring-ai-testcontainers module
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Vector store observations support several key-value pairs, coming from the Spring AI abstractions. Currently, whenever a value is not available (either because not configured by the user or not supported by the vector store provider), span/metrics attributes are generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and non-compliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for vector store observations to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Apply batching when adding Documents to the following vector stores:
- Azure vector store
- Cassandra
- MongoDB Atlas
- OpenSearch
- Oracle
- Pinecone
This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.
Related to #1261
* Consolidate usage of “db.collection.name” attribute to track table name, collection name, index name, document name, or whatever concept a vector database uses to store data. Removed “db.index” that was use sometimes instead of “db.collection.name”. This usage is in line with the OpenTelemetry Semantic Conventions.
* Configure query response content to be included as a “span event” instead of a “span attribute” if the backend system supports that, similar to how we do for the model observations.
* Structure vector store observation attributes in dedicated enums, including one for the Spring AI Kinds to avoid hard-coding the same value in a lot of places. This follows the OpenTelemetry Semantic Conventions as much as possible. Also, adopt Spring usual non-null-by-default strategy as much as possible.
* Align vector store conventions to the chat model ones, and follow alphabetical order for values. This is particularly useful for the convention classes, for which the Micrometer performance of exporting telemetry data improves when key values are added already sorted to the context.
* Fix flaky test in Mistral AI.
* Improve Qdrant integration tests.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Change default schema initialization of vector stores from `true` to `false.`
Users need to explicitly opt-in for schema initialization by setting the
`initialize-schema` property on the corresponding vector store.
* Update integration tests
* Update docs
Fixes#907
- implement OpensSearchVectorStore
- add opensearch auto-configuration and boot starter
- add documentation for OpenSearch VectorStore
- add bom dependecies
- align with to new Spirng AI API
