- Introduce toolContext to ChatClient, DefaultChatClient, and AdvisedRequest
- Add methods to set and manage toolContext via the FunctionCallingOptions
- Update tests to include toolContext in relevant scenarios
- Implement toolContext handling in function calling options
This enhancement allows users to specify a JSON Pointer to extract
specific parts of a JSON document when reading. Key changes include:
- New get(String pointer) method in JsonReader class
- Updated JsonReaderTests with pointer-based extraction tests
- Added documentation for JSON Pointer usage in etl-pipeline.adoc
This feature enables more flexible parsing of complex JSON structures,
allowing users to easily target nested data for extraction.
- Improve ChromaVectorStore to throw exception on missing collection with disabled schema
- Add test case to verify exception is thrown when collection doesn't exist and schema init is false
- Correctly set Chroma collection id when initializeSchema set to false
Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
This change allows the ElasticsearchVectorStoreAutoConfiguration
class to be excluded, enabling custom configurations. It brings
consistency with other vector store auto-configuration classes.
- Resolved issue where index name was not being sent during
similaritySearch
- Updated similaritySearch method to include index in the SearchRequest
- Implemented test to verify documents can be added and retrieved from
two different indices using separate OpenSearchVectorStore instances
- Ensured similarity search results are correctly returned for the
respective indices
Fixes#885
* Update attributes
* Add more information on optional attributes and risks
* Include documentation for events.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Unify image definition for vector stores in vector-store modules
* Unify image definition for vector stores in spring-ai-testcontainers module
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Make ChatClient and Advisor observation logic null-safe
* Simplify naming for Advisor observations
* Include high-cardinality attributes only if a value is present
* Fix condition to include system test to chat client observations
* Add Advisor order information to context
* Streamline usage of enums and utils to reduce hard-coded/duplications
* Fix pending Chroma integration test
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Unlike other metrics implementations supported by Micrometer, when using the Prometheus integration, all metric attributes are supposed to have a value, or else the related metrics are dropped.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Vector store observations support several key-value pairs, coming from the Spring AI abstractions. Currently, whenever a value is not available (either because not configured by the user or not supported by the vector store provider), span/metrics attributes are generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and non-compliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for vector store observations to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Chat model observations support several key-value pairs, coming from the Spring AI abstractions.
Currently, whenever a value is not available (either because not configured by the user or not supported by the model provider), a span/metrics is generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and incompliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for chat model observation to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit removes the duplicate entry of
'spring.ai.vectorstore.elasticsearch.initialize-schema' property from
the Elasticsearch Vector Store documentation.
This commit adds support for tool context in various chat options classes across
different AI model implementations and enhances function calling capabilities.
The tool context allows passing additional contextual information to function callbacks.
- Add toolContext field to chat options classes
- Update builder classes to support setting toolContext
- Enhance FunctionCallback interface to support context-aware function calls
- Update AbstractFunctionCallback to implement BiFunction instead of Function
- Modify FunctionCallbackWrapper to support both Function and BiFunction and
to use the new SchemaType location
- Add support for BiFunction in TypeResolverHelper
- Update ChatClient interface and DefaultChatClient implementation to support
new function calling methods with Function, BiFunction and FunctionCallback arguments
- Refactor AbstractToolCallSupport to pass tool context to function execution
- Update all affected <Model>ChatOptions with tool context support
- Simplify OpenAiChatClientMultipleFunctionCallsIT test
- Add tests for function calling with tool context
- Add new test cases for function callbacks with context in various integration tests
- Modify existing tests to incorporate new context-aware function calling capabilities
- Add docs in in openai function calling
Resolves#864, #1303, #991
Implement observability for imperative calls in AzureOpenAiChatModel:
* Integrate ObservationRegistry to track metrics and events
* Create observation context and define conventions for consistency
* Add integration test to verify chat model observations
* Update AiProvider enum with new AZURE_OPENAI entry
Add support for PortableFunctionCallingOptions across AI models
- Modify FunctionCallingOptions interface to extend ChatOptions for better integration
- Refactor option handling in chat models to accommodate both ChatOptions and FunctionCallingOptions
- Implement handling of FunctionCallingOptions in Anthropic, Azure OpenAI,
MistralAI, Ollama, OpenAI, VertexAI Gemini, and other models
- Update existing function calling tests to use new FunctionCallingOptions.
Resolves#624
- Useful for not Web apps.
- Introduce ErrorLoggingObservationHandler for tracing errors across various AI contexts
- Add error logging configuration option to ChatObservationProperties
- Include ErrorLoggingObservationHandler bean in ChatObservationAutoConfiguration
- Update docs
Resolves#1440
This commit addresses the NPE issue in TextReader's source metadata
handling. It introduces a new method getResourceIdentifier() to
robustly extract identifiers from various Resource types.
The fix ensures that:
1. Filename is used if available
2. Falls back to URI, then URL if filename is not present
3. Uses resource description as a last resort
Additionally, the commit includes updated tests to verify the behavior
with different Resource types, particularly ByteArrayResource
This change prevents NPEs when dealing with Resources that lack
certain properties, improving the overall reliability of TextReader.
Fixes https://github.com/spring-projects/spring-ai/issues/1386
This commit adds support for new Meta Llama 3.1 and 3.2 instruct models
to the LlamaChatBedrockApi enum. It includes model IDs for:
Llama 3.1: 8B, 70B, 405B
Llama 3.2: 1B, 3B, 11B, 90B
These additions allow users to specify the latest Llama models when
using the Bedrock API. The commit also updates the class Javadoc with
a link to AWS documentation for model IDs and bumps the @since version
to 1.0.0.
- Switch to OpenAIAsyncClient for streaming operations
- Modify AzureOpenAiChatModel constructor to accept OpenAIClientBuilder
- Update getChatCompletionsStream to use non-blocking async client
- Refactor related classes and tests to support OpenAIClientBuilder
- Revise AzureOpenAiAutoConfiguration to provide OpenAIClientBuilder
- Add AzureOpenAiChatClientTest to verify streaming functionality
- Adjust existing tests for compatibility with OpenAIClientBuilder
Resolves https://github.com/spring-projects/spring-ai/issues/981
This change improves support for asynchronous streaming operations
in the AzureOpenAiChatModel, addressing potential issues in reactive environments.
Implement service connection support for MongoDB Atlas using both
TestContainers and Docker Compose. This change enables easier
integration testing and local development with MongoDB Atlas.
- Leverage TestContainers 1.20.2 which introduces MongoDBAtlasLocalContainer
- Add TestContainers support using MongoDBAtlasLocalContainer
- Implement Docker Compose configuration for MongoDB Atlas
- Create connection details factories for both TestContainers and Docker Compose
- Update dependency management for MongoDB Atlas integration
- Add integration tests for both TestContainers and Docker Compose setups
- Update documentation to include MongoDB Atlas support
Testcontainers 1.20.2 offers `MongoDBAtlasLocalContainer`. Previous
image used during tests was pulling images in every execution. The
new image is cached locally and can be executed offline.
Resolves https://github.com/spring-projects/spring-ai/issues/832
Introduces retry functionality to VertexAI embedding and
chat models, enhancing their resilience against transient failures.
It also corrects a typo in the VertexAiEmbeddingConnectionDetails
class name.
Key changes:
* Add RetryTemplate to VertexAiTextEmbeddingModel and VertexAiGeminiChatModel
* Introduce spring-ai-retry dependency
* Refactor code to support retry logic
* Update auto-configuration classes to incorporate retry functionality
* Fix typo in VertexAiEmbeddingConnectionDetails class name
remove extraneous commented out code
Add missing copyright headers, author etc.
This commit introduces a major overhaul of the advisor system in Spring AI,
improving modularity, type safety, and consistency
Core Changes:
- Replace RequestAdvisor and ResponseAdvisor with CallAroundAdvisor and StreamAroundAdvisor
- Introduce AdvisedRequest and AdvisedResponse classes for better encapsulation
- Deprecate RequestResponseAdvisor in favor of new advisor types
- Remove AdvisorObservableHelper class
Advisor Implementation Updates:
- Update AbstractChatMemoryAdvisor, MessageChatMemoryAdvisor, PromptChatMemoryAdvisor,
QuestionAnswerAdvisor, SafeGuardAroundAdvisor, SimpleLoggerAdvisor, and
VectorStoreChatMemoryAdvisor to implement new advisor interfaces
- Remove CacheAroundAdvisor (functionality likely moved elsewhere)
- Make CallAroundAdvisor and StreamAroundAdvisor extend Ordered interface
Client and Chain Management:
- Modify DefaultChatClient to use new advisor chain approach
- Refactor DefaultAroundAdvisorChain for better ordering and observation
- Implement builder pattern for advisor chain construction in DefaultChatClient
- Separate call and stream advisors in DefaultAroundAdvisorChain
Observation and Context Handling:
- Update observation conventions and context handling in advisors
- Add order field to AdvisorObservationContext
- Modify DefaultAdvisorObservationConvention to include order in high cardinality key values
Testing and Integration:
- Refactor ChatClientAdvisorTests and add new AdvisorsTests
- Update integration tests to reflect new advisor structure
- Enhance AdvisorsTests to verify correct advisor execution order
New Features:
- Generalize the Protect From Blocking functionality across all advisors
- Add (experimental) Re2 advisor to enhance reasoning capabilities of LLMs
- Add disabled Re2 test in OpenAiChatClientIT
Documentation:
- Add Advisors documentation
- Enhance advisors documentation with order explanation and Re2 example
Advisor Ordering:
- Introduce Advisor constants for precedence ordering
- Update AbstractChatMemoryAdvisor to use new precedence constant
- Improve advisor ordering and management in DefaultAroundAdvisorChain.Builder
- Remove redundant reordering logic from DefaultAroundAdvisorChain
These changes aim to provide a more flexible and powerful advisor system,
allowing for easier implementation of complex AI-driven interactions
Co-authored-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
- Add new option 'maxCompletionTokens' to spring.ai.openai.chat.options
- Mark 'maxTokens' as deprecated
- Update documentation to reflect these changes in OpenAI chat configuration
Related to #1411 and #1412
An upper bound for the number of tokens that can be generated for a completion,
including visible output tokens and reasoning tokens.
Replaces max_tokens field which is now deprecated.
This commit adds comprehensive documentation for the BatchingStrategy
in vector stores and enhances the TokenCountBatchingStrategy class.
Key changes:
- Explain batching necessity due to embedding model thresholds
- Describe BatchingStrategy interface and its purpose
- Detail TokenCountBatchingStrategy default implementation
- Provide guidance on using and customizing batching strategies
- Note pre-configured vector stores with default strategy
- Add new constructor for custom TokenCountEstimator in
TokenCountBatchingStrategy
- Implement null checks with Spring's Assert utility
- Update docs with new customization options and code examples
This is related to https://github.com/spring-projects/spring-ai/issues/889
This commit addresses a bug where certain fields were being made
indirectly mandatory due to Assert.notNull checks in the Builder's
with* methods. Specifically:
- Removed Assert.notNull checks from withResponseFormat, withSeed,
withLogprobs, withTopLogprobs, and withEnhancements methods.
These checks were causing exceptions in AzureOpenAiChatModel.getDefaultOptions
when not all fields were set, leading to failures in methods like
ChatClient.create, even when using the AzureOpenAiChatModel constructor
with OpenAIClient.
The removal of these checks aligns with the @JsonInclude(Include.NON_NULL)
annotation on AzureOpenAiChatOptions, which already ignores null options.
This change maintains the intended flexibility while preventing unintended
mandatory requirements.
This commit introduces support for Oracle Cloud Infrastructure (OCI)
GenAI embedding models in Spring AI. It includes:
* New OCIEmbeddingModel class for interacting with OCI GenAI API
* Auto-configuration for easy setup and integration
* Properties for configuring OCI connection and embedding options
* Documentation updates explaining usage and configuration
* Integration tests to verify functionality
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Correct the OllamaEmbeddingModel initialization and options syntax in
the ollama-embeddings.adoc file. Remove unnecessary .toMap() call and
fix typo in withTruncate method name.
- Removed the assertion checking for a non-null 'name' in ToolResponseMessage
- This change affects multiple AI model implementations:
AzureOpenAiChatModel, MiniMaxChatModel, MistralAiChatModel,
MoonshotChatModel, OpenAiChatModel, and ZhiPuAiChatModel
Resolves#1410
Update mongodb.adoc to include links to both beginner and intermediate
content for Spring AI and MongoDB integration. Add a new section
"Tutorials and Code Examples" with:
- Link to the Getting Started guide for basic integration
- Link to a detailed RAG tutorial for more advanced usage
This change provides users with a clear path from initial setup to
more complex implementations using the MongoDB Atlas Vector Store.
Modify getHost method to return a properly formatted URL string. This
ensures that the Chroma client can correctly connect to the service
when using Docker Compose.
Fixes#1395
Replace parallelStream with stream to prevent thread-unsafe appends to
the shared StringBuilder. This fixes the issue of intermingled key-value
pairs in the generated Document content. Also, replace StringBuffer
with StringBuilder for better performance in single-threaded context.
The change ensures correct ordering of extracted JSON keys and their
values in the resulting Document, improving the reliability and
readability of the parsed output.
This change introduces a new field for tracking reasoning tokens in the
OpenAI API response. It extends the Usage record to include
CompletionTokenDetails, allowing for more granular token usage
reporting. The OpenAiUsage class is updated to expose this new data,
and corresponding unit tests are added to verify the behavior.
This enhancement provides more detailed insights into token usage,
particularly for advanced AI models that separate reasoning from other
generation processes.
