This commit adds support for new Meta Llama 3.1 and 3.2 instruct models
to the LlamaChatBedrockApi enum. It includes model IDs for:
Llama 3.1: 8B, 70B, 405B
Llama 3.2: 1B, 3B, 11B, 90B
These additions allow users to specify the latest Llama models when
using the Bedrock API. The commit also updates the class Javadoc with
a link to AWS documentation for model IDs and bumps the @since version
to 1.0.0.
- Switch to OpenAIAsyncClient for streaming operations
- Modify AzureOpenAiChatModel constructor to accept OpenAIClientBuilder
- Update getChatCompletionsStream to use non-blocking async client
- Refactor related classes and tests to support OpenAIClientBuilder
- Revise AzureOpenAiAutoConfiguration to provide OpenAIClientBuilder
- Add AzureOpenAiChatClientTest to verify streaming functionality
- Adjust existing tests for compatibility with OpenAIClientBuilder
Resolves https://github.com/spring-projects/spring-ai/issues/981
This change improves support for asynchronous streaming operations
in the AzureOpenAiChatModel, addressing potential issues in reactive environments.
Resolves https://github.com/spring-projects/spring-ai/issues/832
Introduces retry functionality to VertexAI embedding and
chat models, enhancing their resilience against transient failures.
It also corrects a typo in the VertexAiEmbeddingConnectionDetails
class name.
Key changes:
* Add RetryTemplate to VertexAiTextEmbeddingModel and VertexAiGeminiChatModel
* Introduce spring-ai-retry dependency
* Refactor code to support retry logic
* Update auto-configuration classes to incorporate retry functionality
* Fix typo in VertexAiEmbeddingConnectionDetails class name
remove extraneous commented out code
Add missing copyright headers, author etc.
This commit introduces a major overhaul of the advisor system in Spring AI,
improving modularity, type safety, and consistency
Core Changes:
- Replace RequestAdvisor and ResponseAdvisor with CallAroundAdvisor and StreamAroundAdvisor
- Introduce AdvisedRequest and AdvisedResponse classes for better encapsulation
- Deprecate RequestResponseAdvisor in favor of new advisor types
- Remove AdvisorObservableHelper class
Advisor Implementation Updates:
- Update AbstractChatMemoryAdvisor, MessageChatMemoryAdvisor, PromptChatMemoryAdvisor,
QuestionAnswerAdvisor, SafeGuardAroundAdvisor, SimpleLoggerAdvisor, and
VectorStoreChatMemoryAdvisor to implement new advisor interfaces
- Remove CacheAroundAdvisor (functionality likely moved elsewhere)
- Make CallAroundAdvisor and StreamAroundAdvisor extend Ordered interface
Client and Chain Management:
- Modify DefaultChatClient to use new advisor chain approach
- Refactor DefaultAroundAdvisorChain for better ordering and observation
- Implement builder pattern for advisor chain construction in DefaultChatClient
- Separate call and stream advisors in DefaultAroundAdvisorChain
Observation and Context Handling:
- Update observation conventions and context handling in advisors
- Add order field to AdvisorObservationContext
- Modify DefaultAdvisorObservationConvention to include order in high cardinality key values
Testing and Integration:
- Refactor ChatClientAdvisorTests and add new AdvisorsTests
- Update integration tests to reflect new advisor structure
- Enhance AdvisorsTests to verify correct advisor execution order
New Features:
- Generalize the Protect From Blocking functionality across all advisors
- Add (experimental) Re2 advisor to enhance reasoning capabilities of LLMs
- Add disabled Re2 test in OpenAiChatClientIT
Documentation:
- Add Advisors documentation
- Enhance advisors documentation with order explanation and Re2 example
Advisor Ordering:
- Introduce Advisor constants for precedence ordering
- Update AbstractChatMemoryAdvisor to use new precedence constant
- Improve advisor ordering and management in DefaultAroundAdvisorChain.Builder
- Remove redundant reordering logic from DefaultAroundAdvisorChain
These changes aim to provide a more flexible and powerful advisor system,
allowing for easier implementation of complex AI-driven interactions
Co-authored-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
An upper bound for the number of tokens that can be generated for a completion,
including visible output tokens and reasoning tokens.
Replaces max_tokens field which is now deprecated.
This is related to https://github.com/spring-projects/spring-ai/issues/889
This commit addresses a bug where certain fields were being made
indirectly mandatory due to Assert.notNull checks in the Builder's
with* methods. Specifically:
- Removed Assert.notNull checks from withResponseFormat, withSeed,
withLogprobs, withTopLogprobs, and withEnhancements methods.
These checks were causing exceptions in AzureOpenAiChatModel.getDefaultOptions
when not all fields were set, leading to failures in methods like
ChatClient.create, even when using the AzureOpenAiChatModel constructor
with OpenAIClient.
The removal of these checks aligns with the @JsonInclude(Include.NON_NULL)
annotation on AzureOpenAiChatOptions, which already ignores null options.
This change maintains the intended flexibility while preventing unintended
mandatory requirements.
This commit introduces support for Oracle Cloud Infrastructure (OCI)
GenAI embedding models in Spring AI. It includes:
* New OCIEmbeddingModel class for interacting with OCI GenAI API
* Auto-configuration for easy setup and integration
* Properties for configuring OCI connection and embedding options
* Documentation updates explaining usage and configuration
* Integration tests to verify functionality
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
- Removed the assertion checking for a non-null 'name' in ToolResponseMessage
- This change affects multiple AI model implementations:
AzureOpenAiChatModel, MiniMaxChatModel, MistralAiChatModel,
MoonshotChatModel, OpenAiChatModel, and ZhiPuAiChatModel
Resolves#1410
This change introduces a new field for tracking reasoning tokens in the
OpenAI API response. It extends the Usage record to include
CompletionTokenDetails, allowing for more granular token usage
reporting. The OpenAiUsage class is updated to expose this new data,
and corresponding unit tests are added to verify the behavior.
This enhancement provides more detailed insights into token usage,
particularly for advanced AI models that separate reasoning from other
generation processes.
Add missing options from Azure ChatCompletionsOptions to Spring AI
AzureOpenAiChatOptions. The following fields have been added:
- seed
- logprobs
- topLogprobs
- enhancements
This change ensures better alignment between the two option sets,
improving compatibility and feature parity.
Resolves https://github.com/spring-projects/spring-ai/issues/889
- Add handling for cases where ChatCompletion has a stop reason but empty generations.
Creates a Generation with empty content and metadata when this occurs.
- Update AnthropicChatModelObservationIT to expect "end_turn" finish reason.
- Update javadoc.
- When ChatClientRequestSpec#prompt(Prompt) is used, unseal the prompt instance.
Convert the last message instance (if user message) into spec#user and spec#media
and add the remaining messages (excluding the last) to the spec#messages.
Add the prompt#options to the spec#options.
- Improve DefaultChatClient to handle UserMessage media and content separately.
- Update AbstractToolCallSupport to use new hasToolCalls() method.
- Add hasToolCalls() method to AssistantMessage.
- Enhance ChatClientTest with additional test cases for media handling.
- Disable Groq and Nvidia integration tests due to rate limiting and credit requirements.
This commit introduces a new proxyToolCalls option for various chat
models in the Spring AI project. When enabled, it allows the client to
handle function calls externally instead of being processed internally
by Spring AI.
The change affects multiple chat model implementations, including:
AnthropicChatModel
AzureOpenAiChatModel
MiniMaxChatModel
MistralAiChatModel
MoonshotChatModel
OllamaChatModel
OpenAiChatModel
VertexAiGeminiChatModel
ZhiPuAiChatModel
The proxyToolCalls option is added to the respective chat options
classes and integrated into the AbstractToolCallSupport class for
consistent handling across different implementations.
The proxyToolCalls option can be set either programmatically via
the <ModelName>ChatOptions.builder().withProxyToolCalls() method
or the spring.ai.<model-name>.chat.options.proxy-tool-calls
application property.
Documentation for the new option is also updated in the relevant
Antora pages.
Added ITs for proxy tool calls
Remove ChatClientPromptRequestSpec and all ChatClient.prompt() overloads can how take advantage of the full fluent API. Docs updated
Resolves#1367
This commit introduces support for the Watsonx.ai embedding model.
It includes:
- Watsonx embedding options class with tests
- Watsonx embedding model implementation
- Auto-configuration and properties for the embedding model
- Tests for the Watsonx embedding model
- Documentation for using the Watsonx embedding model
Also removed use of deprecated APIs in WatsonAIChatModel
Sometimes, the MiniMax stream mode function calls might get split,
resulting in an empty tool call ID. This indicates that the previous
call is not finished, which is an unusual API design.
The issue occurs when tool calls return in a format like:
[{"id":"1","function":{"name":"a"}},{"id":"","function":{"arguments":"[1]"}}]
These need to be merged into:
[{"id":"1","name":"a","arguments":"[1]"}]
This commit addresses the merging process to handle split function calls.
authored-by: mxsl-gr <mxsl-gr@users.noreply.github.com>
This change updates the type of portable chat options from Float to
Double. Affected options include:
- frequencyPenalty
- presencePenalty
- temperature
- topP
The motivation for this change is to simplify coding. In Java, Float
values require an "f" suffix (e.g., 0.5f), while Double values don't
need any suffix. This makes Double easier to type and reduces
potential errors from forgetting the "f" suffix.
APIs, tests, and documentation have been updated to reflect this
change.
Fixes gh-712
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Add web search mode response in choice.message for enhanced
compatibility
- Implement web search mode for stream mode
- Add comprehensive unit tests for new features
Related to #1292
feat: enhance the compatibility of the minimax model and tests, related issue #1292
Refactoring and enhancements to the advisor functionality.
New Advisor interfaces in the org.springframework.ai.chat.client.advisor.api package:
- Advisor: Base interface for all advisor types.
- RequestAdvisor: For advising on request data before execution.
- ResponseAdvisor: For advising on response data after execution, with enhanced streaming modes.
- CallAroundAdvisor and StreamAroundAdvisor: For around advice on synchronous and streaming requests respectively.
- AroundAdvisorChain and DefaultAroundAdvisorChain: To manage chaining of around advisors.
Advisor Chain and Prompt Generation:
- Added the DefaultAroundAdvisorChain class to manage the sequence of advisors applied around chat model methods.
- Adjusted the prompt generation (toPrompt) to integrate with the refactored AdvisedRequest object.
Refactoring and Updates:
- Replaced the deprecated RequestResponseAdvisor interface with RequestAdvisor and ResponseAdvisor across the spring-ai-core and test modules.
- Updated the DefaultChatClient and related classes to use the new Advisor interface, improving modularity and consistency.
- Refactored DefaultAdvisorSpec and DefaultChatClientRequestSpec to handle the new Advisor type, and revised advisor lists and methods accordingly.
- Enhanced the handling of streaming responses, introducing StreamResponseMode for better control during streaming scenarios.
* Fixes an issue with advisor name resolution
* Streamlines repeating code
* Add a new advisor strategy for ON_FINISH_REASON streaming responses, which is used by the Q&A advisor
* Improve observable instrumentation by passing the parent observation to the advisor observation
Implement web search functionality for the MiniMax model.
Includes unit tests
This enhancement expands the model's ability to access and utilize current information from the internet.
Resolves#1245
Implement function call capability for MiniMax model and add unit tests based on new tool classes.
Address most scenarios, but note limitations in complex English contexts
with multiple function calls. Weather query example: may stop
prematurely when querying multiple locations due to single-location
parameter limit. This behavior stems from model performance constraints.
Streaming function calling is not passing tests, will be address seperately.
Resolves#1077
Implement function call capability for the Moonshot model. Include unit
tests to verify the new functionality. This feature addresses the
requirements outlined in issue #1058.
fix: MiniMax function call
review
Resolve issue with AzureOpenAiChatModel processing non-public images as
byte arrays. Implement handling for both URL strings and byte arrays,
converting latter to base64 encoded data URLs. Add test case for
resource-based media data. Update documentation with sample for
classpath resources.
This commit introduces the OpenAiModerationModel and related classes:
- Add OpenAiModerationModel for content moderation
- Create OpenAiModerationOptions for configuration
- Implement OpenAiModerationProperties for Spring Boot setup
- Add integration tests in OpenAiModerationModelIT
- Add documentation
Co-authored-by: hemeda3 <hemeda3@users.noreply.github.com>
This commit introduces a new boolean option 'maskSensitiveInfo' for
the MiniMax API model support in Spring AI. This feature allows users
to control whether sensitive information in the output is masked.
Relevant unit tests have been added to ensure proper functionality.
Resolves: #1216
* Consolidate usage of “db.collection.name” attribute to track table name, collection name, index name, document name, or whatever concept a vector database uses to store data. Removed “db.index” that was use sometimes instead of “db.collection.name”. This usage is in line with the OpenTelemetry Semantic Conventions.
* Configure query response content to be included as a “span event” instead of a “span attribute” if the backend system supports that, similar to how we do for the model observations.
* Structure vector store observation attributes in dedicated enums, including one for the Spring AI Kinds to avoid hard-coding the same value in a lot of places. This follows the OpenTelemetry Semantic Conventions as much as possible. Also, adopt Spring usual non-null-by-default strategy as much as possible.
* Align vector store conventions to the chat model ones, and follow alphabetical order for values. This is particularly useful for the convention classes, for which the Micrometer performance of exporting telemetry data improves when key values are added already sorted to the context.
* Fix flaky test in Mistral AI.
* Improve Qdrant integration tests.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Handle the case where AnthropicChatResponse content is empty to prevent
an out of bounds exception. This ensures that the model behaves correctly
even when no content is returned in the response.
- When embedding documents, allow batching the documents using some criteria.
- `BatchingStrategy` interface with a `TokenCountBatchingStrategy` implementation that uses
the openai max input token size of 8191 as the default.
- Add a default method in EmbeddingModel to embed document using this new batching strategy.
- Change `MilvusVectorStore` to make use of this new batching API.
- Adding unit tests for `TokenCountBatchingStrategy`.
- Adding openai integration test to call the embed API that uses batching.
Resolves https://github.com/spring-projects/spring-ai/issues/1214
Other vector stores will be updated seperately
