- Deprecate existing ChatOptionsBuilder and its inner class DefaultChatOptions
- Create a new builder interface ChatOptions.Builder for building the Chat options
- Create an explicit DefaultChatOptions
- Create DefaultChatOptionBuilder which can create DefaultChatOptions
- Add javadoc for the deprecated Builder
Resolves#1875
- Fix OpenAI ChatModel's call() operation
- When toolcalling is used, calculate cumulative usage from the preceding ChatResponses
- Fix OpenAI ChatModel's stream() operation
- Make sure that cumulative usage is calculated from the ChatResponse which has a valid usage
- Use overlapping buffer to check and store the usage from the response that holds the usage.
- Add tests for both call() and stream()
The Document class previously allowed multiple media entries while also having a
text field, leading to ambiguity in content handling. This change enforces a
clear separation between text and media documents to prevent content type
confusion and simplify document processing.
A Document now must contain either text content or a single media entry, but
never both. This aligns with the class's primary use in ETL pipelines where
clear content type boundaries are essential for proper embedding generation and
vector database storage.
Additional architectural changes:
- Document now implements a cleaner API by removing deprecated methods
- Removed MediaContent interface implementation from Document class
- Document.getMedia() now returns a single Media object instead of Collection
- Removed EMPTY_TEXT constant in favor of proper null handling
- Constructor signatures simplified and streamlined
- Builder pattern improved to enforce single content type constraint
The breaking changes include:
- Media is now a single entry instead of a collection
- Content field renamed to text for clarity
- Removed support for mixed content types
- Simplified builder API to prevent ambiguous construction
Prefer using text-related methods over deprecated content methods to
better reflect the actual content type being handled and improve API clarity.
- Add BedrockMediaFormat class to handle media format conversions for documents, images and videos
- Enhance Media class with builder pattern and comprehensive format constants
- Refactor BedrockProxyChatModel to support multimodal content handling
- Add integration tests for PDF, image and video processing
- Add unit tests for Media and BedrockMediaFormat classes
- Upgrade AWS SDK version from 2.26.7 to 2.29.29
- Remove redundant aws.sdk.version property in favor of awssdk.version
Documentation updates for Bedrock Converse API
- Added multimodal support documentation (images, video, documents)
- Added deprecation notices for existing Bedrock model implementations
- Updated feature comparison table
- Added warning notes about transitioning to Converse API
- Upgrade Ollama version used for IT from 0.3.14 to 0.5.1
- Fix string comparison pattern in OllamaModelManager
- Re-enable stream function calling test
- Improve multimodal test assertions
- Implement streaming tool call support in OllamaApi and OllamaChatModel
- Add OllamaApiHelper to manage merging of streaming chat response chunks
- Remove @Disabled annotations for streaming function call tests
- Update documentation to reflect new streaming function call capabilities
- Add a new default constructor for ChatResponse
- Update Ollama chat documentation to clarify streaming support requirements
- Deprecated withContent(), withImages(), and withToolCalls() methods
- Replaced with content(), images(), and toolCalls() methods
Add token and duration aggregation for Ollama chat responses
- Modify OllamaChatModel to support accumulating tokens and durations across multiple responses
- Update ChatResponse metadata generation to aggregate usage and duration metrics
- Add tests to verify metadata aggregation behavior
Refactor Ollama duration fields and tests
- Replace Duration fields in OllamaApi.ChatResponse with Long to represent durations in nanoseconds, ensuring precision and compatibility.
- Update methods to convert Long nanoseconds to Duration objects (getTotalDuration, getLoadDuration, getEvalDuration, getPromptEvalDuration).
- Adjust merge logic in OllamaApiHelper to sum Long values for duration fields.
- Modify test cases in OllamaChatModelTests to align with Long duration representation and Duration.ofNanos conversions.
- Add new test class OllamaDurationFieldsTests to validate JSON deserialization and Duration conversion for duration fields.
Resolves#1847
Related to #1800Resolves#1796
Related to #1307
test: Update OllamaWithOpenAiChatModelIT integration tests
- Remove @Disabled annotation for streamFunctionCallTest
- Add inputType for function callback in stream function call test
- Since the Document object's reference to the `embedding` is deprecated and will be removed, the VectorStore implementations require a way to store the embedding of the corresponding Document objects
- One way to fix this is, to have the EmbeddingModel#embed to return the embeddings in the same order as that of the Documents passed to it.
- Since both the Document and embedding collections use the List object, their iteration operation will make sure to keep them in line with the same order.
- A fix is required to preserve the order when batching strategy is applied.
- Updated the Javadoc for BatchingStrategy
- Fixed the Document List order in TokenCountBatchingStrategy
- Refactored the vector store implementations to update this change
Resolves #GH-1826
- PaymentStatusFunctionCallingIT in org.springframework.ai.mistralai.api.tool
is failing. Needs investigation.
See https://github.com/spring-projects/spring-ai/issues/1853
- OpenSearchVectorStoreWithOllamaIT updated to pull model if not available.
Document
* Introduced “score” attribute in Document API. It stores the similarity score.
* Consolidate “distance” metadata for Documents. It stores the distance measurement.
* Adopted prefix-less naming convention in Document.Builder and deprecated old methods.
* Deprecated the many overloaded Document constructors in favour of Document.Builder.
Vector Stores
* Every vector store implementation now configures a “score” attribute with the similarity score of the Document embedding. It also includes the “distance” metadata with the distance measurement.
* Fixed error in Elasticsearch where distance and similarity were mixed up.
* Added missing integration tests for SimpleVectorStore.
* The Azure Vector Store and HanaDB Vector Store do not include those measurements because the product documentation do not include information about how the similarity score is returned, and without access to the cloud products I could not verify that via debugging.
* Improved tests to actually assert the result of the similarity search based on the returned score.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Introduced new options for audio output modalities in ChatCompletionRequest
- Added AudioParameters configuration for voice and audio format selection
- Enhanced OpenAiChatModel to handle audio generation and embedding
- Updated AssistantMessage and Media classes to support audio media
- Added integration tests for audio output functionality
- Implemented support for text and audio multi-modal responses
- Updated Spring AI's chat model comparison table to clarify OpenAI's input/output modalities
- Added new configuration properties for audio output:
* spring.ai.openai.chat.options.output-modalities
* spring.ai.openai.chat.options.output-audio
- Extended documentation to explain audio output generation with the gpt-4o-audio-preview model
- Updated Spring Boot configuration metadata to support new audio-related properties
- Included auto-configuration integration test for chat model with audio response generation
Resolves#1841
Extracts shared function callback builder functionality into DefaultCommonCallbackInvokingSpec
base class, reducing code duplication across builder implementations.
Makes FunctionInvokingSpec and MethodInvokingSpec extend CommonCallbackInvokingSpec for better
code organization. Also fixes function/description builder order in Anthropic tests.
- Introduced a common base class for function callback builders to centralize shared logic
- Standardized the order of method chaining for function and description in multiple AI model test classes
- Refactored test cases across various AI model integrations
- Corrected builder method order from .description().function() to .function().description()
and .description().method() to .method().description()
- Updated multiple test files to consistently use .function() before .description()
- Updated documentation examples to reflect new builder method order
- Modified DefaultFunctionCallbackResolver to maintain new builder method order
- Updated DefaultChatClient and ChatClient test classes to reflect new builder pattern
- Simplified callback specification by removing parent spec reference
- Removed cascading getter logic for description, schema type, and other properties
- Minor adjustments to function callback builder and invoking specs
- Introduced `PerplexityWithOpenAiChatModelIT` integration test for Perplexity AI with OpenAI Chat Model.
- Includes various test cases for role-based prompts, streaming responses, token usage validation, and output converters.
- Added tests for function calls and metadata validation.
- Updated Antora navigation (`nav.adoc`) to include Perplexity AI documentation link.
- Enhanced chat model comparison documentation to highlight Perplexity AI integration.
- Added a dedicated `perplexity-chat.adoc` page under `spring-ai-docs` to provide detailed documentation for integrating Perplexity AI.
- Covers API prerequisites, auto-configuration, and runtime options.
- Explains configuration properties such as `spring.ai.openai.base-url`, `spring.ai.openai.chat.model`, and `spring.ai.openai.chat.options.*`.
- Provides examples for environment variable setup and runtime overrides.
- Highlights limitations like lack of multimodal support and explicit function calling.
- Includes a sample Spring Boot controller demonstrating integration usage.
- Links to Perplexity documentation for further reference.
When calling the OpenAiChatModel with ChatOptions, they are always ignored. Options are considered only when using OpenAiChatOptions or FunctionCallingOptions.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Added support for audio input and output in OpenAI Chat Completion API
- Introduced new audio-related parameters, enums, and record types
- Updated ChatCompletionMessage, ChatCompletionChunk, and related classes
- Added new AudioParameters, AudioOutput, and InputAudio record types
- Implemented method to handle audio media content conversion
- Included new model enum for GPT-4o audio preview
- Extended existing API classes to accommodate audio modalities
- Modified usage tracking and metadata classes to handle audio-specific token details
- Improved ModelOptionsUtils with additional JSON utility methods
Tests:
- Updated test classes to validate audio input and output functionality
- Added integration tests for multimodal audio input with streaming and non-streaming methods
- Created parameterized tests for audio-enabled models
- Enhanced OpenAI API integration tests to cover audio-related scenarios
Docs:
- Updated documentation in spring-ai-docs to explain audio multimodality support
Resolves#1560
- Introduces a new FunctionCallbackResolver interface to define the strategy for
resolving FunctionCallback instances.
- Renames FunctionCallbackContext to DefaultFunctionCallbackResolver to better reflect its
implementation role. Updates all related components to use the new interface.
- Update the affected AI model implementations
- Replaces FunctionCallbackContext parameter with FunctionCallbackResolver in all
model constructors
- Updates builder patterns to use functionCallbackResolver() method instead of
withFunctionCallbackContext()
- Deprecates old withFunctionCallbackContext() methods in builders to guide
migration
- Updates integration tests to use DefaultFunctionCallbackResolver
- Improves documentation to clarify the resolver's role in function callbacks
- Moves SchemaType enum from FunctionCallbackContext to FunctionCallback
Resolves#758
- ChatGenerationMetadata provides Map for additional metadata
- Add Builder interface and DefaultChatGenerationMetadataBuilder
- Add DefaultChatGenerationMetadata implementation
- Update all AI model implementations to use the new builder pattern
- Deprecate ChatGenerationMetadata.from() factory method
- Remove commented out legacy code from ChatGenerationMetadata interface
- Make metadata and content filter collections immutable via Collections.unmodifiableSet()
- Add null validation in DefaultChatGenerationMetadata constructor
- Improve toString() method with more informative output
- Enhance AnthropicChatModel to support PDF and document content types
- Introduce getContentBlockTypeByMedia method for flexible media type handling
- Update ContentBlock handling to dynamically determine content type for media
- Add multimodal PDF support test case for Claude 3.5 Sonnet
- Update documentation to reflect PDF and multimodal capabilities
- Modify comparison chart to show PDF support for Anthropic Claude
Fixese #1819
review
- Add new Maven profile 'ci-fast-integration-tests' for running selective ITs
- Remove redundant vector store skip flags from properties section
- Update maven-failsafe-plugin to version 3.5.2
- Configure test exclusions for various components:
- Most model integration tests (Anthropic and OpenAI)
- Most vector store tests (except PgVector and Chroma)
- Most auto-configuration tests
- All test containers and docker compose tests
- AI evaluation tests
- Convert the docker-compose tests into ITs
- Convert the testcontainers tests into ITs
- Updated README.md
- Explain the new profile and also the new integration tests repo
- Describe ways to run integration tests for specific modules
- Add badge for https://github.com/spring-projects/spring-ai-integration-tests
- Add test case for PDF document summarization using Gemini multimodal capabilities
- Update documentation to reflect PDF support in model comparison table
- Add PDF format to multimodal capabilities documentation
- Upgrade Spring Boot to 3.3.6
- Update swagger-codegen-maven-plugin to 3.0.64
- Add custom template for HttpBasicAuth
- Fix mockwebserver version to 4.12.0
- Fix parent version reference in opensearch-store
- Add Pixtral models (PIXTRAL and PIXTRAL_LARGE) to MistralAiApi.ChatModel
- Update MistralAiChatClientIT to use Pixtral model for testing
- Add new Ollama models:
* QWEN_2_5_7B
* LLAMA3_2_VISION_11b
* LLAMA3_2_VISION_90b
Fixes: #1753https://github.com/spring-projects/spring-ai/issues/1753
- Add character-based truncation (max 2048 chars) for Cohere embedding requests
- Support both START and END truncation strategies
- Add unit tests verifying truncation behavior for both strategies
Truncation is applied before sending requests to Bedrock API to avoid
ValidationException when text exceeds maximum length. The END strategy
(default) keeps the first 2048 characters while START keeps the last
2048 characters.
- Replace hardcoded "tool_use" with StopReason enum value
- Add tests for token usage aggregation with tool calls
- Add handling for null response metadata
- Modify stream method to support recursive tool call handling
- Update token tracking and metadata merging for streamed responses
- Improve token usage calculation for tool use events
- Update test cases to handle new response processing
- Modify call method to support recursive tool call handling
- Add support for cumulative token tracking across tool call iterations
- Introduce internal call method to track and aggregate token usage
- Merge previous chat response tokens with current response tokens
Resolves#1743
- Add TOOL_CALL_HISTORY constant to store tool call history
- Extend ToolContext with getToolConversationHistory method
- Include tool call history in tool context during function execution
- Add test coverage for tool call history verification
Resolves#1202
