Introduces support for Amazon Bedrock Converse API through a new BedrockProxyChatModel
implementation. This enables integration with Bedrock's conversation models with features
including:
- Support for sync/async chat completions
- Stream response handling
- Tool/function calling capabilities
- System message support
- Image input support
- Observation and metrics integration
- Configurable model parameters and AWS credentials
Adds core support classes:
- BedrockUsage: Implements Usage interface for token tracking
- ConverseApiUtils: Utility class for handling Bedrock API responses including:
- Tool use event aggregation and processing
- Chat response transformation from stream outputs
- Model options conversion
- Support for metadata aggregation
- URLValidator: Utility for URL validation and normalization with support for:
- Basic and strict URL validation
- URL normalization
- Multimodal input handling
- Enhanced FunctionCallingOptionsBuilder with merge capabilities for both ChatOptions
and FunctionCallingOptions
- Added BEDROCK_CONVERSE to AiProvider enum for metrics tracking
- Extended AWS credentials support with session token capability
- Added configurable session token property to BedrockAwsConnectionProperties
Adds new auto-configuration support:
- BedrockConverseProxyChatAutoConfiguration for automatic setup of the Bedrock Converse chat model
- BedrockConverseProxyChatProperties for configuration including:
- Model selection (defaults to Claude 3 Sonnet)
- Timeout settings (defaults to 5 minutes)
- Temperature and token control
- Top-K and Top-P sampling parameters
- Integration with existing BedrockAwsConnectionConfiguration for AWS credentials
Updates to testing infrastructure:
- Adds comprehensive test suite for Bedrock Converse properties and auto-configuration
- Integration tests for chat completion and streaming scenarios
- Property validation tests for configuration options
- Temporarily disabled other Bedrock tests due to AWS quota limitations
- Added ObjectMapper configuration for proper JSON handling
Added new spring-ai-bedrock-converse-spring-boot-starter module
Updates module configuration in parent POM and BOM to include new bedrock-converse
modules and starters. Adds necessary auto-configuration imports for seamless integration
with Spring Boot applications.
Unrelated changes:
- Disabled several Bedrock model tests (Jurassic2, Llama, Titan) due to AWS quota limitations
- Disabled PaLM2 tests due to API decommissioning by Google
Resolves#809, #802
Add docs and fix configs
- Move timeout configuration from chat properties to connection properties
- Add comprehensive documentation for Bedrock Converse API usage and configuration
- Update tests to reflect configuration changes
Co-authored-by: maxjiang153 <maxjiang153@users.noreply.github.com>
Standardize AWS credential handling in integration tests
- Improve how we manage AWS credentials across our integration test
suite and ensures consistent test configuration. We're replacing individual
environment variable checks with @RequiresAwsCredentials
annotation and standardizing the use of BedrockTestUtils for context creation
in tests
We also align all AWS regions to US_EAST_1 for consistency and add missing
dependency versioning for Oracle Free.
These changes make our AWS tests more easier to maintain.
Key changes:
- Replace @EnabledIfEnvironmentVariable with @RequiresAwsCredentials
- Standardize context creation via BedrockTestUtils
- Set AWS region to US_EAST_1
- Add Oracle Free dependency version in pom.xml
Following removal of Spring Boot's ConstructorBinding, convert record types
to regular POJOs to maintain JSON serialization. Update API classes:
- Convert FunctionTool and Function records to standard Java classes with
getters/setters
- Update test assertions to use getter methods instead of record accessors
- Fix failing tests in MiniMax, OpenAi and ZhiPu API implementations
This keeps core API models independent of Spring Boot while ensuring proper
serialization through standard Java beans.
- MistralAIApi FunctionTool ConstructorBinding removal changes
- Remove @NestedConfigurationProperty and @ConstructorBinding annotations from model options classes
- Remove spring-boot dependency from spring-ai-openai module
- Add exclusion for spring-boot-autoconfigure in spring-cloud-function-context dependency
- Move configuration metadata to spring-ai-spring-boot-autoconfigure module
This change decouples the core AI model implementations from Spring Boot,
making the core modules more lightweight and allowing them to be used in
non-Spring Boot applications. Configuration properties are now handled by
the spring-ai-spring-boot-autoconfigure module instead.
Fixes#1540
Adds @EnabledIfEnvironmentVariable annotation to integration tests that
use OpenAI embeddings. Tests will be skipped if OPENAI_API_KEY is not set,
making the build process more reliable for contributors who don't have
access to OpenAI services.
The PostgresMLEmbeddingModel autoconfiguration previously always executed
"CREATE EXTENSION IF NOT EXISTS pgml" on startup. This could cause issues
for users without superuser privileges or those who manage extensions
through other means.
Added a new configuration property 'createExtension' (default false) to
make this behavior optional. Users can now explicitly enable extension
creation when needed.
Updated documentation to explain the new configuration option and its
implications for deployment.
This fix ensures proper inheritance of defaults while maintaining the
correct precedence order for both generic and StabilityAI-specific options.
Added tests to verify the merge behavior for runtime options, default options,
and generic ImageOptions cases.
All output parser functionality has already been migrated to the converter package
since milestone M1.
Changes:
- Replace all instances of 'parser' with 'converter' in variable names and test cases
- Delete (pre M1) deprecated parser package and all associated classes
- Remove Output Parsers section from README.md
- Update Javadoc comments to reference converters instead of parsers
ObjectMapper instantiation is costly, so unless its usage
is one-shot, it is better to create a reusable instance
for upcoming usage.
Also, before this commit, serialization of most Kotlin
classes was not supported due to the lack of proper
Jackson KotlinModule detection.
This commit:
- Avoids per invocation ObjectMapper instantiation when
relevant
- Automatically detects and enables well-known Jackson
modules including the Kotlin one
- Removes org.springframework.ai.vectorstore.JsonUtils
which looks not needed anymore
More optimizations are possible like reusing more
ObjectMapper instances, but this could introduce more breaking
changes so this commit intends to be a good first step.
Kotlin tests will be provided in a follow-up commit.
Additional changes:
- Update ModelOptionsUtils to use JacksonUtils.instantiateAvailableModules()
- Add missing license headers
- Add missing author Javadoc comments
- Update Claude 3.5 Sonnet model version from 20240620 to 20241022 across:
- AnthropicApi model definitions
- Integration tests
- Sample events JSON
- Documentation pages
- Upgrade Ollama container to 0.3.14 in tests
- Add llama3.2:1b model to Ollama tests
- Convert Ollama functionCallTest to parameterized test
- Add PromptTokensDetails record to track cached tokens in prompt
- Update Usage record to include promptTokensDetails field
- Add getCachedTokens() method to OpenAiUsage
- Add test cases for cached tokens handling
Resolves#1506
In order to support edge cases due to different naming formats, this PR introduced an explicit normalization logic to ensure the correct matching when checking for the availability of a certain model. Integration tests have been added to cover the different scenarios, including models from Ollama and from Hugging Face.
Also fix the container creation on the useTestcontainers flag (christian)
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Consolidate the Ollama auto-pull logic at startup time, supporting the auto-pull for the default models specified via configuration properties and for optional models specified for initialization.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Fix configuration inheritance issue when default value is not specified.
* Make it possible to enable the auto-pull feature only for specific model types (e.g. for chat models only).
* Add the possibility to list explicit models to auto-pull at startup time.
Update Ollama model defaults and add new embedding model
* Change default chat model to Mistral
* Change default embedding model to mxbai-embed-large
* Add MXBAI_EMBED_LARGE to OllamaModel enum
* Remove DEFAULT_MODEL constant from OllamaOptions
* Update relevant classes to use new defaults
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Co-authored-by:Christian Tzolov <ctzolov@vmware.com>
* Introduce support for Ollama model auto-pull at startup time
* Enhance support for Ollama model auto-pull at run time
* Update documentation about integrating with Ollama and managing models
* Adopt Builder pattern in Ollama Model classes for better code readability
* Unify Ollama model auto-pull functionality in production and test code
* Improve integration tests for Ollama with Testcontainers
- Enhance OllamaModelPuller with configurable retry timeout
- Update test cases to use LLAMA3_1 instead of LLAMA3_2
- Improve Ollama documentation with clearer prerequisites and model pulling instructions
- Update Spring AI introduction page with new logo and diagram
- Remove unnecessary main method from OllamaModelPuller
- Remove unnecessary spring-web dependencies
- Update third-party library versions
- Refactor API classes to use consistent header handling
- Remove ApiUtils class and inline its functionality
- Adjust RestClient and WebClient builder usage in autoconfiguration
- Replace direct RestClient.Builder injections with ObjectProvider<RestClient.Builder>
and WebClient.Builder injections with ObjectProvider<WebClient.Builder>
- Update ChromaVectorStoreAutoConfiguration to use ObjectProvider
- Rename MongoDbAtlasLocalContainerConnectionDetailsFactoryTest to IT
- Switch spring-ai-chroma-store dependency from spring-web to spring-webflux
- Simplify ChromaApi constructor by using method reference for default headers
- Adjust import order
Resolves#1066Resolves#524
* Extend the OllamaApi to support listing, copying, deleting, and pulling models programmatically.
* Improve setup for integration testing with Ollama and Testcontainers.
Enables gh-526
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit enhances the Bedrock Anthropic model's output:
- Add response ID, model name, and usage data to ChatResponseMetadata
- Introduce DefaultUsage class for token usage information
- Update BedrockAnthropic3ChatModel to include new metadata
- Add Jackson annotations for serialization/deserialization
- Implement unit tests for DefaultUsage
These changes provide structured, serializable metadata in the
ChatResponse, improving the model's output with additional
information.
- Extend the OllamaApi.EmbeddingsResponse with total_duration,
load_duration and prompt_eval_count fields
- Rename OllamaUsage to OllamaChatUsage for clarity
- Add OllamaEmbeddingUsage to track embedding-specific usage metrics
- Update OllamaEmbeddingModel to use OllamaEmbeddingUsage
- Extend EmbeddingsResponse with additional metadata fields
- Update tests to reflect new usage tracking for embeddings
Resolves#1536
