In order to support edge cases due to different naming formats, this PR introduced an explicit normalization logic to ensure the correct matching when checking for the availability of a certain model. Integration tests have been added to cover the different scenarios, including models from Ollama and from Hugging Face.
Also fix the container creation on the useTestcontainers flag (christian)
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Consolidate the Ollama auto-pull logic at startup time, supporting the auto-pull for the default models specified via configuration properties and for optional models specified for initialization.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Fix configuration inheritance issue when default value is not specified.
* Make it possible to enable the auto-pull feature only for specific model types (e.g. for chat models only).
* Add the possibility to list explicit models to auto-pull at startup time.
Update Ollama model defaults and add new embedding model
* Change default chat model to Mistral
* Change default embedding model to mxbai-embed-large
* Add MXBAI_EMBED_LARGE to OllamaModel enum
* Remove DEFAULT_MODEL constant from OllamaOptions
* Update relevant classes to use new defaults
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Co-authored-by:Christian Tzolov <ctzolov@vmware.com>
* Introduce support for Ollama model auto-pull at startup time
* Enhance support for Ollama model auto-pull at run time
* Update documentation about integrating with Ollama and managing models
* Adopt Builder pattern in Ollama Model classes for better code readability
* Unify Ollama model auto-pull functionality in production and test code
* Improve integration tests for Ollama with Testcontainers
- Enhance OllamaModelPuller with configurable retry timeout
- Update test cases to use LLAMA3_1 instead of LLAMA3_2
- Improve Ollama documentation with clearer prerequisites and model pulling instructions
- Update Spring AI introduction page with new logo and diagram
- Remove unnecessary main method from OllamaModelPuller
- Remove unnecessary spring-web dependencies
- Update third-party library versions
- Refactor API classes to use consistent header handling
- Remove ApiUtils class and inline its functionality
- Adjust RestClient and WebClient builder usage in autoconfiguration
- Replace direct RestClient.Builder injections with ObjectProvider<RestClient.Builder>
and WebClient.Builder injections with ObjectProvider<WebClient.Builder>
- Update ChromaVectorStoreAutoConfiguration to use ObjectProvider
- Rename MongoDbAtlasLocalContainerConnectionDetailsFactoryTest to IT
- Switch spring-ai-chroma-store dependency from spring-web to spring-webflux
- Simplify ChromaApi constructor by using method reference for default headers
- Adjust import order
Resolves#1066Resolves#524
* Extend the OllamaApi to support listing, copying, deleting, and pulling models programmatically.
* Improve setup for integration testing with Ollama and Testcontainers.
Enables gh-526
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit enhances the Bedrock Anthropic model's output:
- Add response ID, model name, and usage data to ChatResponseMetadata
- Introduce DefaultUsage class for token usage information
- Update BedrockAnthropic3ChatModel to include new metadata
- Add Jackson annotations for serialization/deserialization
- Implement unit tests for DefaultUsage
These changes provide structured, serializable metadata in the
ChatResponse, improving the model's output with additional
information.
- Extend the OllamaApi.EmbeddingsResponse with total_duration,
load_duration and prompt_eval_count fields
- Rename OllamaUsage to OllamaChatUsage for clarity
- Add OllamaEmbeddingUsage to track embedding-specific usage metrics
- Update OllamaEmbeddingModel to use OllamaEmbeddingUsage
- Extend EmbeddingsResponse with additional metadata fields
- Update tests to reflect new usage tracking for embeddings
Resolves#1536
- Integrate ObservationRegistry and EmbeddingModelObservationConvention
- Update TransformersEmbeddingModel to use observations
- Add TransformersEmbeddingModelObservationTests
- Update TransformersEmbeddingModelAutoConfiguration for observation support
- Add ONNX to AiProvider enum
- Adding observability for Vertex Gemini chat model streaming call
- Updating auto-configuration for gemini chat model with the observability components
- Integration test for streaming observability
- Implement observation instrumentation for AzureOpenAiEmbeddingModel
- Create AzureOpenAiEmbeddingModelObservationIT for integration testing
- Update AzureOpenAiAutoConfiguration to support observation
- Introduce ObservationRegistry and observation conventions
- Integrate ObservationRegistry for instrumentation in VertexAiGeminiChatModel's non-streaming call
- Add ChatModelObservationContext and related classes for observation
- Refactor createGeminiRequest method to support runtime options for imperative calls
- Implement observation for non-streaming chat operations in VertexAiChatModelObservationIT
- Add VertexAiGeminiConstants class for provider name
This commit enhances the VertexAiGeminiChatModel with observability features
for the non-streaming, imperative call() method. It allows for better
monitoring and instrumentation of synchronous chat operations. The changes
include refactoring to improve runtime option handling for these calls and
adds integration tests for the new observability features.
- Replaced `Map<String, Object>` with `ToolContext` in the `FunctionCallback`, `AbstractFunctionCallback`, and related classes.
- Updated all BiFunction definitions to use `ToolContext` as the second parameter,
enhancing the clarity and structure of the tool context management.
- Modified `FunctionCallbackWrapper` and `FunctionCallbackContext` to adapt to the new `ToolContext` parameter.
- Adjusted the handling of tool context in the documentation and test classes.
- Updated relevant test cases to reflect the API changes and modified the function handling logic to ensure consistency.
- Introduce toolContext to ChatClient, DefaultChatClient, and AdvisedRequest
- Add methods to set and manage toolContext via the FunctionCallingOptions
- Update tests to include toolContext in relevant scenarios
- Implement toolContext handling in function calling options
Unlike other metrics implementations supported by Micrometer, when using the Prometheus integration, all metric attributes are supposed to have a value, or else the related metrics are dropped.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Chat model observations support several key-value pairs, coming from the Spring AI abstractions.
Currently, whenever a value is not available (either because not configured by the user or not supported by the model provider), a span/metrics is generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and incompliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for chat model observation to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit adds support for tool context in various chat options classes across
different AI model implementations and enhances function calling capabilities.
The tool context allows passing additional contextual information to function callbacks.
- Add toolContext field to chat options classes
- Update builder classes to support setting toolContext
- Enhance FunctionCallback interface to support context-aware function calls
- Update AbstractFunctionCallback to implement BiFunction instead of Function
- Modify FunctionCallbackWrapper to support both Function and BiFunction and
to use the new SchemaType location
- Add support for BiFunction in TypeResolverHelper
- Update ChatClient interface and DefaultChatClient implementation to support
new function calling methods with Function, BiFunction and FunctionCallback arguments
- Refactor AbstractToolCallSupport to pass tool context to function execution
- Update all affected <Model>ChatOptions with tool context support
- Simplify OpenAiChatClientMultipleFunctionCallsIT test
- Add tests for function calling with tool context
- Add new test cases for function callbacks with context in various integration tests
- Modify existing tests to incorporate new context-aware function calling capabilities
- Add docs in in openai function calling
Resolves#864, #1303, #991
Implement observability for imperative calls in AzureOpenAiChatModel:
* Integrate ObservationRegistry to track metrics and events
* Create observation context and define conventions for consistency
* Add integration test to verify chat model observations
* Update AiProvider enum with new AZURE_OPENAI entry
