- Extends the ToolCallingAutoConfiguration to support both FunctionCallback and ToolCallback types.
- The toolCallbackResolver bean now handles both callback types through ObjectProvider injection.
- Added comprehensive tests to verify the resolution of multiple function and tool callbacks.
- Introduce new StaticToolCallbackProvider implementation
- Update ToolCallbackProvider to return FunctionCallback[]
- Migrate from List to ToolCallbackProvider in configurations
- Update tests to use new provider pattern
- Enhance tool callback providers to support multiple clients
- Refactor AsyncMcpToolCallbackProvider and SyncMcpToolCallbackProvider to handle multiple MCP clients
- Add ToolCallbackProvider support to ChatClient API
- Deprecate direct tool callback list methods in favor of providers
- Fix typos in Closeable class names
- Update MCP documentation with new examples and usage patterns
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
- Update AnthropicChatModel to use the new ToolCallingManager API, while ensuring full API backward compatibility.
- Introduce Builder to instantiate a new AnthropicChatModel since the number of overloaded constructors is growing too big.
- Update documentation about tool calling and Anthropic support for that.
Part of the #2207 epic
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Each ModelObservationContext takes both a request object (e.g. Prompt) and an options object (e.g. ChatOptions). However, the options are already included in the request object. This PR deprecates the additional field, which will be removed in a subsequent release.
The reason why the extra field was there in the first place was due to the model implementations not handling request options correctly, requiring a dedicated setter. We started fixing the model implementations now, so we are deprecating te extra field, and we'll remove it in the next release, once we have completed the implementation of a fix for all model implementations.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Migrate from function calling to tool calling API
- Add support for Gemini 2.0 models (flash, flash-lite)
- Implement JSON schema to OpenAPI schema conversion
- Add builder pattern for improved configuration
- Deprecate legacy function calling constructors and methods
- Update default model to GEMINI_2_0_FLASH
- Add comprehensive test coverage for tool calling
- Upgrade victools dependency to 4.37.0
- Update the Vertex Tool calling docs
Part of the #2207 epic
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Additiona fixes: API key validation and tool calling backward compatibility
- Fix API key validation in OpenAiApi builder
- Standardize API key validation using Assert.notNull
- Add backward compatibility support for FunctionCallback in tool calling
- Update integration tests to use LegacyToolCallingManager
Co-authored-by:Christian Tzolov <christian.tzolov@broadcom.com>
Signed-off-by: Ricken Bazolo <ricken.bazolo@gmail.com>
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
- Update OpenAiChatModel to use the new ToolCallingManager API, while ensuring full API backward compatibility.
- Introduce Builder to instantiate a new OpenAiChatModel since the number of overloaded constructors is growing too big.
- Update documentation about tool calling and OpenAI support for that.
- Add extra validation to ensure the uniqueness of tool names when aggregated from different sources.
- Ensure consistent merging of options, following Spring Boot strategy.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Change VectorStore.delete() and related implementations to return void instead of Optional<Boolean>
- Remove unnecessary boolean return values and success status checks across all vector store implementations
- Clean up tests by removing redundant assertions
- Implementations continue using runtime exceptions for error signaling
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
When fields in OllamaOptions are marked as ignored in Jackson, they require explicit merge of runtime and default options.
Added tests to validate the different merge combinations for all tool-related options.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Spring AI's dependencies management to derive from Spring Boot 3.4.2
- Remove explicit versioning of dependencies
- Update upgrade notes for Spring Boot 3.4.2
- Fix ElasticSearch client changes with the latest dependency derived from Spring Boot 3.4.2
Adds comprehensive Model Context Protocol (MCP) integration to Spring AI, including:
Core Features:
- MCP client implementation with Spring AI tool calling capabilities
- Spring-friendly abstractions for MCP clients and servers
- Both synchronous and asynchronous MCP server operation modes
- Add MCP client autoconfiguration with support for STDIO, WebMVC and WebFlux transports
- Auto-configuration for MCP server components
- Spring Boot starter (spring-ai-starter-mcp) with WebFlux and WebMVC support
- MCP dependency management with BOM
- Add close() method to McpToolCallback for proper resource cleanup
- Add initialize flag to control MCP client initialization
- Add comprehensive integration tests and documentation for MCP client configuration
Technical Improvements:
- Split WebMvc and WebFlux configurations into separate auto-configuration classes
- Server type configurable via 'spring.ai.mcp.server.type' property (SYNC/ASYNC)
- Comprehensive test coverage including McpServerAutoConfigurationIT
- Utility classes for converting between Spring AI tools and MCP tools
- MCP SDK version management in parent pom
Reorganize MCP tool utilities and client configuration
- Rename ToolUtils to McpToolUtils for better MCP-specific naming
- Rename McpToolCallbackProvider to SyncMcpToolCallbackProvider
- Add utility methods for handling tool callbacks in McpToolUtils
- Extract client configuration logic into new McpClientDefinitions class
- Add tool callback support to ChatClient interface and implementations
- Remove redundant integration test
Introduce MCP client customization support
- Add McpSyncClientCustomizer interface for customizing MCP sync clients
- Replace McpClientDefinitions with McpSyncClientConfigurer
- Refactor MCP client initialization to support customization
- Remove redundant close() method from McpToolCallback
- Fix conditional class dependencies in WebMvc/Flux configurations
Add MCP AOT hints
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Introduces a new API key interface that allows users to customize how
API keys are provided and managed in their Spring AI applications. This
change improves security and flexibility by:
- Adding core ApiKey interface and SimpleApiKey implementation
- Adding builder pattern for OpenAiApi creation
- Deprecating public constructors in favor of builder API (since 1.0.0.M6)
- Added docs
The new system enables users to implement their own key management
strategies while maintaining backward compatibility with property-based
configuration.
Introduce a utility class `LoggingMarkers` providing an SLF4J marker for tagging log entries with Personally Identifiable Information (PII). Update `BeanOutputConverter` to use the `PII_MARKER` in error logs for invalid JSON conversions. Enhance tests to verify PII marker usage in logging.
- Set Java version dynamically and configure Kotlin compiler
- Updated Maven configurations to dynamically reference the Java version using `${java.version}`. Added Kotlin compiler settings, including `jvmTarget` alignment with Java version and enabling `javaParameters`. This ensures consistency and better compatibility across builds.
- Fix log assertion in BeanOutputConverterTest to use Java 17
- Updated the test to assert log size explicitly before accessing the first log entry. This ensures the test is more robust and avoids potential issues with accessing logs unexpectedly.
- Use placeholders in logger.error to prevent string concatenation.
- Replaced string concatenation with a placeholder in the logger.error call to improve performance and maintain consistency with logging best practices. This helps avoid unnecessary overhead when logging is disabled.
- Update logging markers and improve data classification
- Replaced `PII_MARKER` with `SENSITIVE_DATA_MARKER`. Introduced `RESTRICTED_DATA_MARKER`, `REGULATED_DATA_MARKER` and `PUBLIC_DATA_MARKER`
- Updated associated logging logic and tests to reflect these changes.
- Fix punctuation in Javadoc comments for LoggingMarkers.
- Added missing periods to improve consistency and clarity in the Javadoc comments. This change ensures proper formatting and adheres to standard writing conventions.
Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
* Completed new documentation for Tool Calling
* Added deprecation notes and migration guide to documentation
* Made “call” methods explicit in ToolCallback API
* Consolidated naming: ToolCallExceptionConverter -> ToolExecutionExceptionProcessor
* Consolidated naming: ToolCallResultConvert.apply() -> ToolCallResultConvert.convert()
* Redraw diagrams for consistency
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Introduced new ToolParam annotation for defining a description for tool parameters and marking them as (non)required.
* Improved the JSON Schema generation for tools, solving inconsistencies between methods and functions, and ensuring a predictable outcome.
* Added support for returning tool results directly to the user instead of passing them back to the model. Introduced new ToolExecutionResult API to propagate this information.
* Consolidated naming of tool-related options in ToolCallingChatOptions.
* Fixed varargs issue in ChatClient when passing ToolCallback[].
* Introduced new documentation for the tool calling capabilities in Spring AI, and deprecated the old one.
* Bumped jsonschema dependency to 4.37.0.
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Adds getNativeClient API to VectorStore interface allowing access to the underlying native client implementation.
This change:
- Adds getNativeClient() default method to VectorStore interface returning Optional<T>
- Implements getNativeClient() in all vector store implementations exposing their respective native clients
- Adds integration tests verifying native client access for all implementations
Fixes: #2137
Signed-off-by: Soby Chacko <soby.chacko@broadcom.com>
* Adopted new tool calling logic in OllamaChatModel, while maintaining full API backward compatibility thanks to the LegacyToolCallingManager.
* Improved efficiency and robustness of merging options in prompts for Ollama.
* Update Ollama Autoconfiguration to use the new ToolCallingManager.
* Improved troubleshooting for new tool calling APIs and finalised changes for full backward compatibility.
* Updated Ollama Testcontainers dependency to 0.5.7.
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Refactors the test codebase to use tools instead of functions.
- Rename FunctionCallback to FunctionToolCallback
- Rename FunctionCallingOptions to ToolCallingChatOptions
- Update API methods from functions() to tools()
- Deprecate function-related methods in favor of tool alternatives
- Refactor MethodToolCallback implementation with improved builder pattern
- Update all tests to use new tool-based APIs
- Add funcs to tools migration guide
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
* Introduced ToolCallingManager to manage the tool calling activities for resolving and executing tools. A default implementation is provided. It can be used to handle explicit tool execution on the client-side, superseding the previous FunctionCallingHelper class. It’s ready to be instrumented via Micrometer, and support exception handling when tool calls fail.
* Introduced ToolCallExceptionConverter to handle exceptions in tool calling, and provided a default implementation propagating the error message to the chat morel.
* Introduced ToolCallbackResolver to resolve ToolCallback instances. A default implementation is provided (DelegatingToolCallbackResolver), capable of delegating the resolution to a series of resolvers, including static resolution (StaticToolCallbackResolver) and dynamic resolution from the Spring context (SpringBeanToolCallbackResolver).
* Improved configuration in ToolCallingChatOptions to enable/disable the tool execution within a ChatModel (superseding the previous proxyToolCalls option).
* Added unit and integration tests to cover all the new use cases and existing functionality which was not covered by autotests (tool resolution from Spring context).
* Deprecated FunctionCallbackResolver, AbstractToolCallSupport, and FunctionCallingHelper.
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Remove model specific Usage implementations
- Add `Object getNativeUsage()` to Usage interface
- This will allow the model specific Usage data to be returned
- At the client side, client needs to cast the return type of `getNativeUsage` into the corresponding Usage returned by the model API
- Rename `generationTokens` to `completionTokens`
- Since `completion` token name is more common among the models, renaming generation tokens into completion tokens
- Maintain JSON deserialization compatibility for legacy `generationTokens` field
- Remove deprecated Long-based constructors to avoid API ambiguity
- Change the prompt, completion and total token return types from Long to Integer
- This is a breaking change that requires updating all constructor calls
- Integer is sufficient for token counts and aligns better with most model APIs
- Use DefaultUsage for most of the model specific usage handling
- When initializing set the native usage to the model specific usage type
- Ensure immutability by making all fields final and removing setters
- Add comprehensive test coverage for all functionality including edge cases
Resolves#1407
* Enhanced support for functions as tools via FunctionToolCallback (deprecating the existing FunctionInvokingFunctionCallback).
* Aligned JSON Schema generation and parsing logic between function-based and method-based tools.
* Deprecated previous client-side function calling APIs.
* Included AOT configuration for Tool-annotated methods in Spring beans.
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Add string-based filter deletion alongside the Filter.Expression-based deletion
for vector stores. This provides a more convenient API for simple filter cases
while maintaining the full flexibility of expression-based filtering.
Key changes:
- Add delete(Filter.Expression) and delete(String) methods to VectorStore
- Add default string filter implementation in VectorStore interface
- Implement filter deletion in Chroma/Elasticsearch/PgVector stores
- Add integration tests for both filter APIs across implementations
This extends vector store deletion capabilities while maintaining consistent
behavior across implementations.
* Defined new APIs consolidating the “tool” naming as opposed to the current “function”, aligning with the industry and solving the confusion between “function tool” and “Java Function”: ToolCallback and ToolCallingChatOptions. They extend the current ones to ensure backward compatibility, but FunctionCallback and FunctionCallingOptions can be considered deprecated.
* Enhanced support for methods as tools, introducing support for declarative Tool-annotated methods via MethodToolCallback and MethodToolCallbackProvider (deprecating the existing MethodInvokingFunctionCallback).
* Improved tool execution logic with granular support for returning the result directly to the client and exception handling.
* Improved JSON Schema generation and parsing logic, consolidating the usage of the victools/jsonschema-generator library and dropping the non-maintained Jackson JSON Schema Module. This makes it possible to use tools with input lists/arrays, which the latter library was not supporting.
* Extended ChatClient API with new methods tools() and toolCallbacks(). The existing functions() methods can be considered deprecated.
Relates to gh-2049
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Extend Query with conversation history in RetrievalAugmentationAdvisor
* Add integration tests for query compression and rewrite
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Moving BatchingStrategy configuration from individual vector store implementations
to the base AbstractVectorStoreBuilder to reduce code duplication and provide consistent
batching behavior across all vector stores.
The default TokenCountBatchingStrategy is now set in the base builder class.
* Remove use of Document.getContext method from spring-ai-core, use getText
* Remove deprecated ChatOptionsBuilder class
* Remove deprecated FunctionCallingOptionsBuilder class
