Prevent external modification of internal state by returning unmodifiable
collections and adding defensive copies in setters and builders.
Co-authored-by: youngmoneee@users.noreply.github.com
- Adds special handling for BiFunction class types
- Adds test cases to verify BiFunction class type resolution
- Removes deprecated AbstractFunctionCallSupport class
- Cleans up unused imports in FunctionCallback
Resolves#1576
- Convert TokenTextSplitter configuration parameters to final fields
- Add static Builder class for fluent construction
- Define default constants for configuration values
- Add unit tests validating builder and default configurations
- Test metadata handling and content splitting behavior
- Implement core vector store module for CosmosDB integration
- Add Spring Boot auto-configuration capabilities
- Integrate batch processing strategy for optimized operations
- Include comprehensive tests for core and auto-config modules
- Add reference docs for the CosmosDB vector store support
- Update TextSplitter to filter out metadata entries with null keys or values
- Simplify text substring call in TextSplitterTests
- Add test case for document splitting with null metadata values
- Remove unnecessary spring-web dependencies
- Update third-party library versions
- Refactor API classes to use consistent header handling
- Remove ApiUtils class and inline its functionality
- Adjust RestClient and WebClient builder usage in autoconfiguration
- Replace direct RestClient.Builder injections with ObjectProvider<RestClient.Builder>
and WebClient.Builder injections with ObjectProvider<WebClient.Builder>
- Update ChromaVectorStoreAutoConfiguration to use ObjectProvider
- Rename MongoDbAtlasLocalContainerConnectionDetailsFactoryTest to IT
- Switch spring-ai-chroma-store dependency from spring-web to spring-webflux
- Simplify ChromaApi constructor by using method reference for default headers
- Adjust import order
Resolves#1066Resolves#524
- Update ChatResponse.Builder to copy all metadata fields when using from()
- Expand test case to verify correct metadata copying in QA advisor
Resolves#1537
This commit enhances the Bedrock Anthropic model's output:
- Add response ID, model name, and usage data to ChatResponseMetadata
- Introduce DefaultUsage class for token usage information
- Update BedrockAnthropic3ChatModel to include new metadata
- Add Jackson annotations for serialization/deserialization
- Implement unit tests for DefaultUsage
These changes provide structured, serializable metadata in the
ChatResponse, improving the model's output with additional
information.
- Add line breaks and clarify context boundaries in user text advice.
This improve the performance of a Llama3.x
- Update corresponding test to reflect new formatting
- Introduce builder pattern for MessageChatMemoryAdvisor, PromptChatMemoryAdvisor,
QuestionAnswerAdvisor, SafeGuardAdvisor, and VectorStoreChatMemoryAdvisor.
- Add 'order' parameter to control advisor execution priority
- Modify constructors to include the new 'order' parameter
- Update AbstractChatMemoryAdvisor to support the new 'order' parameter
- Update docs
- Add customizable failure response instead of empty result
- Add order parameter for advisor prioritization
- Implement default values for failure response and order
- Introduce builder pattern for flexible configuration
Resolves#1393
- Integrate ObservationRegistry and EmbeddingModelObservationConvention
- Update TransformersEmbeddingModel to use observations
- Add TransformersEmbeddingModelObservationTests
- Update TransformersEmbeddingModelAutoConfiguration for observation support
- Add ONNX to AiProvider enum
Improve clarity of instructions for model responses. The changes:
- Specify no leading or trailing text in the output
- Reword for more direct and unambiguous instructions
Fixes#718
- Replaced `Map<String, Object>` with `ToolContext` in the `FunctionCallback`, `AbstractFunctionCallback`, and related classes.
- Updated all BiFunction definitions to use `ToolContext` as the second parameter,
enhancing the clarity and structure of the tool context management.
- Modified `FunctionCallbackWrapper` and `FunctionCallbackContext` to adapt to the new `ToolContext` parameter.
- Adjusted the handling of tool context in the documentation and test classes.
- Updated relevant test cases to reflect the API changes and modified the function handling logic to ensure consistency.
- Introduce FactCheckingEvaluator class for LLM response validation
- Implement evaluation logic using ChatClient for fact-checking
- Add comprehensive JavaDoc explaining the evaluator's purpose and usage
- Reference Bespoke-Minicheck model for efficient implementation options
- Include links to Ollama blog post and MiniCheck research paper
- Distinguish from 'closed book' scenario testing in documentation
This new evaluator enables detection and reduction of hallucinations
in LLM outputs by checking claims against provided context. It provides
a foundation for implementing advanced fact-checking methodologies in
Spring AI applications.
See https://ollama.com/blog/reduce-hallucinations-with-bespoke-minicheck
- Introduce toolContext to ChatClient, DefaultChatClient, and AdvisedRequest
- Add methods to set and manage toolContext via the FunctionCallingOptions
- Update tests to include toolContext in relevant scenarios
- Implement toolContext handling in function calling options
This enhancement allows users to specify a JSON Pointer to extract
specific parts of a JSON document when reading. Key changes include:
- New get(String pointer) method in JsonReader class
- Updated JsonReaderTests with pointer-based extraction tests
- Added documentation for JSON Pointer usage in etl-pipeline.adoc
This feature enables more flexible parsing of complex JSON structures,
allowing users to easily target nested data for extraction.
* Make ChatClient and Advisor observation logic null-safe
* Simplify naming for Advisor observations
* Include high-cardinality attributes only if a value is present
* Fix condition to include system test to chat client observations
* Add Advisor order information to context
* Streamline usage of enums and utils to reduce hard-coded/duplications
* Fix pending Chroma integration test
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Unlike other metrics implementations supported by Micrometer, when using the Prometheus integration, all metric attributes are supposed to have a value, or else the related metrics are dropped.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Vector store observations support several key-value pairs, coming from the Spring AI abstractions. Currently, whenever a value is not available (either because not configured by the user or not supported by the vector store provider), span/metrics attributes are generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and non-compliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for vector store observations to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Chat model observations support several key-value pairs, coming from the Spring AI abstractions.
Currently, whenever a value is not available (either because not configured by the user or not supported by the model provider), a span/metrics is generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and incompliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for chat model observation to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit adds support for tool context in various chat options classes across
different AI model implementations and enhances function calling capabilities.
The tool context allows passing additional contextual information to function callbacks.
- Add toolContext field to chat options classes
- Update builder classes to support setting toolContext
- Enhance FunctionCallback interface to support context-aware function calls
- Update AbstractFunctionCallback to implement BiFunction instead of Function
- Modify FunctionCallbackWrapper to support both Function and BiFunction and
to use the new SchemaType location
- Add support for BiFunction in TypeResolverHelper
- Update ChatClient interface and DefaultChatClient implementation to support
new function calling methods with Function, BiFunction and FunctionCallback arguments
- Refactor AbstractToolCallSupport to pass tool context to function execution
- Update all affected <Model>ChatOptions with tool context support
- Simplify OpenAiChatClientMultipleFunctionCallsIT test
- Add tests for function calling with tool context
- Add new test cases for function callbacks with context in various integration tests
- Modify existing tests to incorporate new context-aware function calling capabilities
- Add docs in in openai function calling
Resolves#864, #1303, #991
Implement observability for imperative calls in AzureOpenAiChatModel:
* Integrate ObservationRegistry to track metrics and events
* Create observation context and define conventions for consistency
* Add integration test to verify chat model observations
* Update AiProvider enum with new AZURE_OPENAI entry
Add support for PortableFunctionCallingOptions across AI models
- Modify FunctionCallingOptions interface to extend ChatOptions for better integration
- Refactor option handling in chat models to accommodate both ChatOptions and FunctionCallingOptions
- Implement handling of FunctionCallingOptions in Anthropic, Azure OpenAI,
MistralAI, Ollama, OpenAI, VertexAI Gemini, and other models
- Update existing function calling tests to use new FunctionCallingOptions.
Resolves#624
- Useful for not Web apps.
- Introduce ErrorLoggingObservationHandler for tracing errors across various AI contexts
- Add error logging configuration option to ChatObservationProperties
- Include ErrorLoggingObservationHandler bean in ChatObservationAutoConfiguration
- Update docs
Resolves#1440
This commit addresses the NPE issue in TextReader's source metadata
handling. It introduces a new method getResourceIdentifier() to
robustly extract identifiers from various Resource types.
The fix ensures that:
1. Filename is used if available
2. Falls back to URI, then URL if filename is not present
3. Uses resource description as a last resort
Additionally, the commit includes updated tests to verify the behavior
with different Resource types, particularly ByteArrayResource
This change prevents NPEs when dealing with Resources that lack
certain properties, improving the overall reliability of TextReader.
Fixes https://github.com/spring-projects/spring-ai/issues/1386
This commit introduces a major overhaul of the advisor system in Spring AI,
improving modularity, type safety, and consistency
Core Changes:
- Replace RequestAdvisor and ResponseAdvisor with CallAroundAdvisor and StreamAroundAdvisor
- Introduce AdvisedRequest and AdvisedResponse classes for better encapsulation
- Deprecate RequestResponseAdvisor in favor of new advisor types
- Remove AdvisorObservableHelper class
Advisor Implementation Updates:
- Update AbstractChatMemoryAdvisor, MessageChatMemoryAdvisor, PromptChatMemoryAdvisor,
QuestionAnswerAdvisor, SafeGuardAroundAdvisor, SimpleLoggerAdvisor, and
VectorStoreChatMemoryAdvisor to implement new advisor interfaces
- Remove CacheAroundAdvisor (functionality likely moved elsewhere)
- Make CallAroundAdvisor and StreamAroundAdvisor extend Ordered interface
Client and Chain Management:
- Modify DefaultChatClient to use new advisor chain approach
- Refactor DefaultAroundAdvisorChain for better ordering and observation
- Implement builder pattern for advisor chain construction in DefaultChatClient
- Separate call and stream advisors in DefaultAroundAdvisorChain
Observation and Context Handling:
- Update observation conventions and context handling in advisors
- Add order field to AdvisorObservationContext
- Modify DefaultAdvisorObservationConvention to include order in high cardinality key values
Testing and Integration:
- Refactor ChatClientAdvisorTests and add new AdvisorsTests
- Update integration tests to reflect new advisor structure
- Enhance AdvisorsTests to verify correct advisor execution order
New Features:
- Generalize the Protect From Blocking functionality across all advisors
- Add (experimental) Re2 advisor to enhance reasoning capabilities of LLMs
- Add disabled Re2 test in OpenAiChatClientIT
Documentation:
- Add Advisors documentation
- Enhance advisors documentation with order explanation and Re2 example
Advisor Ordering:
- Introduce Advisor constants for precedence ordering
- Update AbstractChatMemoryAdvisor to use new precedence constant
- Improve advisor ordering and management in DefaultAroundAdvisorChain.Builder
- Remove redundant reordering logic from DefaultAroundAdvisorChain
These changes aim to provide a more flexible and powerful advisor system,
allowing for easier implementation of complex AI-driven interactions
Co-authored-by: Dariusz Jędrzejczyk <dariusz.jedrzejczyk@broadcom.com>
This commit adds comprehensive documentation for the BatchingStrategy
in vector stores and enhances the TokenCountBatchingStrategy class.
Key changes:
- Explain batching necessity due to embedding model thresholds
- Describe BatchingStrategy interface and its purpose
- Detail TokenCountBatchingStrategy default implementation
- Provide guidance on using and customizing batching strategies
- Note pre-configured vector stores with default strategy
- Add new constructor for custom TokenCountEstimator in
TokenCountBatchingStrategy
- Implement null checks with Spring's Assert utility
- Update docs with new customization options and code examples
This commit introduces support for Oracle Cloud Infrastructure (OCI)
GenAI embedding models in Spring AI. It includes:
* New OCIEmbeddingModel class for interacting with OCI GenAI API
* Auto-configuration for easy setup and integration
* Properties for configuring OCI connection and embedding options
* Documentation updates explaining usage and configuration
* Integration tests to verify functionality
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
Replace parallelStream with stream to prevent thread-unsafe appends to
the shared StringBuilder. This fixes the issue of intermingled key-value
pairs in the generated Document content. Also, replace StringBuffer
with StringBuilder for better performance in single-threaded context.
The change ensures correct ordering of extracted JSON keys and their
values in the resulting Document, improving the reliability and
readability of the parsed output.
Replace Objects::nonNull and instanceof checks with StringUtils::hasText
for more efficient and cleaner content filtering. This change simplifies
the stream operation in the getContent method, improving readability
and potentially performance.
- When ChatClientRequestSpec#prompt(Prompt) is used, unseal the prompt instance.
Convert the last message instance (if user message) into spec#user and spec#media
and add the remaining messages (excluding the last) to the spec#messages.
Add the prompt#options to the spec#options.
- Improve DefaultChatClient to handle UserMessage media and content separately.
- Update AbstractToolCallSupport to use new hasToolCalls() method.
- Add hasToolCalls() method to AssistantMessage.
- Enhance ChatClientTest with additional test cases for media handling.
- Disable Groq and Nvidia integration tests due to rate limiting and credit requirements.
This commit introduces a new proxyToolCalls option for various chat
models in the Spring AI project. When enabled, it allows the client to
handle function calls externally instead of being processed internally
by Spring AI.
The change affects multiple chat model implementations, including:
AnthropicChatModel
AzureOpenAiChatModel
MiniMaxChatModel
MistralAiChatModel
MoonshotChatModel
OllamaChatModel
OpenAiChatModel
VertexAiGeminiChatModel
ZhiPuAiChatModel
The proxyToolCalls option is added to the respective chat options
classes and integrated into the AbstractToolCallSupport class for
consistent handling across different implementations.
The proxyToolCalls option can be set either programmatically via
the <ModelName>ChatOptions.builder().withProxyToolCalls() method
or the spring.ai.<model-name>.chat.options.proxy-tool-calls
application property.
Documentation for the new option is also updated in the relevant
Antora pages.
Added ITs for proxy tool calls
Remove ChatClientPromptRequestSpec and all ChatClient.prompt() overloads can how take advantage of the full fluent API. Docs updated
Resolves#1367
This commit introduces a new overloaded method prompt(String content)
to the ChatClient API. It provides a convenient way to create prompts
using only a string of user content, simplifying the process for basic
chat interactions.
Fixes gh-1286
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This change updates the type of portable chat options from Float to
Double. Affected options include:
- frequencyPenalty
- presencePenalty
- temperature
- topP
The motivation for this change is to simplify coding. In Java, Float
values require an "f" suffix (e.g., 0.5f), while Double values don't
need any suffix. This makes Double easier to type and reduces
potential errors from forgetting the "f" suffix.
APIs, tests, and documentation have been updated to reflect this
change.
Fixes gh-712
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
