- Integrate ObservationRegistry and EmbeddingModelObservationConvention
- Update TransformersEmbeddingModel to use observations
- Add TransformersEmbeddingModelObservationTests
- Update TransformersEmbeddingModelAutoConfiguration for observation support
- Add ONNX to AiProvider enum
- Adding observability for Vertex Gemini chat model streaming call
- Updating auto-configuration for gemini chat model with the observability components
- Integration test for streaming observability
- Implement observation instrumentation for AzureOpenAiEmbeddingModel
- Create AzureOpenAiEmbeddingModelObservationIT for integration testing
- Update AzureOpenAiAutoConfiguration to support observation
- Introduce ObservationRegistry and observation conventions
- Integrate ObservationRegistry for instrumentation in VertexAiGeminiChatModel's non-streaming call
- Add ChatModelObservationContext and related classes for observation
- Refactor createGeminiRequest method to support runtime options for imperative calls
- Implement observation for non-streaming chat operations in VertexAiChatModelObservationIT
- Add VertexAiGeminiConstants class for provider name
This commit enhances the VertexAiGeminiChatModel with observability features
for the non-streaming, imperative call() method. It allows for better
monitoring and instrumentation of synchronous chat operations. The changes
include refactoring to improve runtime option handling for these calls and
adds integration tests for the new observability features.
Improve clarity of instructions for model responses. The changes:
- Specify no leading or trailing text in the output
- Reword for more direct and unambiguous instructions
Fixes#718
- Replaced `Map<String, Object>` with `ToolContext` in the `FunctionCallback`, `AbstractFunctionCallback`, and related classes.
- Updated all BiFunction definitions to use `ToolContext` as the second parameter,
enhancing the clarity and structure of the tool context management.
- Modified `FunctionCallbackWrapper` and `FunctionCallbackContext` to adapt to the new `ToolContext` parameter.
- Adjusted the handling of tool context in the documentation and test classes.
- Updated relevant test cases to reflect the API changes and modified the function handling logic to ensure consistency.
- Introduce FactCheckingEvaluator class for LLM response validation
- Implement evaluation logic using ChatClient for fact-checking
- Add comprehensive JavaDoc explaining the evaluator's purpose and usage
- Reference Bespoke-Minicheck model for efficient implementation options
- Include links to Ollama blog post and MiniCheck research paper
- Distinguish from 'closed book' scenario testing in documentation
This new evaluator enables detection and reduction of hallucinations
in LLM outputs by checking claims against provided context. It provides
a foundation for implementing advanced fact-checking methodologies in
Spring AI applications.
See https://ollama.com/blog/reduce-hallucinations-with-bespoke-minicheck
- Introduce toolContext to ChatClient, DefaultChatClient, and AdvisedRequest
- Add methods to set and manage toolContext via the FunctionCallingOptions
- Update tests to include toolContext in relevant scenarios
- Implement toolContext handling in function calling options
This enhancement allows users to specify a JSON Pointer to extract
specific parts of a JSON document when reading. Key changes include:
- New get(String pointer) method in JsonReader class
- Updated JsonReaderTests with pointer-based extraction tests
- Added documentation for JSON Pointer usage in etl-pipeline.adoc
This feature enables more flexible parsing of complex JSON structures,
allowing users to easily target nested data for extraction.
- Improve ChromaVectorStore to throw exception on missing collection with disabled schema
- Add test case to verify exception is thrown when collection doesn't exist and schema init is false
- Correctly set Chroma collection id when initializeSchema set to false
Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
This change allows the ElasticsearchVectorStoreAutoConfiguration
class to be excluded, enabling custom configurations. It brings
consistency with other vector store auto-configuration classes.
- Resolved issue where index name was not being sent during
similaritySearch
- Updated similaritySearch method to include index in the SearchRequest
- Implemented test to verify documents can be added and retrieved from
two different indices using separate OpenSearchVectorStore instances
- Ensured similarity search results are correctly returned for the
respective indices
Fixes#885
* Update attributes
* Add more information on optional attributes and risks
* Include documentation for events.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Unify image definition for vector stores in vector-store modules
* Unify image definition for vector stores in spring-ai-testcontainers module
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Make ChatClient and Advisor observation logic null-safe
* Simplify naming for Advisor observations
* Include high-cardinality attributes only if a value is present
* Fix condition to include system test to chat client observations
* Add Advisor order information to context
* Streamline usage of enums and utils to reduce hard-coded/duplications
* Fix pending Chroma integration test
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Unlike other metrics implementations supported by Micrometer, when using the Prometheus integration, all metric attributes are supposed to have a value, or else the related metrics are dropped.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Vector store observations support several key-value pairs, coming from the Spring AI abstractions. Currently, whenever a value is not available (either because not configured by the user or not supported by the vector store provider), span/metrics attributes are generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and non-compliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for vector store observations to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Chat model observations support several key-value pairs, coming from the Spring AI abstractions.
Currently, whenever a value is not available (either because not configured by the user or not supported by the model provider), a span/metrics is generated anyway with value none.
That causes several issues, including an unneeded increase in time series, challenges in alerting/monitoring (especially for integer/double attributes that suddenly are populated with a string), and incompliance with the OpenTelemetry Semantic Conventions (according to which, attributes should be excluded altogether if there's no value).
This pull request changes the conventions for chat model observation to exclude the generation of span/metrics attributes for optional values which don't have any value.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit removes the duplicate entry of
'spring.ai.vectorstore.elasticsearch.initialize-schema' property from
the Elasticsearch Vector Store documentation.
This commit adds support for tool context in various chat options classes across
different AI model implementations and enhances function calling capabilities.
The tool context allows passing additional contextual information to function callbacks.
- Add toolContext field to chat options classes
- Update builder classes to support setting toolContext
- Enhance FunctionCallback interface to support context-aware function calls
- Update AbstractFunctionCallback to implement BiFunction instead of Function
- Modify FunctionCallbackWrapper to support both Function and BiFunction and
to use the new SchemaType location
- Add support for BiFunction in TypeResolverHelper
- Update ChatClient interface and DefaultChatClient implementation to support
new function calling methods with Function, BiFunction and FunctionCallback arguments
- Refactor AbstractToolCallSupport to pass tool context to function execution
- Update all affected <Model>ChatOptions with tool context support
- Simplify OpenAiChatClientMultipleFunctionCallsIT test
- Add tests for function calling with tool context
- Add new test cases for function callbacks with context in various integration tests
- Modify existing tests to incorporate new context-aware function calling capabilities
- Add docs in in openai function calling
Resolves#864, #1303, #991
Implement observability for imperative calls in AzureOpenAiChatModel:
* Integrate ObservationRegistry to track metrics and events
* Create observation context and define conventions for consistency
* Add integration test to verify chat model observations
* Update AiProvider enum with new AZURE_OPENAI entry
Add support for PortableFunctionCallingOptions across AI models
- Modify FunctionCallingOptions interface to extend ChatOptions for better integration
- Refactor option handling in chat models to accommodate both ChatOptions and FunctionCallingOptions
- Implement handling of FunctionCallingOptions in Anthropic, Azure OpenAI,
MistralAI, Ollama, OpenAI, VertexAI Gemini, and other models
- Update existing function calling tests to use new FunctionCallingOptions.
Resolves#624
- Useful for not Web apps.
- Introduce ErrorLoggingObservationHandler for tracing errors across various AI contexts
- Add error logging configuration option to ChatObservationProperties
- Include ErrorLoggingObservationHandler bean in ChatObservationAutoConfiguration
- Update docs
Resolves#1440
This commit addresses the NPE issue in TextReader's source metadata
handling. It introduces a new method getResourceIdentifier() to
robustly extract identifiers from various Resource types.
The fix ensures that:
1. Filename is used if available
2. Falls back to URI, then URL if filename is not present
3. Uses resource description as a last resort
Additionally, the commit includes updated tests to verify the behavior
with different Resource types, particularly ByteArrayResource
This change prevents NPEs when dealing with Resources that lack
certain properties, improving the overall reliability of TextReader.
Fixes https://github.com/spring-projects/spring-ai/issues/1386
