This change updates the type of portable chat options from Float to
Double. Affected options include:
- frequencyPenalty
- presencePenalty
- temperature
- topP
The motivation for this change is to simplify coding. In Java, Float
values require an "f" suffix (e.g., 0.5f), while Double values don't
need any suffix. This makes Double easier to type and reduces
potential errors from forgetting the "f" suffix.
APIs, tests, and documentation have been updated to reflect this
change.
Fixes gh-712
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Refactoring and enhancements to the advisor functionality.
New Advisor interfaces in the org.springframework.ai.chat.client.advisor.api package:
- Advisor: Base interface for all advisor types.
- RequestAdvisor: For advising on request data before execution.
- ResponseAdvisor: For advising on response data after execution, with enhanced streaming modes.
- CallAroundAdvisor and StreamAroundAdvisor: For around advice on synchronous and streaming requests respectively.
- AroundAdvisorChain and DefaultAroundAdvisorChain: To manage chaining of around advisors.
Advisor Chain and Prompt Generation:
- Added the DefaultAroundAdvisorChain class to manage the sequence of advisors applied around chat model methods.
- Adjusted the prompt generation (toPrompt) to integrate with the refactored AdvisedRequest object.
Refactoring and Updates:
- Replaced the deprecated RequestResponseAdvisor interface with RequestAdvisor and ResponseAdvisor across the spring-ai-core and test modules.
- Updated the DefaultChatClient and related classes to use the new Advisor interface, improving modularity and consistency.
- Refactored DefaultAdvisorSpec and DefaultChatClientRequestSpec to handle the new Advisor type, and revised advisor lists and methods accordingly.
- Enhanced the handling of streaming responses, introducing StreamResponseMode for better control during streaming scenarios.
* Fixes an issue with advisor name resolution
* Streamlines repeating code
* Add a new advisor strategy for ON_FINISH_REASON streaming responses, which is used by the Q&A advisor
* Improve observable instrumentation by passing the parent observation to the advisor observation
* Consolidate usage of “db.collection.name” attribute to track table name, collection name, index name, document name, or whatever concept a vector database uses to store data. Removed “db.index” that was use sometimes instead of “db.collection.name”. This usage is in line with the OpenTelemetry Semantic Conventions.
* Configure query response content to be included as a “span event” instead of a “span attribute” if the backend system supports that, similar to how we do for the model observations.
* Structure vector store observation attributes in dedicated enums, including one for the Spring AI Kinds to avoid hard-coding the same value in a lot of places. This follows the OpenTelemetry Semantic Conventions as much as possible. Also, adopt Spring usual non-null-by-default strategy as much as possible.
* Align vector store conventions to the chat model ones, and follow alphabetical order for values. This is particularly useful for the convention classes, for which the Micrometer performance of exporting telemetry data improves when key values are added already sorted to the context.
* Fix flaky test in Mistral AI.
* Improve Qdrant integration tests.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Enhance JsonReader to detect and process both JSON objects and arrays at
the root level. This update introduces type checking for the JSON root node
using JsonNode and leverages the Stream API to handle JSON arrays.
- Refactor parsing logic to support both JSON arrays and objects
- Rename variable: resourceArray to arrayResource for clarity
- Convert for-loop to Stream API for processing JSON keys
- Add additional tests to ensure proper array handling
- When embedding documents, allow batching the documents using some criteria.
- `BatchingStrategy` interface with a `TokenCountBatchingStrategy` implementation that uses
the openai max input token size of 8191 as the default.
- Add a default method in EmbeddingModel to embed document using this new batching strategy.
- Change `MilvusVectorStore` to make use of this new batching API.
- Adding unit tests for `TokenCountBatchingStrategy`.
- Adding openai integration test to call the embed API that uses batching.
Resolves https://github.com/spring-projects/spring-ai/issues/1214
Other vector stores will be updated seperately
Implementation:
- Introduce AbstractObservationVectorStore with instrumentation for add, delete, and similaritySearch methods
- Create VectorStoreObservationContext to capture operation details
- Implement DefaultVectorStoreObservationConvention for naming and tagging
- Add VectorStoreObservationDocumentation for defining observation keys
- Create VectorStoreObservationAutoConfiguration for auto-configuring observations
- Add VectorStoreObservationProperties to control optional observation content filters
- Update VectorStore interface with getName() method
- Modify PgVectorStore and SimpleVectorStore to extend AbstractObservationVectorStore
- Add vector_store Spring AI kind
Filters:
- Implement VectorStoreQueryResponseObservationFilter
- Add VectorStoreDeleteRequestContentObservationFilter and VectorStoreAddRequestContentObservationFilter
Enhancements:
- Update PgVectorStoreAutoConfiguration to support observations
- Add observation support to PgVectorStore's Builder
- Add VectorStoreObservationContext.Operation enum with ADD, DELETE, and QUERY options
Tests:
- Add tests for VectorStore context, convention, and filters
- Add VectorStoreObservationAutoConfiguration tests
- Add PgVectorObservationIT
Resolves#1205
- Implement observable chat responses in DefaultChatClient
- Add ChatClientObservationContext and related classes for metrics
- Update ChatClient and builder methods to support ObservationRegistry
- Enhance RequestResponseAdvisor with getName() method
- Add ChatClient streaming observability support
- Introduce ChatClientObservationDocumentation for metric key names
- Create DefaultChatClientObservationConvention for implementing conventions
- Add ChatClientInputContentObservationFilter for optional input content logging
- Update ChatClientAutoConfiguration to include new observation components
- Extend ChatClientBuilderProperties with observation configuration options
- Add unit tests for new observation classes and configurations
- Update AiOperationType and AiProvider enums with new values
- Implement safeguards and warnings for sensitive data in observations
Resolves#1206
* Surface more configuration APIs to ChatOptions
* Use abstraction in Observations directly instead of dedicated implementation
* Simplify metadata config in observations for defined models
* Improve merging of runtime and default options in OpenAI
* Fix missing option in Mistral AI
Relates to gh-1148
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Added support for OpenAI's structured outputs feature, which allows specifying a JSON schema for the model to match
- Introduced new record to configure the desired response format
- Added support for configuring the response format via application properties or the chat options builder
- Extend teh BeanOutputConverter to help generate JSON schema from a target domain object and convert the response.
- Added comprehensive tests to cover the new response format functionality
Resolves#1196
* Add model and dimensions to option abstraction
* Use abstraction in Observations directly instead of dedicated implementation
* Clean-up the merge of runtime and default embedding options in OpenAI
Relates to #gh-1148
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Add style to option abstraction
* Use abstraction in Observations directly instead of dedicated implementation
* Clean-up the merge of runtime and default image options in OpenAI and Stability AI
Related to #1148
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Observation APIs for chat, embedding and image models
* Conventions based on OpenTelemetry Semantic Conventions for GenAI
* Instrumentation for OpenAI chat, embedding, and image models
* Autoconfiguration for observability for OpenAI
Fixes gh-953
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Remove inheritance from HashMap
* No more subclasses per model provider
* Builder class for ChatResponse
* Fix the AbstractResponseMetadata#AI_METADATA_STRING parameter order
* ChatResponseMetadata ignore Null values.
- add new spring-ai-vertex-ai-embedding project.
- add VertexAiTextEmbeddingModel and VertexAiMultimodalEmbeddingMode with related options configuration classes.
- add ITs
- add auto-configuraiton and boot starters.
- register to BOM.
- add documentation.
- add multimodal embedding documentation
- extend the Embedding metdata so that it can keep references to the source document's data, Id, mediatype
Resolves#1013
Related to #1009
ChatClient already provides the .chatResponse() method to return the entire ChatResponse instance.
It also provides a set of overloaded .entity(Type) methods to provide Type-converted responses.
The new .responseEntity(Type) method returns a ResponseEntity<ChatResponse, T> instance, encapsulating
both the ChatResponse and the requested Type-converted response entity.
This change allows for more flexibility when handling different response types and facilitates
easier integration with other components that expect ResponseEntity instances.
- Add RequestResponseAdvisor interface with adviseReqeust and adviseResponse methods.
The adviseRequest method takes and returns AdvisedRequest.
The adviseResponse method takes and returns ChatRequest.
- Add ChatClient#ChatClientRequets advisor(...) methods to register advisros.
ChatClient call the registered advisors in order before sealing the ChatClientRequest into a Prompt and call the model
and after the model response.
- Implement PromptChatMemoryAdvisor that uses the ChatMemory and the systemem prompt.
- Implement MessageChatMemoryAdvisor that usees the ChatMemory and the prompt messages.
- Implement VectorStoreChatMemoryAdvisor that uses VecrStore for long term message history.
- Add tests.
- Add shared context to the RequestResponseAdvisor's flow, Context is shared between the request and the response
- Add advisor parameters that are passed through the context
- add support for custom StructuredOutputConverters usng entity(outputConverterInstance)
- add mutate() method that returns ChatClient.Builder to create a new ChatClient whose
settings are replicated from the ChatClient's default settings.
- add prompt().mutate() method that returns ChatClient.Builder to create a new ChatClient
whose settings are replicated from the current default and prompt settings.
* Move ChatClient and related classes into the chat.client package
* Move ChatModel and related class into the chat.model package
* Smaller refactorings to remove DSM cycles
* Update README.md
* Rename the ModelClient class hierarchy into Model:
- Rename ModelClient into Model. Update all code and doc references.
- Rename ChatClient to ChatModel. Update all ChatClient suffixes and chatClient fields and variables in code and doc.
- Rename EmbeddingClient into EmbeddingModel. Update the XxxEmbeddingClient class and variable suffixes and embeddingClient variables and fields in code and docs.
- Rename ImageClient into ImageModel.
- Rename SpeechClient into SpeechModel.
- Rename TranscriptionClient into TranscriptionModel.
- Update all javadocs and antora pages. Update the related diagrams.
* Create fluent API in ChatClient interface that now includes streaming support
* Add OpenAI FunctionCallbackWrapper2IT auto-config tests.
* Add ChatClientTest mockito testing.
* Add ChatModel#getDefaultOptions(), and remove @FunctionalInterface
* ChatModel enums extend the new ModelDescription interface.
* Implement fromOptions copy method in every ChatOptions implementation.
* Extend ChatClient to use the model default options if not provided explicitly.
* Update readme to provide guidance on how to adapt to breaking changes.
Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
Co-authored-by: Mark Pollack <mpollack@vmware.com>
Add ParametrizedTypeReference constructoers along the Clas<T> such.
Convert the Class into ParametrizedTypeReference internally.
Update tests and docs.
* Add ChatBot and basic DefaultChatBot
* Add streaming ChatBot support.
* Add Evaluator interface and RelevancyEvaluator implementation
* Add Content data type abstraction for Document and Message
* Renaming and package refactoring
* update .gitignore to allow node package name
* Add List<Media> to node and move ai.transformer package to ai.prompt.transformer
* Add Short/Long term memory support.
* Add mixing transformers support
Docs TBD
- Old OutputParser, BeanOutputParser, ListOutputParser and MapOutputParser classes are depredated
in favour of the new StructuredOutputConverter, BeanOutputConverter, ListOutputConverter and
MapOutputConverter implementations.
Later are drop-in replacements for the former ones, and provide the same functionality.
- Keep the existing parser package and classes for backward compatibility.
- Adjust the PromptTemplate for backward compatibility
- Update all existing tests to use the new Structured Output API.
- Improve the documentation for structured outputs.
