The RelevancyEvaluator is key for validating RAG flows. This pull request improves it by making the PromptTemplate configurable, improving the format of the default one, introducing a Builder, and extending the documentation with more details on how to use it.
I added some unit tests. The RelevancyEvaluator is used in lots of integration tests in the project to test the QuestionAnswerAdvisor and RetrievalAugmentationAdvisor, that also help assessing the evaluator itself.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
The features are disabled by default and users need to explicitly
enable them. Probably it makes more sense to log on INFO level so that
users don't need to change the log level if they use the default
behavior of Spring Boot.
See gh-3003
See gh-3127
- The fix overrides the adviseStream method to use ChatClientMessageAggregator to properly aggregate streaming chunks before storing them in memory, similar to how PromptChatMemoryAdvisor handles streaming responses
- Added test
Signed-off-by: Mark Pollack <mark.pollack@broadcom.com>
* The defaultConversationId was configurable, but not used. It’s now being used correctly when a custom defaultConversationId is defined.
* The memory advisors were missing the required configuration of a Schedule due to a default value missing. Now as default Scheduler is used, automatically protecting from blocking. It can be customised via “scheduler()”, replacing the old “protectFromBlocking()” method.
* The new defaultTopK options in VectorStoreChatMemoryAdvisor were documented, but not implemented. That is fixed now.
* The memory advisors were not null-safe. Now they are.
* Improved tests to check the null-safe behaviour.
* Updated the documentation accordingly.
Fixes gh-3133
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
Improve separation of concerns by keeping model-related functionality in the model module
while maintaining client-specific functionality in the client module.
- Move MessageAggregator class from spring-ai-client-chat to spring-ai-model module
- Create new ChatClientMessageAggregator in spring-ai-client-chat module to handle client-specific aggregation
- Extract client-specific aggregation logic from MessageAggregator to ChatClientMessageAggregator
- Update references in advisor classes to use the new ChatClientMessageAggregator
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
- Remove deprecated ChatMemory.get(String conversationId, int lastN) method
- Replace AbstractChatMemoryAdvisor with BaseChatMemoryAdvisor interface in api package
- Make constructors private in all memory advisor implementations to enforce builder usage
- Rename CHAT_MEMORY_CONVERSATION_ID_KEY to CONVERSATION_ID and move to ChatMemory interface
- In VectorStoreChatMemoryAdvisor:
- Rename DEFAULT_CHAT_MEMORY_RESPONSE_SIZE (100) to DEFAULT_TOP_K (20)
- Rename builder method chatMemoryRetrieveSize() to topK()
- Remove systemTextAdvise() builder method
- In PromptChatMemoryAdvisor:
- Remove systemTextAdvise() builder method
- Fix bug where only the last user message was stored from prompts with multiple messages
- Enhance logging in memory advisors to aid in debugging
- Add comprehensive tests for all advisor implementations:
- Unit tests for builder behavior
- Integration tests for the various chat memory advisors
Signed-off-by: Mark Pollack <mark.pollack@broadcom.com>
This removes the possible large amount of data that was attached
to spans and it logs the data out. This change also removes the direct
dependency on the OTel SDK. approach for content in Spring AI.
Refactors the observability approach for content in Spring AI:
- Replace content observation filters with logging handlers
- Rename configuration properties to better reflect their purpose:
- `include-prompt` → `log-prompt`
- `include-completion` → `log-completion`
- `include-query-response` → `log-query-response`
- Add TracingAwareLoggingObservationHandler for trace-aware logging
- Replace micrometer-tracing-bridge-otel with micrometer-tracing
- Remove event-based tracing in favor of direct logging
- Update documentation to reflect these changes (add breaking-changes section)
- Rename includePrompt to logPrompt in observation properties. Updated in ChatClientBuilderProperties, ChatObservationProperties, and ImageObservationProperties.
Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
Remove circular dependencies:
Move utility classes from various packages to dedicated support packages:
- Move ToolCallbacks from ai.tool to ai.support
- Move UsageUtils to UsageCalculator in ai.support
- Move tool.util to tool.support
- Create new ToolDefinitions utility class
Support packages for the classes in question is more idomatic in spring than util packages.
Signed-off-by: Mark Pollack <mark.pollack@broadcom.com>
The built-in advisors that perform prompt augmentation have been updated to use self-contained templates. The goal is for each advisor to be able to perform templating operations without affecting nor being affected by templating and prompt decisions in other advisors.
* QuestionAnswerAdvisor
* PromptChatMemoryAdvisor
* VectorStoreChatMemoryAdvisor
Documentation and upgrade notes have been updated accordingly
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* Update remaining Advisors and related classes to use the new APIs.
* In AbstractChatMemoryAdvisor, the “doNextWithProtectFromBlockingBefore()” protected method has been changed from accepting AdvisedRequest to ChatClientRequest. It’s a breaking change since the alternative was not part of M8.
* MessageAggregator has a new method to aggregate messages from ChatClientRequest. The previous method aggregating messages from AdvisedRequest has been removed. Warning since it wasn’t marked as deprecated in M8.
* In SimpleLoggerAdvisor, the “requestToString” input argument needs to be updated to use ChatClientRequest. It’s a breaking change since the alternative was not part of M8. Same thing about the constructor.
* The “getTemplateRenderer” method has been removed from BaseAdvisorChain. Each Advisor is encouraged to accept a PromptTemplate to achieve self-contained prompt augmentation operations.
* Remove deprecations in ChatClient and Advisors, and update tests accordingly.
* When building a Prompt from the ChatClient input, the SystemMessage passed via systemText() is placed first in the message list. Before, it was put last, resulting in errors with several model providers.
Relates to gh-2655
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Remove deprecated constructor method Media(MimeType mimeType, URL url) from Media
- Remove deprecated builder method data(URL url) from Media builder
- Update references to use the builder and the data(URI) methods
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Remove the deprecated constructors from PromptTemplate and replace the references using the builder methods
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Remove deprecated DocumentCompressor/Ranker/Selector
- These are replaced by the org.springframework.ai.rag.postretrieval.document.DocumentPostProcessor
Remove deprecated Chatclient/DefaultChatClient methods
In ChatClient:
- Remove defaultTools and tools as these are replaced by defaultToolCallbacks and defaultToolNames respectively.
In DefaultChatClient:
- Remove tools method implementation
- Remove unused DefaultCallPromptResponseSpec and DefaultStreamPromptResponseSpec
Remove deprecated addMessage from Prompt
Remove deprecated isInternalToolExecutionEnabled() from ToolCallingChatOptions
- This is replaced by getInternalToolExecutionEnabled()
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
Output format instructions should not be included until the very last advisor runs, otherwise there's a risk of templating failure if more than one advisor tries to render the prompt template. This change guarantees the output format instructions are always included right before calling the chat model, without the risk of previous advisors interfering with it.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Move evaluation core classes (Request, Response, Evaluator) to spring-ai-commons.
- Move chat-specific evaluators (FactChecking, Relevancy) to spring-ai-client-chat#evaluation package.
- Move OutputConverter implementations and related classes to spring-ai-model#converter package.
- Move MetadataEnricher transformer classes to spring-ai-model#transformer package.
- Move PromptAssert utility to spring-ai-rag#util package.
- Update relevant pom.xml files and adjust imports in affected classes.
- Document class movements in upgrade-notes.adoc for M8 release.
This refactoring improves module organization and separation of concerns.
Signed-off-by: Mark Pollack <mark.pollack@broadcom.com>
Renames the ambiguous overloaded `tools` methods in `ChatClient.PromptRequestSpec`
to `toolNames` and `toolCallbacks` respectively. This improves clarity
and prevents potential issues with method dispatching based on argument types.
Updates relevant code examples and adds documentation to upgrade notes.
Signed-off-by: Mark Pollack <mark.pollack@broadcom.com>
This reverts commit 4eeeb83d3f.
Revert "Make ImageOptions non-null when constructing ImagePrompt"
This reverts commit 0c0787b849.
Need to review more the impact of making these options not null.
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Extend the ChatClient with a new templateRenderer() method to pass a custom TemplateRenderer object used to render user and system templates.
- Evolve the QuestionAnswerAdvisor to accept a PromptTemplate for customising the RAG prompt and templating logic while maintaining backward compatibility.
- Introduce integration tests for the QuestionAnswerAdvisor.
- Document the TemplateRenderer API and how to use it to build PromptTemplate with custom templating logic.
- Document how to customise the templating logic used internally by the ChatClient via the TemplateRendererAPI.
Add validation tests and improve PromptTemplate resource handling
Enhance robustness and reliability of the PromptTemplate class with better
resource handling and comprehensive input validation:
- Add dedicated validation tests for builder methods with null/invalid inputs
- Improve renderResource method to gracefully handle edge cases:
- Null resources return empty string
- ByteArrayResource handling with proper charset (UTF-8)
- Empty resources check with proper existence test
- Better error handling with logging instead of exception propagation
- Add input validation assertions to all Builder methods
- Fix typo in deprecated annotation comment ("fahvor" → "favor")
Update documentation to clarify template rendering in different contexts:
- Add clear notes about TemplateRenderer usage in ChatClient vs Advisors
- Document how advisor template customization differs from ChatClient template rendering
- Add comprehensive API upgrade notes for template-related deprecations
- Include detailed migration examples for PromptTemplate and QuestionAnswerAdvisor
Fixes gh-355, gh-1687, gh-2448, gh-1849, gh-1428
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* ChatClient observations now include the full prompt content instead of just the userText and systemText. Furthermore, they include consistent telemetry for the tools passed via the ChatClient and a first-class conversation ID when using memory advisors. Incomplete or unsafe attributes have been deprecated.
* Adopted the new robust Advisor APIs for BaseAdvisor and RetrievalAugmentationAdvisor.
* Improved the prompt augmentation facilities in ChatClientRequest and Prompt for performance and immutability.
* Fixed integration test racing condition.
* Updated the documentation for ChatClient and Observability accordingly.
* Documented changes in upgrade notes.
* Introduced `prompt.augmentUserMessage(String text)` to directly replace the user message content.
* Added `prompt.augmentUserMessage(Function<UserMessage, UserMessage> augmenter)` for more granular updates using the `userMessage.mutate()` pattern, allowing modification of text, media, and metadata.
Relates to gh-2655
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Remove the deprecated FunctionCallback related classes, references
- Replace them with the corresponding ToolCallback usage
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Remove @Nullable on ChatOptions
- By default, pass the default ChatOptions from the DefaultChatOptionsBuilder
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
- Remove deprecation from EvaluationResponse's constructor as it is still being used
Signed-off-by: Ilayaperumal Gopinathan <ilayaperumal.gopinathan@broadcom.com>
The DefaultChatClientObservationConvention is now not removing entries from the observation context, resulting in advisor context entries being removed in the ChatClientRequest.
Fixes gh-2876
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
* ChatMemory will become a generic interface to implement different memory management strategies. It’s been moved from the “”spring-ai-client-chat” package to “spring-ai-model” package while retaining the same package, so it’s transparent to users.
* A MessageWindowChatMemory has been introduced to provide support for a chat memory that keeps at most N messages in the memory.
* A ChatMemoryRepository interface has been introduced to support different storage strategies for the chat memory. It’s meant to be used as part of a ChatMemory implementation. This is different than before, where the storage-specific implementation was directly tied to the ChatMemory. This design is familiar to Spring users since it’s used already in the ecosystem. The goal was to use a programming model similar to Spring Session and Spring Data.
* The JdbcChatMemory has been supersed by JdbcChatMemoryRepository.
* A ChatMemory bean is auto-configured for you whenever using one of the Spring AI Model starters. By default, it uses the MessageWindowChatMemory implementation and stores the conversation history in memory. If a different repository is already configured (e.g., Cassandra, JDBC, or Neo4j), Spring AI will use that instead.
* First-class documentation has been introduced to describe the ChatMemory API and related features.
* All the changes introduced in this PR are backward-compatible.
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Introduce new TemplateRenderer API providing the logic for rendering an input template.
- Update the PromptTemplate API to accept a TemplateRenderer object at construction time.
- Move ST logic to StTemplateRenderer implementation, used by default in PromptTemplate. Additionally, make start and end delimiter character configurable.
Relates to gh-2655
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Introduce “ChatClientRequest” and “ChatClientResponse” for propagating requests/responses in a ChatClient advisor chain.
- Structure a Prompt at the beginning of the chain, to ensure a consistent view across execution chain and observations. Any template is rendered at the beginning so that every advisor doesn’t have to do it again.
- Improve observations to include the complete view of the prompt messages, instead of only considering userText and systemText.
- Remove legacy “around” advisor type concept.
- Keep backward compatibility for AdvisedRequest, AdvisedResponse, and legacy Advisor APIs.
Relates to gh-2655
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
- Avoid overlapping package names
Changed in spring-ai-commons package from org.sf.ai.model to org.sf.ai.content
Refactor advisor module name to be spring-ai-advisors-vector-store
Moved advisors into org.springframework.ai.chat.client.advisor.vectorstore
- Created top level memory directory
- Create new module spring-ai-model-chat-memory-neo4j and moved neo4j memory classes out of the vectorstore module
Updated neo4j autoconfiguation
