Self-contained prompt templates in advisors

The built-in advisors that perform prompt augmentation have been updated to use self-contained templates. The goal is for each advisor to be able to perform templating operations without affecting nor being affected by templating and prompt decisions in other advisors.

* QuestionAnswerAdvisor
* PromptChatMemoryAdvisor
* VectorStoreChatMemoryAdvisor

Documentation and upgrade notes have been updated accordingly

Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit is contained in:
Thomas Vitale
2025-05-08 00:38:32 +01:00
committed by Mark Pollack
parent 2ea518686f
commit d619e25cb5
7 changed files with 95 additions and 48 deletions

View File

@@ -244,6 +244,32 @@ chatClient.prompt()
.content();
----
=== PromptChatMemoryAdvisor
==== Custom Template
The `PromptChatMemoryAdvisor` uses a default template to augment the system message with the retrieved conversation memory. You can customize this behavior by providing your own `PromptTemplate` object via the `.promptTemplate()` builder method.
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved memory with the system message. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
* an `instructions` placeholder to receive the original system message.
* a `memory` placeholder to receive the retrieved conversation memory.
=== VectorStoreChatMemoryAdvisor
==== Custom Template
The `VectorStoreChatMemoryAdvisor` uses a default template to augment the system message with the retrieved conversation memory. You can customize this behavior by providing your own `PromptTemplate` object via the `.promptTemplate()` builder method.
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved memory with the system message. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
* an `instructions` placeholder to receive the original system message.
* a `long_term_memory` placeholder to receive the retrieved conversation memory.
== Memory in Chat Model
If you're working directly with a `ChatModel` instead of a `ChatClient`, you can manage the memory explicitly:

View File

@@ -82,13 +82,18 @@ The `QuestionAnswerAdvisor` uses a default template to augment the user question
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved context with the user query. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain a placeholder to receive the retrieved context, which the advisor provides under the key `question_answer_context`.
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
* a `query` placeholder to receive the user question.
* a `question_answer_context` placeholder to receive the retrieved context.
[source,java]
----
PromptTemplate customPromptTemplate = PromptTemplate.builder()
.renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
.template("""
<query>
Context information is below.
---------------------

View File

@@ -36,6 +36,21 @@ For details, refer to:
* `MessageAggregator` has a new method to aggregate messages from `ChatClientRequest`. The previous method aggregating messages from the old `AdvisedRequest` has been removed, since it was already marked as deprecated in M8.
* In `SimpleLoggerAdvisor`, the `requestToString` input argument needs to be updated to use `ChatClientRequest`. Its a breaking change since the alternative was not part of M8 yet. Same thing about the constructor.
==== Self-contained Templates in Advisors
The built-in advisors that perform prompt augmentation have been updated to use self-contained templates. The goal is for each advisor to be able to perform templating operations without affecting nor being affected by templating and prompt decisions in other advisors.
If you were providing custom templates for the following advisors, you'll need to update them to ensure all expected placeholders are included.
* The `QuestionAnswerAdvisor` expects a template with the following placeholders (see xref:api/retrieval-augmented-generation.adoc#_questionansweradvisor[more details]):
** a `query` placeholder to receive the user question.
** a `question_answer_context` placeholder to receive the retrieved context.
* The `PromptChatMemoryAdvisor` expects a template with the following placeholders (see xref:api/chat-memory.adoc#_promptchatmemoryadvisor[more details]):
** an `instructions` placeholder to receive the original system message.
** a `memory` placeholder to receive the retrieved conversation memory.
* The `VectorStoreChatMemoryAdvisor` expects a template with the following placeholders (see xref:api/chat-memory.adoc#_vectorstorechatmemoryadvisor[more details]):
** an `instructions` placeholder to receive the original system message.
** a `long_term_memory` placeholder to receive the retrieved conversation memory.
=== Breaking Changes
The Watson AI model was removed as it was based on the older text generation that is considered outdated as there is a new chat generation model available.
Hopefully Watson will reappear in a future version of Spring AI