Self-contained prompt templates in advisors
The built-in advisors that perform prompt augmentation have been updated to use self-contained templates. The goal is for each advisor to be able to perform templating operations without affecting nor being affected by templating and prompt decisions in other advisors. * QuestionAnswerAdvisor * PromptChatMemoryAdvisor * VectorStoreChatMemoryAdvisor Documentation and upgrade notes have been updated accordingly Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit is contained in:
committed by
Mark Pollack
parent
2ea518686f
commit
d619e25cb5
@@ -244,6 +244,32 @@ chatClient.prompt()
|
||||
.content();
|
||||
----
|
||||
|
||||
=== PromptChatMemoryAdvisor
|
||||
|
||||
==== Custom Template
|
||||
|
||||
The `PromptChatMemoryAdvisor` uses a default template to augment the system message with the retrieved conversation memory. You can customize this behavior by providing your own `PromptTemplate` object via the `.promptTemplate()` builder method.
|
||||
|
||||
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved memory with the system message. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
|
||||
|
||||
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
|
||||
|
||||
* an `instructions` placeholder to receive the original system message.
|
||||
* a `memory` placeholder to receive the retrieved conversation memory.
|
||||
|
||||
=== VectorStoreChatMemoryAdvisor
|
||||
|
||||
==== Custom Template
|
||||
|
||||
The `VectorStoreChatMemoryAdvisor` uses a default template to augment the system message with the retrieved conversation memory. You can customize this behavior by providing your own `PromptTemplate` object via the `.promptTemplate()` builder method.
|
||||
|
||||
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved memory with the system message. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
|
||||
|
||||
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
|
||||
|
||||
* an `instructions` placeholder to receive the original system message.
|
||||
* a `long_term_memory` placeholder to receive the retrieved conversation memory.
|
||||
|
||||
== Memory in Chat Model
|
||||
|
||||
If you're working directly with a `ChatModel` instead of a `ChatClient`, you can manage the memory explicitly:
|
||||
|
||||
@@ -82,13 +82,18 @@ The `QuestionAnswerAdvisor` uses a default template to augment the user question
|
||||
|
||||
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved context with the user query. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
|
||||
|
||||
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain a placeholder to receive the retrieved context, which the advisor provides under the key `question_answer_context`.
|
||||
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain the following two placeholders:
|
||||
|
||||
* a `query` placeholder to receive the user question.
|
||||
* a `question_answer_context` placeholder to receive the retrieved context.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
PromptTemplate customPromptTemplate = PromptTemplate.builder()
|
||||
.renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
|
||||
.template("""
|
||||
<query>
|
||||
|
||||
Context information is below.
|
||||
|
||||
---------------------
|
||||
|
||||
@@ -36,6 +36,21 @@ For details, refer to:
|
||||
* `MessageAggregator` has a new method to aggregate messages from `ChatClientRequest`. The previous method aggregating messages from the old `AdvisedRequest` has been removed, since it was already marked as deprecated in M8.
|
||||
* In `SimpleLoggerAdvisor`, the `requestToString` input argument needs to be updated to use `ChatClientRequest`. It’s a breaking change since the alternative was not part of M8 yet. Same thing about the constructor.
|
||||
|
||||
==== Self-contained Templates in Advisors
|
||||
|
||||
The built-in advisors that perform prompt augmentation have been updated to use self-contained templates. The goal is for each advisor to be able to perform templating operations without affecting nor being affected by templating and prompt decisions in other advisors.
|
||||
If you were providing custom templates for the following advisors, you'll need to update them to ensure all expected placeholders are included.
|
||||
|
||||
* The `QuestionAnswerAdvisor` expects a template with the following placeholders (see xref:api/retrieval-augmented-generation.adoc#_questionansweradvisor[more details]):
|
||||
** a `query` placeholder to receive the user question.
|
||||
** a `question_answer_context` placeholder to receive the retrieved context.
|
||||
* The `PromptChatMemoryAdvisor` expects a template with the following placeholders (see xref:api/chat-memory.adoc#_promptchatmemoryadvisor[more details]):
|
||||
** an `instructions` placeholder to receive the original system message.
|
||||
** a `memory` placeholder to receive the retrieved conversation memory.
|
||||
* The `VectorStoreChatMemoryAdvisor` expects a template with the following placeholders (see xref:api/chat-memory.adoc#_vectorstorechatmemoryadvisor[more details]):
|
||||
** an `instructions` placeholder to receive the original system message.
|
||||
** a `long_term_memory` placeholder to receive the retrieved conversation memory.
|
||||
|
||||
=== Breaking Changes
|
||||
The Watson AI model was removed as it was based on the older text generation that is considered outdated as there is a new chat generation model available.
|
||||
Hopefully Watson will reappear in a future version of Spring AI
|
||||
|
||||
Reference in New Issue
Block a user