Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Configure TemplateRenderer in ChatClient

Browse Source 
- Extend the ChatClient with a new templateRenderer() method to pass a custom TemplateRenderer object used to render user and system templates.
- Evolve the QuestionAnswerAdvisor to accept a PromptTemplate for customising the RAG prompt and templating logic while maintaining backward compatibility.
- Introduce integration tests for the QuestionAnswerAdvisor.
- Document the TemplateRenderer API and how to use it to build PromptTemplate with custom templating logic.
- Document how to customise the templating logic used internally by the ChatClient via the TemplateRendererAPI.

Add validation tests and improve PromptTemplate resource handling

Enhance robustness and reliability of the PromptTemplate class with better
resource handling and comprehensive input validation:

- Add dedicated validation tests for builder methods with null/invalid inputs
- Improve renderResource method to gracefully handle edge cases:
  - Null resources return empty string
  - ByteArrayResource handling with proper charset (UTF-8)
  - Empty resources check with proper existence test
  - Better error handling with logging instead of exception propagation
- Add input validation assertions to all Builder methods
- Fix typo in deprecated annotation comment ("fahvor" → "favor")

Update documentation to clarify template rendering in different contexts:
- Add clear notes about TemplateRenderer usage in ChatClient vs Advisors
- Document how advisor template customization differs from ChatClient template rendering
- Add comprehensive API upgrade notes for template-related deprecations
- Include detailed migration examples for PromptTemplate and QuestionAnswerAdvisor

Fixes gh-355, gh-1687, gh-2448, gh-1849, gh-1428

Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit is contained in:
Thomas Vitale
2025-04-28 23:47:58 +02:00
committed by Mark Pollack
parent b0d671944a
commit 5527d037f2
20 changed files with 1005 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chatclient.adoc
View File

@@ -165,6 +165,39 @@ String content = this.flux.collectList().block().stream().collect(Collectors.joi
List<ActorFilms> actorFilms = this.converter.convert(this.content);
----
== Prompt Templates
The `ChatClient` fluent API lets you provide user and system text as templates with variables that are replaced at runtime.
[source,java]
----
String answer = ChatClient.create(chatModel).prompt()
.user(u -> u
.text("Tell me the names of 5 movies whose soundtrack was composed by {composer}")
.param("composer", "John Williams"))
.call()
.content();
----
Internally, the ChatClient uses the `PromptTemplate` class to handle the user and system text and replace the variables with the values provided at runtime relying on a given `TemplateRenderer` implementation. By default, Spring AI uses the `StTemplateRenderer` implementation, which is based on the open-source https://www.stringtemplate.org/[StringTemplate] engine developed by Terence Parr.
NOTE: The `TemplateRenderer` configured directly on the `ChatClient` (via `.templateRenderer()`) applies only to the prompt content defined directly in the `ChatClient` builder chain (e.g., via `.user()`, `.system()`). It does *not* affect templates used internally by xref:api/retrieval-augmented-generation.adoc#_questionansweradvisor[Advisors] like `QuestionAnswerAdvisor`, which have their own template customization mechanisms (see xref:api/retrieval-augmented-generation.adoc#_custom_template[Custom Advisor Templates]).
If you'd rather use a different template engine, you can provide a custom implementation of the `TemplateRenderer` interface directly to the ChatClient. You can also keep using the default `StTemplateRenderer`, but with a custom configuration.
For example, by default, template variables are identified by the `{}` syntax. If you're planning to include JSON in your prompt, you might want to use a different syntax to avoid conflicts with JSON syntax. For example, you can use the `<` and `>` delimiters.
[source,java]
----
String answer = ChatClient.create(chatModel).prompt()
.user(u -> u
.text("Tell me the names of 5 movies whose soundtrack was composed by <composer>")
.param("composer", "John Williams"))
.templateRenderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
.call()
.content();
----
== call() return values
After specifying the `call()` method on `ChatClient`, there are a few different options for the response type.

97
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/prompt.adoc
View File

@@ -20,7 +20,6 @@ Initially, prompts were simple strings.
Over time, they grew to include placeholders for specific inputs, like "USER:", which the AI model recognizes.
OpenAI have introduced even more structure to prompts by categorizing multiple message strings into distinct roles before they are processed by the AI model.
== API Overview
=== Prompt
@@ -34,14 +33,15 @@ This arrangement enables intricate and detailed interactions with AI models, as
Below is a truncated version of the Prompt class, with constructors and utility methods omitted for brevity:
```java
[source,java]
----
public class Prompt implements ModelRequest<List<Message>> {
private final List<Message> messages;
private ChatOptions chatOptions;
}
```
----
=== Message
@@ -49,7 +49,8 @@ The `Message` interface encapsulates a `Prompt` textual content, a collection of
The interface is defined as follows:
```java
[source,java]
----
public interface Content {
String getContent();
@@ -61,17 +62,18 @@ public interface Message extends Content {
MessageType getMessageType();
}
```
----
The multimodal message types implement also the `MediaContent` interface providing a list of `Media` content objects.
```java
[source,java]
----
public interface MediaContent extends Content {
Collection<Media> getMedia();
}
```
----
Various implementations of the `Message` interface correspond to different categories of messages that an AI model can process.
The Models distinguish between message categories based on conversational roles.
@@ -99,7 +101,8 @@ It's like a special feature in the AI, used when needed to perform specific func
Roles are represented as an enumeration in Spring AI as shown below
```java
[source,java]
----
public enum MessageType {
USER("user"),
@@ -112,20 +115,31 @@ public enum MessageType {
...
}
```
----
=== PromptTemplate
A key component for prompt templating in Spring AI is the `PromptTemplate` class.
This class uses the OSS https://www.stringtemplate.org/[StringTemplate] engine, developed by Terence Parr, for constructing and managing prompts.
The `PromptTemplate` class is designed to facilitate the creation of structured prompts that are then sent to the AI model for processing
A key component for prompt templating in Spring AI is the `PromptTemplate` class, designed to facilitate the creation of structured prompts that are then sent to the AI model for processing
```java
[source,java]
----
public class PromptTemplate implements PromptTemplateActions, PromptTemplateMessageActions {
// Other methods to be discussed later
}
```
----
This class uses the `TemplateRenderer` API to render templates. By default, Spring AI uses the `StTemplateRenderer` implementation, which is based on the open-source https://www.stringtemplate.org/[StringTemplate] engine developed by Terence Parr. Template variables are identified by the `{}` syntax, but you can configure the delimiters to use other syntax as well.
[source,java]
----
public interface TemplateRenderer extends BiFunction<String, Map<String, Object>, String> {
@Override
String apply(String template, Map<String, Object> variables);
}
----
The interfaces implemented by this class support different aspects of prompt creation:
@@ -139,7 +153,8 @@ While these interfaces might not be used extensively in many projects, they show
The implemented interfaces are
```java
[source,java]
----
public interface PromptTemplateStringActions {
String render();
@@ -147,13 +162,14 @@ public interface PromptTemplateStringActions {
String render(Map<String, Object> model);
}
```
----
The method `String render()`: Renders a prompt template into a final string format without external input, suitable for templates without placeholders or dynamic content.
The method `String render(Map<String, Object> model)`: Enhances rendering functionality to include dynamic content. It uses a `Map<String, Object>` where map keys are placeholder names in the prompt template, and values are the dynamic content to be inserted.
```java
[source,java]
----
public interface PromptTemplateMessageActions {
Message createMessage();
@@ -163,7 +179,7 @@ public interface PromptTemplateMessageActions {
Message createMessage(Map<String, Object> model);
}
```
----
The method `Message createMessage()`: Creates a `Message` object without additional data, used for static or predefined message content.
@@ -172,7 +188,8 @@ The method `Message createMessage(List<Media> mediaList)`: Creates a `Message` o
The method `Message createMessage(Map<String, Object> model)`: Extends message creation to integrate dynamic content, accepting a `Map<String, Object>` where each entry represents a placeholder in the message template and its corresponding dynamic value.
```java
[source,java]
----
public interface PromptTemplateActions extends PromptTemplateStringActions {
Prompt create();
@@ -184,7 +201,7 @@ public interface PromptTemplateActions extends PromptTemplateStringActions {
Prompt create(Map<String, Object> model, ChatOptions modelOptions);
}
```
----
The method `Prompt create()`: Generates a `Prompt` object without external data inputs, ideal for static or predefined prompts.
@@ -198,18 +215,19 @@ The method `Prompt create(Map<String, Object> model, ChatOptions modelOptions)`:
A simple example taken from the https://github.com/Azure-Samples/spring-ai-azure-workshop/blob/main/2-README-prompt-templating.md[AI Workshop on PromptTemplates] is shown below.
```java
[source,java]
----
PromptTemplate promptTemplate = new PromptTemplate("Tell me a {adjective} joke about {topic}");
Prompt prompt = promptTemplate.create(Map.of("adjective", adjective, "topic", topic));
return chatModel.call(prompt).getResult();
```
----
Another example taken from the https://github.com/Azure-Samples/spring-ai-azure-workshop/blob/main/3-README-prompt-roles.md[AI Workshop on Roles] is shown below.
```java
[source,java]
----
String userText = """
Tell me about three famous pirates from the Golden Age of Piracy and why they did.
Write at least a sentence for each pirate.
@@ -229,28 +247,47 @@ Message systemMessage = systemPromptTemplate.createMessage(Map.of("name", name,
Prompt prompt = new Prompt(List.of(userMessage, systemMessage));
List<Generation> response = chatModel.call(prompt).getResults();
```
----
This shows how you can build up the `Prompt` instance by using the `SystemPromptTemplate` to create a `Message` with the system role passing in placeholder values.
The message with the role `user` is then combined with the message of the role `system` to form the prompt.
The prompt is then passed to the ChatModel to get a generative response.
=== Using a custom template renderer
You can use a custom template renderer by implementing the `TemplateRenderer` interface and passing it to the `PromptTemplate` constructor. You can also keep using the default `StTemplateRenderer`, but with a custom configuration.
By default, template variables are identified by the `{}` syntax. If you're planning to include JSON in your prompt, you might want to use a different syntax to avoid conflicts with JSON syntax. For example, you can use the `<` and `>` delimiters.
[source,java]
----
PromptTemplate promptTemplate = PromptTemplate.builder()
.renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
.template("""
Tell me the names of 5 movies whose soundtrack was composed by <composer>.
""")
.build();
String prompt = promptTemplate.render(Map.of("composer", "John Williams"));
----
=== Using resources instead of raw Strings
Spring AI supports the `org.springframework.core.io.Resource` abstraction, so you can put prompt data in a file that can directly be used in a `PromptTemplate`.
For example, you can define a field in your Spring managed component to retrieve the `Resource`.
```java
[source,java]
----
@Value("classpath:/prompts/system-message.st")
private Resource systemResource;
```
----
and then pass that resource to the `SystemPromptTemplate` directly.
```java
[source,java]
----
SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemResource);
```
----
== Prompt Engineering

60
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/retrieval-augmented-generation.adoc
View File

@@ -25,8 +25,7 @@ To use the `QuestionAnswerAdvisor` or `RetrievalAugmentationAdvisor`, you need t
=== QuestionAnswerAdvisor
A vector database stores data that the AI model is unaware of.
When a user question is sent to the AI model, a `QuestionAnswerAdvisor` queries the vector database for documents related to the user question.
A vector database stores data that the AI model is unaware of. When a user question is sent to the AI model, a `QuestionAnswerAdvisor` queries the vector database for documents related to the user question.
The response from the vector database is appended to the user text to provide context for the AI model to generate a response.
@@ -42,17 +41,17 @@ ChatResponse response = ChatClient.builder(chatModel)
.chatResponse();
----
In this example, the `QuestionAnswerAdvisor` will perform a similarity search over all documents in the Vector Database.
To restrict the types of documents that are searched, the `SearchRequest` takes an SQL like filter expression that is portable across all `VectorStores`.
In this example, the `QuestionAnswerAdvisor` will perform a similarity search over all documents in the Vector Database. To restrict the types of documents that are searched, the `SearchRequest` takes an SQL like filter expression that is portable across all `VectorStores`.
This filter expression can be configured when creating the `QuestionAnswerAdvisor` and hence will always apply to all `ChatClient` requests or it can be provided at runtime per request.
This filter expression can be configured when creating the `QuestionAnswerAdvisor` and hence will always apply to all `ChatClient` requests, or it can be provided at runtime per request.
Here is how to create an instance of `QuestionAnswerAdvisor` where the threshold is `0.8` and to return the top `6` results.
[source,java]
----
var qaAdvisor = new QuestionAnswerAdvisor(this.vectorStore,
SearchRequest.builder().similarityThreshold(0.8d).topK(6).build());
var qaAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
.searchRequest(SearchRequest.builder().similarityThreshold(0.8d).topK(6).build())
.build();
----
==== Dynamic Filter Expressions
@@ -62,7 +61,9 @@ Update the `SearchRequest` filter expression at runtime using the `FILTER_EXPRES
[source,java]
----
ChatClient chatClient = ChatClient.builder(chatModel)
.defaultAdvisors(new QuestionAnswerAdvisor(vectorStore, SearchRequest.builder().build()))
.defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore)
.searchRequest(SearchRequest.builder().build())
.build())
.build();
// Update filter expression at runtime
@@ -75,6 +76,49 @@ String content = this.chatClient.prompt()
The `FILTER_EXPRESSION` parameter allows you to dynamically filter the search results based on the provided expression.
==== Custom Template
The `QuestionAnswerAdvisor` uses a default template to augment the user question with the retrieved documents. You can customize this behavior by providing your own `PromptTemplate` object via the `.promptTemplate()` builder method.
NOTE: The `PromptTemplate` provided here customizes how the advisor merges retrieved context with the user query. This is distinct from configuring a `TemplateRenderer` on the `ChatClient` itself (using `.templateRenderer()`), which affects the rendering of the initial user/system prompt content *before* the advisor runs. See xref:api/chatclient.adoc#_prompt_templates[ChatClient Prompt Templates] for more details on client-level template rendering.
The custom `PromptTemplate` can use any `TemplateRenderer` implementation (by default, it uses `StPromptTemplate` based on the https://www.stringtemplate.org/[StringTemplate] engine). The important requirement is that the template must contain a placeholder to receive the retrieved context, which the advisor provides under the key `question_answer_context`.
[source,java]
----
PromptTemplate customPromptTemplate = PromptTemplate.builder()
.renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
.template("""
Context information is below.
---------------------
<question_answer_context>
---------------------
Given the context information and no prior knowledge, answer the query.
Follow these rules:
1. If the answer is not in the context, just say that you don't know.
2. Avoid statements like "Based on the context..." or "The provided information...".
""")
.build();
String question = "Where does the adventure of Anacletus and Birba take place?";
QuestionAnswerAdvisor qaAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
.promptTemplate(customPromptTemplate)
.build();
String response = ChatClient.builder(chatModel).build()
.prompt(question)
.advisors(qaAdvisor)
.call()
.content();
----
NOTE: The `QuestionAnswerAdvisor.Builder.userTextAdvise()` method is deprecated in favor of using `.promptTemplate()` for more flexible customization.
=== RetrievalAugmentationAdvisor (Incubating)
Spring AI includes a xref:api/retrieval-augmented-generation.adoc#modules[library of RAG modules] that you can use to build your own RAG flows.

56
spring-ai-docs/src/main/antora/modules/ROOT/pages/upgrade-notes.adoc
View File

@@ -124,6 +124,61 @@ Prompt augmentedPrompt = originalPrompt.augmentUserMessage(userMessage ->
This approach offers more control when you need to conditionally change parts of the `UserMessage` or work with its media and metadata, rather than just replacing the text content.
=== Prompt Templating and Advisors
Several classes and methods related to prompt creation and advisor customization have been deprecated in favor of more flexible approaches using the builder pattern and the `TemplateRenderer` interface.
==== PromptTemplate Deprecations
The `PromptTemplate` class has deprecated several constructors and methods related to the older `templateFormat` enum and direct variable injection:
* Constructors: `PromptTemplate(String template, Map<String, Object> variables)` and `PromptTemplate(Resource resource, Map<String, Object> variables)` are deprecated.
* Fields: `template` and `templateFormat` are deprecated.
* Methods: `getTemplateFormat()`, `getInputVariables()`, and `validate(Map<String, Object> model)` are deprecated.
*Migration:* Use the `PromptTemplate.builder()` pattern to create instances. Provide the template string via `.template()` and optionally configure a custom `TemplateRenderer` via `.renderer()`. Pass variables using `.variables()`.
[source,java]
----
// Before (Deprecated)
PromptTemplate oldTemplate = new PromptTemplate("Hello {name}", Map.of("name", "World"));
String oldRendered = oldTemplate.render(); // Variables passed at construction
// After (Using Builder)
PromptTemplate newTemplate = PromptTemplate.builder()
.template("Hello {name}")
.variables(Map.of("name", "World")) // Variables passed during builder configuration
.build();
Prompt prompt = newTemplate.create(); // Create prompt using baked-in variables
String newRendered = prompt.getContents(); // Or use newTemplate.render()
----
==== QuestionAnswerAdvisor Deprecations
The `QuestionAnswerAdvisor` has deprecated constructors and builder methods that relied on a simple `userTextAdvise` string:
* Constructors taking a `userTextAdvise` String argument are deprecated.
* Builder method: `userTextAdvise(String userTextAdvise)` is deprecated.
*Migration:* Use the `.promptTemplate(PromptTemplate promptTemplate)` builder method to provide a fully configured `PromptTemplate` object for customizing how retrieved context is merged.
[source,java]
----
// Before (Deprecated)
QuestionAnswerAdvisor oldAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
.userTextAdvise("Context: {question_answer_context} Question: {question}") // Simple string
.build();
// After (Using PromptTemplate)
PromptTemplate customTemplate = PromptTemplate.builder()
.template("Context: {question_answer_context} Question: {question}")
.build();
QuestionAnswerAdvisor newAdvisor = QuestionAnswerAdvisor.builder(vectorStore)
.promptTemplate(customTemplate) // Provide PromptTemplate object
.build();
----
=== Chat Memory
* A `ChatMemory` bean is auto-configured for you whenever using one of the Spring AI Model starters. By default, it uses the `MessageWindowChatMemory` implementation and stores the conversation history in memory.
@@ -132,6 +187,7 @@ This approach offers more control when you need to conditionally change parts of
* The `JdbcChatMemory` has been deprecated in favour of using `JdbcChatMemoryRepository` together with a `ChatMemory` implementation such `MessageWindowChatMemory`. If you were relying on an auto-configured `JdbcChatMemory` bean, you can replace that by auto-wiring a `ChatMemory` bean that is auto-configured to use the `JdbcChatMemoryRepository` internally for storing messages whenever the related dependency is in the classpath.
* The `spring.ai.chat.memory.jdbc.initialize-schema` property has been deprecated in favor of `spring.ai.chat.memory.repository.jdbc.initialize-schema`.
* Refer to the new xref:api/chat-memory.adoc[Chat Memory] documentation for more details on the new API and how to use it.
* The `MessageWindowChatMemory.get(String conversationId, int lastN)` method is deprecated. The windowing size is now managed internally based on the configuration provided during instantiation, so only `get(String conversationId)` should be used.
=== Prompt Templating