Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Improve ChatClient API docs

Browse Source
This commit is contained in:
Christian Tzolov
2024-05-24 07:29:42 +02:00
parent a8fa1867d3
commit d6f4b71e1f
4 changed files with 88 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatClientIT.java
View File

@@ -57,6 +57,9 @@ class OpenAiChatClientIT extends AbstractIT {
@Value("classpath:/prompts/system-message.st")
private Resource systemTextResource;
record ActorsFilms(String actor, List<String> movies) {
}
@Test
void roleTest() {
@@ -92,16 +95,15 @@ class OpenAiChatClientIT extends AbstractIT {
void listOutputConverter2() {
// @formatter:off
List<ActorsFilmsRecord> actorsFilms = ChatClient.builder(chatModel).build().prompt()
List<ActorsFilms> actorsFilms = ChatClient.builder(chatModel).build().prompt()
.user("Generate the filmography of 5 movies for Tom Hanks and Bill Murray.")
.call()
.entity(new ParameterizedTypeReference<List<ActorsFilmsRecord>>() {
.entity(new ParameterizedTypeReference<List<ActorsFilms>>() {
});
// @formatter:on
logger.info("" + actorsFilms);
assertThat(actorsFilms).hasSize(2);
}
@Test
@@ -129,20 +131,17 @@ class OpenAiChatClientIT extends AbstractIT {
// @formatter:on
logger.info("" + actorsFilms);
assertThat(actorsFilms.getActor()).isNotBlank();
}
record ActorsFilmsRecord(String actor, List<String> movies) {
assertThat(actorsFilms.actor()).isNotBlank();
}
@Test
void beanOutputConverterRecords() {
// @formatter:off
ActorsFilmsRecord actorsFilms = ChatClient.builder(chatModel).build().prompt()
ActorsFilms actorsFilms = ChatClient.builder(chatModel).build().prompt()
.user("Generate the filmography of 5 movies for Tom Hanks.")
.call()
.entity(ActorsFilmsRecord.class);
.entity(ActorsFilms.class);
// @formatter:on
logger.info("" + actorsFilms);
@@ -153,7 +152,7 @@ class OpenAiChatClientIT extends AbstractIT {
@Test
void beanStreamOutputConverterRecords() {
BeanOutputConverter<ActorsFilmsRecord> outputConverter = new BeanOutputConverter<>(ActorsFilmsRecord.class);
BeanOutputConverter<ActorsFilms> outputConverter = new BeanOutputConverter<>(ActorsFilms.class);
// @formatter:off
Flux<String> chatResponse = ChatClient.builder(chatModel)
@@ -172,7 +171,7 @@ class OpenAiChatClientIT extends AbstractIT {
.collect(Collectors.joining());
// @formatter:on
ActorsFilmsRecord actorsFilms = outputConverter.convert(generationTextFromStream);
ActorsFilms actorsFilms = outputConverter.convert(generationTextFromStream);
logger.info("" + actorsFilms);
assertThat(actorsFilms.actor()).isEqualTo("Tom Hanks");

93
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chatclient.adoc
View File

@@ -3,7 +3,7 @@
The `ChatClient` offers a fluent API for stateless interaction with an AI Model. It supports both a synchronous and reactive programming model.
The fluent API has methods for building up the constituent parts of a `Prompt` that is passed to the AI model as input.
The fluent API has methods for building up the constituent parts of a xref:api/prompt.adoc#_prompts[Prompt] that is passed to the AI model as input.
The `Prompt` contains the instructional text to guide the AI model's output and behavior. From the API point of view, prompts consist of a collection of messages.
The AI model processes two main types of messages: user messages, which are direct inputs from the user, and system messages, which are generated by the system to guide the conversation.
@@ -12,12 +12,18 @@ These messages often contain template placeholders that are substituted at runti
There are also Prompt options that can be specified., such as the name of the AI Model to generate content and the temperature setting that controls the randomness or creativity of the generated output.
== Using an autoconfigured ChatClient.Builder
== Creating a ChatClient
The `ChatClient` is created using a `ChatClient.Builder` object.
You can obtain an autoconfigured `ChatClient.Builder` instance for any xref:api/chatmodel.adoc[ChatModel] Spring Boot autoconfiguration or create one programmatically.
=== Using an autoconfigured ChatClient.Builder
In the most simple use case, Spring AI provides Spring Boot autoconfiguration, creating a prototype `ChatClient.Builder` bean for you to inject into your class.
Here is a simple example of retrieving a String response to a simple user request.
```java
[source,java]
----
@RestController
class MyController {
@@ -35,61 +41,100 @@ class MyController {
.content();
}
}
```
----
In this simple example, the user input sets the contents of the user message. The call method sends a request to the AI model, and the context method returns the AI model's response as a String.
In this simple example, the user input sets the contents of the user message.
The call method sends a request to the AI model, and the context method returns the AI model's response as a String.
=== Create a ChatClient programmatically
== Returing a `ChatResponse`
You can disable the `ChatClient.Builder` autoconfiguration by setting the property `spring.ai.chat.client.enabled=false`.
This is useful if multiple chat models are used together.
Then create a `ChatClient.Builder` instance for for every `ChatModel` programmatically:
The response from the AI model is a rich structure defined by the type ChatResponse.
ChatResponse includes metadata about how the response was generated and can also contain multiple responses, known as generations, each with its own metadata.
The metadata includes the number of tokens (each token is approximately 3/4 of a word) used to create the response. This information is important because hosted AI models charge based on the number of tokens used per request.
[source,java]
----
ChatModel myChatModel = ... // usually autowired
ChatClient.Builder builder = ChatClient.builder(myChatModel);
// or create a ChatClient with the default builder settings:
ChatClient chatClient = ChatClient.create(myChatModel);
----
== ChatClient Responses
The ChatClient API offers several ways to format the response from the AI Model.
=== Returning a ChatResponse
The response from the AI model is a rich structure defined by the type xref:api/chatmodel.adoc#_chatresponse[ChatResponse].
It includes metadata about how the response was generated and can also contain multiple responses, known as xref:api/chatmodel.adoc#_generation[Generation]s, each with its own metadata.
The metadata includes the number of tokens (each token is approximately 3/4 of a word) used to create the response.
This information is important because hosted AI models charge based on the number of tokens used per request.
An example to return the `ChatResponse` object that contains the metadata is shown below by invoking `chatResponse()` after the `call()` method.
```java
ChatResponse chatResponse = this.chatClient.prompt()
[source,java]
----
ChatResponse chatResponse = chatClient.prompt()
.user("Tell me a joke")
.call()
.chatResponse();
```
----
== Returning an Entity
=== Returning an Entity
You often want to return an entity class that is mapped from the returned String. The `entity` method provides this functionality.
You often want to return an entity class that is mapped from the returned `String`.
The `entity` method provides this functionality.
For example, given the Java record:
```java
[source,java]
----
record ActorFilms(String actor, List<String> movies) {
}
```
----
You can easily map the AI model's output to this record using the `entity` method, as shown below:
```java
[source,java]
----
ActorFilms actorFilms = chatClient.prompt()
.user("Generate the filmography for a random actor.")
.call()
.entity(ActorFilms.class);
```
----
There is also an overloaded `entity` method with the signature `entity(ParameterizedTypeReference<T> type)` that lets you specify types such as generic Lists.
There is also an overloaded `entity` method with the signature `entity(ParameterizedTypeReference<T> type)` that lets you specify types such as generic Lists:
== Streaming Responses
[source,java]
----
List<ActorFilms> actorFilms = chatClient.prompt()
.user("Generate the filmography of 5 movies for Tom Hanks and Bill Murray.")
.call()
.entity(new ParameterizedTypeReference<List<ActorsFilms>>() {
});
----
=== Streaming Responses
The `stream` lets you get an asynchronous response as shown below
```java
Flux<String> output = this.chatClient.prompt()
[source,java]
----
Flux<String> output = chatClient.prompt()
.user("Tell me a joke")
.stream()
.content();
```
----
You can also stream the `ChatResponse` using the method `Flux<ChatResponse> chatResponse()`.
TIP: The chunked stream response can not be converted into a Java entities automatically.
Use the xref:api/structured-output-converter.adoc#_structuredoutputconverter[Structured Output Converter] to convert the aggregated response.
== Using defaults and parameters
It is often useful to create a `ChatClient` with default user and/or system text defined at design time.

10
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chatmodel.adoc
View File

@@ -17,7 +17,7 @@ This section provides a guide to the Spring AI Chat Model API interface and asso
=== ChatModel
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatModel.java[ChatModel] interface definition:
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat//model/ChatModel.java[ChatModel] interface definition:
[source,java]
----
@@ -37,7 +37,7 @@ In real-world applications, it is more common to use the `call` method that take
=== StreamingChatModel
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/StreamingChatModel.java[StreamingChatModel] interface definition:
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/model/StreamingChatModel.java[StreamingChatModel] interface definition:
[source,java]
----
@@ -136,6 +136,7 @@ This is a powerful feature that allows developers to use model specific options
image::chat-options-flow.jpg[align="center", width="800px"]
[[ChatResponse]]
=== ChatResponse
The structure of the `ChatResponse` class is as follows:
@@ -157,13 +158,14 @@ public class ChatResponse implements ModelResponse<Generation> {
}
----
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatResponse.java[ChatResponse] class holds the AI Model's output, with each `Generation` instance containing one of potentially multiple outputs resulting from a single prompt.
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/model/ChatResponse.java[ChatResponse] class holds the AI Model's output, with each `Generation` instance containing one of potentially multiple outputs resulting from a single prompt.
The `ChatResponse` class also carries a `ChatResponseMetadata` metadata about the AI Model's response.
[[Generation]]
=== Generation
Finally, the https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/Generation.java[Generation] class extends from the `ModelResult` to represent the output assistant message response and related metadata about this result:
Finally, the https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/model/Generation.java[Generation] class extends from the `ModelResult` to represent the output assistant message response and related metadata about this result:
[source,java]
----

8
spring-ai-docs/src/main/antora/modules/ROOT/pages/concepts.adoc
View File

@@ -136,17 +136,15 @@ Anthropic's Claude AI model features a 100K token limit, and Meta's recent resea
To summarize the collected works of Shakespeare with GPT4, you need to devise software engineering strategies to chop up the data and present the data within the model's context window limits.
The Spring AI project helps you with this task.
== Output Parsing
== Structured Output
The output of AI models traditionally arrives as a `java.lang.String`, even if you ask for the reply to be in JSON.
It may be the correct JSON, but it is not a JSON data structure. It is just a string.
Also, asking "`for JSON`" as part of the prompt is not 100% accurate.
This intricacy has led to the emergence of a specialized field involving the creation of prompts to yield the intended output, followed by parsing the resulting simple string into a usable data structure for application integration.
This intricacy has led to the emergence of a specialized field involving the creation of prompts to yield the intended output, followed by converting the resulting simple string into a usable data structure for application integration.
Output parsing employs meticulously crafted prompts, often necessitating multiple interactions with the model to achieve the desired formatting.
This challenge has prompted OpenAI to introduce 'OpenAI Functions' as a means to specify the desired output format from the model precisely.
The xref:api/structured-output-converter.adoc#_structuredoutputconverter[Structured output conversion] employs meticulously crafted prompts, often necessitating multiple interactions with the model to achieve the desired formatting.
== Bringing Your Data to the AI model