Refactoring of ChatClient to add fluent API and introduce Model as dependent object
* Rename the ModelClient class hierarchy into Model: - Rename ModelClient into Model. Update all code and doc references. - Rename ChatClient to ChatModel. Update all ChatClient suffixes and chatClient fields and variables in code and doc. - Rename EmbeddingClient into EmbeddingModel. Update the XxxEmbeddingClient class and variable suffixes and embeddingClient variables and fields in code and docs. - Rename ImageClient into ImageModel. - Rename SpeechClient into SpeechModel. - Rename TranscriptionClient into TranscriptionModel. - Update all javadocs and antora pages. Update the related diagrams. * Create fluent API in ChatClient interface that now includes streaming support * Add OpenAI FunctionCallbackWrapper2IT auto-config tests. * Add ChatClientTest mockito testing. * Add ChatModel#getDefaultOptions(), and remove @FunctionalInterface * ChatModel enums extend the new ModelDescription interface. * Implement fromOptions copy method in every ChatOptions implementation. * Extend ChatClient to use the model default options if not provided explicitly. * Update readme to provide guidance on how to adapt to breaking changes. Co-authored-by: Christian Tzolov <ctzolov@vmware.com> Co-authored-by: Mark Pollack <mpollack@vmware.com>
This commit is contained in:
Josh Long
Binary file not shown.
|
Before Width: | Height: | Size: 170 KiB After Width: | Height: | Size: 415 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 340 KiB After Width: | Height: | Size: 347 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 147 KiB After Width: | Height: | Size: 270 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 193 KiB After Width: | Height: | Size: 353 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 284 KiB After Width: | Height: | Size: 296 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 386 KiB After Width: | Height: | Size: 340 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 247 KiB After Width: | Height: | Size: 244 KiB |
@@ -46,7 +46,7 @@
|
||||
*** xref:api/image/openai-image.adoc[OpenAI]
|
||||
*** xref:api/image/stabilityai-image.adoc[Stability]
|
||||
*** xref:api/image/zhipuai-image.adoc[ZhiPuAI]
|
||||
** xref:api/audio[Audio API]
|
||||
** xref:api/audio[Audio Model API]
|
||||
*** xref:api/audio/transcriptions.adoc[]
|
||||
**** xref:api/audio/transcriptions/openai-transcriptions.adoc[OpenAI]
|
||||
*** xref:api/audio/speech.adoc[]
|
||||
|
||||
@@ -42,7 +42,7 @@ class MyService {
|
||||
|
||||
Prompt prompt = createPrompt(request);
|
||||
|
||||
ChatResponse response = chatClient.call(prompt);
|
||||
ChatResponse response = chatModel.call(prompt);
|
||||
|
||||
// Process the chat response
|
||||
|
||||
|
||||
@@ -2,4 +2,4 @@
|
||||
= Text-To-Speech (TTS) API
|
||||
|
||||
Spring AI provides support for OpenAI's Speech API.
|
||||
When additional providers for Speech are implemented, a common `SpeechClient` and `StreamingSpeechClient` interface will be extracted.
|
||||
When additional providers for Speech are implemented, a common `SpeechModel` and `StreamingSpeechModel` interface will be extracted.
|
||||
@@ -68,7 +68,7 @@ OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.build();
|
||||
|
||||
SpeechPrompt speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
|
||||
SpeechResponse response = openAiAudioSpeechClient.call(speechPrompt);
|
||||
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
@@ -94,13 +94,13 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an `OpenAiAudioSpeechClient`:
|
||||
Next, create an `OpenAiAudioSpeechModel`:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioSpeechClient = new OpenAiAudioSpeechClient(openAiAudioApi);
|
||||
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
|
||||
|
||||
var speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.withResponseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3)
|
||||
@@ -109,7 +109,7 @@ var speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.build();
|
||||
|
||||
var speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
|
||||
SpeechResponse response = openAiAudioSpeechClient.call(speechPrompt);
|
||||
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
|
||||
|
||||
// Accessing metadata (rate limit info)
|
||||
OpenAiAudioSpeechResponseMetadata metadata = response.getMetadata();
|
||||
@@ -125,7 +125,7 @@ The Speech API provides support for real-time audio streaming using chunk transf
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioSpeechClient = new OpenAiAudioSpeechClient(openAiAudioApi);
|
||||
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
|
||||
|
||||
OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.withVoice(OpenAiAudioApi.SpeechRequest.Voice.ALLOY)
|
||||
@@ -136,9 +136,9 @@ OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
|
||||
SpeechPrompt speechPrompt = new SpeechPrompt("Today is a wonderful day to build something people love!", speechOptions);
|
||||
|
||||
Flux<SpeechResponse> responseStream = openAiAudioSpeechClient.stream(speechPrompt);
|
||||
Flux<SpeechResponse> responseStream = openAiAudioSpeechModel.stream(speechPrompt);
|
||||
----
|
||||
|
||||
== Example Code
|
||||
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/speech/OpenAiSpeechClientIT.java[OpenAiSpeechClientIT.java] test provides some general examples of how to use the library.
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/speech/OpenAiSpeechModelIT.java[OpenAiSpeechModelIT.java] test provides some general examples of how to use the library.
|
||||
|
||||
@@ -2,4 +2,4 @@
|
||||
= Transcription API
|
||||
|
||||
Spring AI provides support for OpenAI's Transcription API.
|
||||
When additional providers for Transcription are implemented, a common `AudioTranscriptionClient` interface will be extracted.
|
||||
When additional providers for Transcription are implemented, a common `AudioTranscriptionModel` interface will be extracted.
|
||||
@@ -37,7 +37,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
=== Transcription Properties
|
||||
|
||||
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI Image client.
|
||||
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI image model.
|
||||
|
||||
[cols="3,5,2"]
|
||||
|====
|
||||
@@ -69,7 +69,7 @@ OpenAiAudioTranscriptionOptions transcriptionOptions = OpenAiAudioTranscriptionO
|
||||
.withResponseFormat(responseFormat)
|
||||
.build();
|
||||
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionClient.call(transcriptionRequest);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
@@ -95,13 +95,13 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `OpenAiAudioTranscriptionClient`
|
||||
Next, create a `OpenAiAudioTranscriptionModel`
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioTranscriptionClient = new OpenAiAudioTranscriptionClient(openAiAudioApi);
|
||||
var openAiAudioTranscriptionModel = new OpenAiAudioTranscriptionModel(openAiAudioApi);
|
||||
|
||||
var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
|
||||
.withResponseFormat(TranscriptResponseFormat.TEXT)
|
||||
@@ -111,8 +111,8 @@ var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
|
||||
var audioFile = new FileSystemResource("/path/to/your/resource/speech/jfk.flac");
|
||||
|
||||
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionClient.call(transcriptionRequest);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
|
||||
----
|
||||
|
||||
== Example Code
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionClientIT.java[OpenAiTranscriptionClientIT.java] test provides some general examples how to use the library.
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionModelIT.java[OpenAiTranscriptionModelIT.java] test provides some general examples how to use the library.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
link:https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock] is a managed service that provides foundation models from various AI providers, available through a unified API.
|
||||
|
||||
Spring AI supports https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[all the Chat and Embedding AI models] available through Amazon Bedrock by implementing the Spring interfaces `ChatClient`, `StreamingChatClient`, and `EmbeddingClient`.
|
||||
Spring AI supports https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[all the Chat and Embedding AI models] available through Amazon Bedrock by implementing the Spring interfaces `ChatModel`, `StreamingChatModel`, and `EmbeddingModel`.
|
||||
|
||||
Additionally, Spring AI provides Spring Auto-Configurations and Boot Starters for all clients, making it easy to bootstrap and configure for the Bedrock models.
|
||||
|
||||
@@ -94,7 +94,7 @@ Here are the supported `<model>` and `<chat|embedding>` combinations:
|
||||
| titan | Yes | Yes | Yes (however, no batch support)
|
||||
|====
|
||||
|
||||
For example, to enable the Bedrock Llama Chat client, you need to set `spring.ai.bedrock.llama.chat.enabled=true`.
|
||||
For example, to enable the Bedrock Llama chat model, you need to set `spring.ai.bedrock.llama.chat.enabled=true`.
|
||||
|
||||
Next, you can use the `spring.ai.bedrock.<model>.<chat|embedding>.*` properties to configure each model as provided.
|
||||
|
||||
|
||||
@@ -56,7 +56,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the Anthropic Chat client.
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the Anthropic chat model.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -88,13 +88,13 @@ The prefix `spring.ai.anthropic` is used as the property prefix that lets you co
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.anthropic.chat` is the property prefix that lets you configure the chat client implementation for Anthropic.
|
||||
The prefix `spring.ai.anthropic.chat` is the property prefix that lets you configure the chat model implementation for Anthropic.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.anthropic.chat.enabled | Enable Anthropic chat client. | true
|
||||
| spring.ai.anthropic.chat.enabled | Enable Anthropic chat model. | true
|
||||
| spring.ai.anthropic.chat.options.model | This is the Anthropic Chat model to use. Supports `claude-3-opus-20240229`, `claude-3-sonnet-20240229`, `claude-3-haiku-20240307` and the legacy `claude-2.1`, `claude-2.0` and `claude-instant-1.2` models. | `claude-3-opus-20240229`
|
||||
| spring.ai.anthropic.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.8
|
||||
| spring.ai.anthropic.chat.options.max-tokens | The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. | 500
|
||||
@@ -102,7 +102,7 @@ The prefix `spring.ai.anthropic.chat` is the property prefix that lets you confi
|
||||
| spring.ai.anthropic.chat.options.top-p | Use nucleus sampling. In nucleus sampling, we compute the cumulative distribution over all the options for each subsequent token in decreasing probability order and cut it off once it reaches a particular probability specified by top_p. You should either alter temperature or top_p, but not both. Recommended for advanced use cases only. You usually only need to use temperature. | -
|
||||
| spring.ai.anthropic.chat.options.top-k | Only sample from the top K options for each subsequent token. Used to remove "long tail" low probability responses. Learn more technical details here. Recommended for advanced use cases only. You usually only need to use temperature. | -
|
||||
| spring.ai.mistralai.chat.options.functions | List of functions, identified by their names, to enable for function calling in a single prompt requests. Functions with those names must exist in the functionCallbacks registry. | -
|
||||
| spring.ai.mistralai.chat.options.functionCallbacks | MistralAI Tool Function Callbacks to register with the ChatClient. | -
|
||||
| spring.ai.mistralai.chat.options.functionCallbacks | MistralAI Tool Function Callbacks to register with the ChatModel. | -
|
||||
|====
|
||||
|
||||
TIP: All properties prefixed with `spring.ai.anthropic.chat.options` can be overridden at runtime by adding a request specific <<chat-options>> to the `Prompt` call.
|
||||
@@ -111,14 +111,14 @@ TIP: All properties prefixed with `spring.ai.anthropic.chat.options` can be over
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/main/java/org/springframework/ai/anthropic/AnthropicChatOptions.java[AnthropicChatOptions.java] provides model configurations, such as the model to use, the temperature, the max token count, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `AnthropicChatClient(api, options)` constructor or the `spring.ai.anthropic.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `AnthropicChatModel(api, options)` constructor or the `spring.ai.anthropic.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
AnthropicChatOptions.builder()
|
||||
@@ -132,7 +132,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
== Function Calling
|
||||
|
||||
You can register custom Java functions with the `AnthropicChatClient` and have the Anthropic Claude model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `AnthropicChatModel` and have the Anthropic Claude model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This is a powerful technique to connect the LLM capabilities with external tools and APIs.
|
||||
Read more about xref:api/chat/functions/anthropic-chat-functions.adoc[Anthropic Function Calling].
|
||||
|
||||
@@ -146,7 +146,7 @@ Check the link:https://docs.anthropic.com/claude/docs/vision[Vision guide] for m
|
||||
Spring AI's `Message` interface supports multimodal AI models by introducing the Media type.
|
||||
This type contains data and information about media attachments in messages, using Spring's `org.springframework.util.MimeType` and a `java.lang.Object` for the raw media data.
|
||||
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic/AnthropicChatClientIT.java[AnthropicChatClientIT.java], demonstrating the combination of user text with an image.
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic/AnthropicChatModelIT.java[AnthropicChatModelIT.java], demonstrating the combination of user text with an image.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -155,7 +155,7 @@ byte[] imageData = new ClassPathResource("/multimodal.test.png").getContentAsByt
|
||||
var userMessage = new UserMessage("Explain what do you see on this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)));
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage)));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage)));
|
||||
|
||||
logger.info(response.getResult().getOutput().getContent());
|
||||
----
|
||||
@@ -182,7 +182,7 @@ The composition and lighting give the image a clean, minimalist aesthetic that h
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-anthropic-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -194,37 +194,37 @@ spring.ai.anthropic.chat.options.max-tokens=450
|
||||
|
||||
TIP: replace the `api-key` with your Anthropic credentials.
|
||||
|
||||
This will create a `AnthropicChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `AnthropicChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final AnthropicChatClient chatClient;
|
||||
private final AnthropicChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(AnthropicChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(AnthropicChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/main/java/org/springframework/ai/anthropic/AnthropicChatClient.java[AnthropicChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Anthropic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/main/java/org/springframework/ai/anthropic/AnthropicChatModel.java[AnthropicChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Anthropic service.
|
||||
|
||||
Add the `spring-ai-anthropic` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -247,24 +247,24 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `AnthropicChatClient` and use it for text generations:
|
||||
Next, create a `AnthropicChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var anthropicApi = new AnthropicApi(System.getenv("ANTHROPIC_API_KEY"));
|
||||
|
||||
var chatClient = new AnthropicChatClient(anthropicApi,
|
||||
var chatModel = new AnthropicChatModel(anthropicApi,
|
||||
AnthropicChatOptions.builder()
|
||||
.withModel("claude-3-opus-20240229")
|
||||
.withTemperature(0.4)
|
||||
.withMaxTokens(200)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -88,13 +88,13 @@ The prefix `spring.ai.azure.openai` is the property prefix to configure the conn
|
||||
| spring.ai.azure.openai.endpoint | The endpoint from the Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.azure.openai.chat` is the property prefix that configures the `ChatClient` implementation for Azure OpenAI.
|
||||
The prefix `spring.ai.azure.openai.chat` is the property prefix that configures the `ChatModel` implementation for Azure OpenAI.
|
||||
|
||||
[cols="3,5,3"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.azure.openai.chat.enabled | Enable Azure OpenAI chat client. | true
|
||||
| spring.ai.azure.openai.chat.enabled | Enable Azure OpenAI chat model. | true
|
||||
| spring.ai.azure.openai.chat.options.deployment-name | * In use with Azure, this refers to the "Deployment Name" of your model, which you can find at https://oai.azure.com/portal. It's important to note that within an Azure OpenAI deployment, the "Deployment Name" is distinct from the model itself. The confusion around these terms stems from the intention to make the Azure OpenAI client library compatible with the original OpenAI endpoint. The deployment structures offered by Azure OpenAI and Sam Altman's OpenAI differ significantly.
|
||||
Deployments model name to provide as part of this completions request.
|
||||
| gpt-35-turbo
|
||||
@@ -115,14 +115,14 @@ TIP: All properties prefixed with `spring.ai.azure.openai.chat.options` can be o
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatOptions.java[AzureOpenAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `AzureOpenAiChatClient(api, options)` constructor or the `spring.ai.azure.openai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `AzureOpenAiChatModel(api, options)` constructor or the `spring.ai.azure.openai.chat.options.*` properties.
|
||||
|
||||
At runtime you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
AzureOpenAiChatOptions.builder()
|
||||
@@ -137,7 +137,7 @@ TIP: In addition to the model specific link:https://github.com/spring-projects/s
|
||||
|
||||
== Function Calling
|
||||
|
||||
You can register custom Java functions with the AzureOpenAiChatClient and have the model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the AzureOpenAiChatModel and have the model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This is a powerful technique to connect the LLM capabilities with external tools and APIs.
|
||||
Read more about xref:api/chat/functions/azure-open-ai-chat-functions.adoc[Azure OpenAI Function Calling].
|
||||
|
||||
@@ -145,7 +145,7 @@ Read more about xref:api/chat/functions/azure-open-ai-chat-functions.adoc[Azure
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-azure-openai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -157,8 +157,8 @@ spring.ai.azure.openai.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `api-key` and `endpoint` with your Azure OpenAI credentials.
|
||||
|
||||
This will create a `AzureOpenAiChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `AzureOpenAiChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
|
||||
[source,java]
|
||||
@@ -166,29 +166,29 @@ Here is an example of a simple `@Controller` class that uses the chat client for
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final AzureOpenAiChatClient chatClient;
|
||||
private final AzureOpenAiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(AzureOpenAiChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(AzureOpenAiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatClient.java[AzureOpenAiChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the link:https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Azure OpenAI Java Client].
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiChatModel.java[AzureOpenAiChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the link:https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Azure OpenAI Java Client].
|
||||
|
||||
To enable it, add the `spring-ai-azure-openai` dependency to your project's Maven `pom.xml` file:
|
||||
[source, xml]
|
||||
@@ -210,9 +210,9 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
TIP: The `spring-ai-azure-openai` dependency also provide the access to the `AzureOpenAiChatClient`. For more information about the `AzureOpenAiChatClient` refer to the link:../chat/azure-openai-chat.html[Azure OpenAI Chat] section.
|
||||
TIP: The `spring-ai-azure-openai` dependency also provide the access to the `AzureOpenAiChatModel`. For more information about the `AzureOpenAiChatModel` refer to the link:../chat/azure-openai-chat.html[Azure OpenAI Chat] section.
|
||||
|
||||
Next, create an `AzureOpenAiChatClient` instance and use it to generate text responses:
|
||||
Next, create an `AzureOpenAiChatModel` instance and use it to generate text responses:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -227,13 +227,13 @@ var openAIChatOptions = AzureOpenAiChatOptions.builder()
|
||||
.withMaxTokens(200)
|
||||
.build();
|
||||
|
||||
var chatClient = new AzureOpenAiChatClient(openAIClient, openAIChatOptions);
|
||||
var chatModel = new AzureOpenAiChatModel(openAIClient, openAIChatOptions);
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
----
|
||||
|
||||
@@ -74,13 +74,13 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.anthropic.chat` is the property prefix that configures the chat client implementation for Claude.
|
||||
The prefix `spring.ai.bedrock.anthropic.chat` is the property prefix that configures the chat model implementation for Claude.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.bedrock.anthropic.chat.enable | Enable Bedrock Anthropic chat client. Disabled by default | false
|
||||
| spring.ai.bedrock.anthropic.chat.enable | Enable Bedrock Anthropic chat model. Disabled by default | false
|
||||
| spring.ai.bedrock.anthropic.chat.model | The model id to use. See the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/api/AnthropicChatBedrockApi.java[AnthropicChatModel] for the supported models. | anthropic.claude-v2
|
||||
| spring.ai.bedrock.anthropic.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.8
|
||||
| spring.ai.bedrock.anthropic.chat.options.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default
|
||||
@@ -100,14 +100,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.anthropic.chat.options` can
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/AnthropicChatOptions.java[AnthropicChatOptions.java] provides model configurations, such as temperature, topK, topP, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockAnthropicChatClient(api, options)` constructor or the `spring.ai.bedrock.anthropic.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockAnthropicChatModel(api, options)` constructor or the `spring.ai.bedrock.anthropic.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
AnthropicChatOptions.builder()
|
||||
@@ -122,7 +122,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -138,37 +138,37 @@ spring.ai.bedrock.anthropic.chat.options.top-k=15
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockAnthropicChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockAnthropicChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockAnthropicChatClient chatClient;
|
||||
private final BedrockAnthropicChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockAnthropicChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockAnthropicChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClient.java[BedrockAnthropicChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatModel.java[BedrockAnthropicChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -191,7 +191,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClient.java[BedrockAnthropicChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatModel.java[BedrockAnthropicChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -202,7 +202,7 @@ AnthropicChatBedrockApi anthropicApi = new AnthropicChatBedrockApi(
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockAnthropicChatClient chatClient = new BedrockAnthropicChatClient(anthropicApi,
|
||||
BedrockAnthropicChatModel chatModel = new BedrockAnthropicChatModel(anthropicApi,
|
||||
AnthropicChatOptions.builder()
|
||||
.withTemperature(0.6f)
|
||||
.withTopK(10)
|
||||
@@ -211,11 +211,11 @@ BedrockAnthropicChatClient chatClient = new BedrockAnthropicChatClient(anthropic
|
||||
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -71,13 +71,13 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.anthropic3.chat` is the property prefix that configures the chat client implementation for Claude.
|
||||
The prefix `spring.ai.bedrock.anthropic3.chat` is the property prefix that configures the chat model implementation for Claude.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.bedrock.anthropic3.chat.enable | Enable Bedrock Anthropic chat client. Disabled by default | false
|
||||
| spring.ai.bedrock.anthropic3.chat.enable | Enable Bedrock Anthropic chat model. Disabled by default | false
|
||||
| spring.ai.bedrock.anthropic3.chat.model | The model id to use. Supports the `anthropic.claude-3-sonnet-20240229-v1:0`,`anthropic.claude-3-haiku-20240307-v1:0` and the legacy `anthropic.claude-v2`, `anthropic.claude-v2:1` and `anthropic.claude-instant-v1` models for both synchronous and streaming responses. | `anthropic.claude-3-sonnet-20240229-v1:0`
|
||||
| spring.ai.bedrock.anthropic3.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.8
|
||||
| spring.ai.bedrock.anthropic3.chat.options.top-p | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default
|
||||
@@ -97,14 +97,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.anthropic3.chat.options` ca
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic3/Anthropic3ChatOptions.java[Anthropic3ChatOptions.java] provides model configurations, such as temperature, topK, topP, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockAnthropicChatClient(api, options)` constructor or the `spring.ai.bedrock.anthropic3.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockAnthropicChatModel(api, options)` constructor or the `spring.ai.bedrock.anthropic3.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
Anthropic3ChatOptions.builder()
|
||||
@@ -126,7 +126,7 @@ Check the link:https://docs.anthropic.com/claude/docs/vision[Vision guide] for m
|
||||
Spring AI's `Message` interface supports multimodal AI models by introducing the Media type.
|
||||
This type contains data and information about media attachments in messages, using Spring's `org.springframework.util.MimeType` and a `java.lang.Object` for the raw media data.
|
||||
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic3/Anthropic3ChatClientIT.java[Anthropic3ChatClientIT.java], demonstrating the combination of user text with an image.
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic3/Anthropic3ChatModelIT.java[Anthropic3ChatModelIT.java], demonstrating the combination of user text with an image.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -135,7 +135,7 @@ Below is a simple code example extracted from https://github.com/spring-projects
|
||||
var userMessage = new UserMessage("Explain what do you see o this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)));
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage)));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage)));
|
||||
|
||||
assertThat(response.getResult().getOutput().getContent()).contains("bananas", "apple", "basket");
|
||||
----
|
||||
@@ -163,7 +163,7 @@ The composition and lighting give the image a clean, minimalist aesthetic that h
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -179,37 +179,37 @@ spring.ai.bedrock.anthropic3.chat.options.top-k=15
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockAnthropicChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockAnthropicChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockAnthropic3ChatClient chatClient;
|
||||
private final BedrockAnthropic3ChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockAnthropic3ChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockAnthropic3ChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatClient.java[BedrockAnthropic3ChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatModel.java[BedrockAnthropic3ChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -232,7 +232,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatClient.java[BedrockAnthropic3ChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatModel.java[BedrockAnthropic3ChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -243,7 +243,7 @@ Anthropic3ChatBedrockApi anthropicApi = new Anthropic3ChatBedrockApi(
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockAnthropic3ChatClient chatClient = new BedrockAnthropic3ChatClient(anthropicApi,
|
||||
BedrockAnthropic3ChatModel chatModel = new BedrockAnthropic3ChatModel(anthropicApi,
|
||||
AnthropicChatOptions.builder()
|
||||
.withTemperature(0.6f)
|
||||
.withTopK(10)
|
||||
@@ -252,11 +252,11 @@ BedrockAnthropic3ChatClient chatClient = new BedrockAnthropic3ChatClient(anthrop
|
||||
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Cohere Chat
|
||||
|
||||
Provides Bedrock Cohere Chat client.
|
||||
Provides Bedrock Cohere chat model.
|
||||
Integrate generative AI capabilities into essential apps and workflows that improve business outcomes.
|
||||
|
||||
The https://aws.amazon.com/bedrock/cohere-command-embed/[AWS Bedrock Cohere Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
|
||||
@@ -64,7 +64,7 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.cohere.chat` is the property prefix that configures the chat client implementation for Cohere.
|
||||
The prefix `spring.ai.bedrock.cohere.chat` is the property prefix that configures the chat model implementation for Cohere.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
@@ -93,14 +93,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.cohere.chat.options` can be
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatOptions.java[BedrockCohereChatOptions.java] provides model configurations, such as temperature, topK, topP, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockCohereChatClient(api, options)` constructor or the `spring.ai.bedrock.cohere.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockCohereChatModel(api, options)` constructor or the `spring.ai.bedrock.cohere.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
BedrockCohereChatOptions.builder()
|
||||
@@ -115,7 +115,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Cohere Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Cohere chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -130,37 +130,37 @@ spring.ai.bedrock.cohere.chat.options.temperature=0.8
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockCohereChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockCohereChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockCohereChatClient chatClient;
|
||||
private final BedrockCohereChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockCohereChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockCohereChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClient.java[BedrockCohereChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatModel.java[BedrockCohereChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -183,7 +183,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClient.java[BedrockCohereChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatModel.java[BedrockCohereChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -193,7 +193,7 @@ CohereChatBedrockApi api = new CohereChatBedrockApi(CohereChatModel.COHERE_COMMA
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockCohereChatClient chatClient = new BedrockCohereChatClient(api,
|
||||
BedrockCohereChatModel chatModel = new BedrockCohereChatModel(api,
|
||||
BedrockCohereChatOptions.builder()
|
||||
.withTemperature(0.6f)
|
||||
.withTopK(10)
|
||||
@@ -201,11 +201,11 @@ BedrockCohereChatClient chatClient = new BedrockCohereChatClient(api,
|
||||
.withMaxTokens(678)
|
||||
.build()
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
|====
|
||||
|
||||
|
||||
The prefix `spring.ai.bedrock.jurassic2.chat` is the property prefix that configures the chat client implementation for Jurassic-2.
|
||||
The prefix `spring.ai.bedrock.jurassic2.chat` is the property prefix that configures the chat model implementation for Jurassic-2.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
@@ -86,14 +86,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.jurassic2.chat.options` can
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/jurassic2/BedrockAi21Jurassic2ChatOptions.java[BedrockAi21Jurassic2ChatOptions.java] provides model configurations, such as temperature, topP, maxTokens, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockAi21Jurassic2ChatClient(api, options)` constructor or the `spring.ai.bedrock.jurassic2.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockAi21Jurassic2ChatModel(api, options)` constructor or the `spring.ai.bedrock.jurassic2.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
BedrockAi21Jurassic2ChatOptions.builder()
|
||||
@@ -108,7 +108,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Jurassic-2 Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Jurassic-2 chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -123,24 +123,24 @@ spring.ai.bedrock.jurassic2.chat.options.temperature=0.8
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockAi21Jurassic2ChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockAi21Jurassic2ChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockAi21Jurassic2ChatClient chatClient;
|
||||
private final BedrockAi21Jurassic2ChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockAi21Jurassic2ChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockAi21Jurassic2ChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
}
|
||||
@@ -148,7 +148,7 @@ public class ChatController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/jurassic2/BedrockAi21Jurassic2ChatClient.java[BedrockAi21Jurassic2ChatClient] implements the `ChatClient` uses the <<low-level-api>> to connect to the Bedrock Jurassic-2 service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/jurassic2/BedrockAi21Jurassic2ChatModel.java[BedrockAi21Jurassic2ChatModel] implements the `ChatModel` uses the <<low-level-api>> to connect to the Bedrock Jurassic-2 service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -171,7 +171,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/jurassic2/BedrockAi21Jurassic2ChatClient.java[BedrockAi21Jurassic2ChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/jurassic2/BedrockAi21Jurassic2ChatModel.java[BedrockAi21Jurassic2ChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -181,13 +181,13 @@ Ai21Jurassic2ChatBedrockApi api = new Ai21Jurassic2ChatBedrockApi(Ai21Jurassic2C
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockAi21Jurassic2ChatClient chatClient = new BedrockAi21Jurassic2ChatClient(api,
|
||||
BedrockAi21Jurassic2ChatModel chatModel = new BedrockAi21Jurassic2ChatModel(api,
|
||||
BedrockAi21Jurassic2ChatOptions.builder()
|
||||
.withTemperature(0.5f)
|
||||
.withMaxTokens(100)
|
||||
.withTopP(0.9f).build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
----
|
||||
|
||||
@@ -69,7 +69,7 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
|====
|
||||
|
||||
|
||||
The prefix `spring.ai.bedrock.llama.chat` is the property prefix that configures the chat client implementation for Llama.
|
||||
The prefix `spring.ai.bedrock.llama.chat` is the property prefix that configures the chat model implementation for Llama.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
@@ -91,14 +91,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.llama.chat.options` can be
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatOptions.java[BedrockLlChatOptions.java] provides model configurations, such as temperature, topK, topP, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockLlamaChatClient(api, options)` constructor or the `spring.ai.bedrock.llama.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockLlamaChatModel(api, options)` constructor or the `spring.ai.bedrock.llama.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
BedrockLlamaChatOptions.builder()
|
||||
@@ -113,7 +113,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -128,37 +128,37 @@ spring.ai.bedrock.llama.chat.options.temperature=0.8
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockLlamaChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockLlamaChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockLlamaChatClient chatClient;
|
||||
private final BedrockLlamaChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockLlamaChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockLlamaChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatClient.java[BedrockLlamaChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatModel.java[BedrockLlamaChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -181,7 +181,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatClient.java[BedrockLlamaChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatModel.java[BedrockLlamaChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -191,17 +191,17 @@ LlamaChatBedrockApi api = new LlamaChatBedrockApi(LlamaChatModel.LLAMA2_70B_CHAT
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockLlamaChatClient chatClient = new BedrockLlamaChatClient(api,
|
||||
BedrockLlamaChatModel chatModel = new BedrockLlamaChatModel(api,
|
||||
BedrockLlamaChatOptions.builder()
|
||||
.withTemperature(0.5f)
|
||||
.withMaxGenLen(100)
|
||||
.withTopP(0.9f).build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -65,13 +65,13 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.titan.chat` is the property prefix that configures the chat client implementation for Titan.
|
||||
The prefix `spring.ai.bedrock.titan.chat` is the property prefix that configures the chat model implementation for Titan.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.bedrock.titan.chat.enable | Enable Bedrock Titan chat client. Disabled by default | false
|
||||
| spring.ai.bedrock.titan.chat.enable | Enable Bedrock Titan chat model. Disabled by default | false
|
||||
| spring.ai.bedrock.titan.chat.model | The model id to use. See the link:https://github.com/spring-projects/spring-ai/blob/4839a6175cd1ec89498b97d3efb6647022c3c7cb/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/api/TitanChatBedrockApi.java#L220[TitanChatBedrockApi#TitanChatModel] for the supported models. | amazon.titan-text-lite-v1
|
||||
| spring.ai.bedrock.titan.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.7
|
||||
| spring.ai.bedrock.titan.chat.options.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default
|
||||
@@ -89,14 +89,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.titan.chat.options` can be
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanChatOptions.java[BedrockTitanChatOptions.java] provides model configurations, such as temperature, topP, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockTitanChatClient(api, options)` constructor or the `spring.ai.bedrock.titan.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockTitanChatModel(api, options)` constructor or the `spring.ai.bedrock.titan.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
BedrockTitanChatOptions.builder()
|
||||
@@ -111,7 +111,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Titan Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Titan chat model:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -126,37 +126,37 @@ spring.ai.bedrock.titan.chat.options.temperature=0.8
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockTitanChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockTitanChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final BedrockTitanChatClient chatClient;
|
||||
private final BedrockTitanChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(BedrockTitanChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(BedrockTitanChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanChatClient.java[BedrockTitanChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Titanic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanChatModel.java[BedrockTitanChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Bedrock Titanic service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -179,7 +179,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanChatClient.java[BedrockTitanChatClient] and use it for text generations:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanChatModel.java[BedrockTitanChatModel] and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -190,18 +190,18 @@ TitanChatBedrockApi titanApi = new TitanChatBedrockApi(
|
||||
new ObjectMapper(),
|
||||
Duration.ofMillis(1000L));
|
||||
|
||||
BedrockTitanChatClient chatClient = new BedrockTitanChatClient(titanApi,
|
||||
BedrockTitanChatModel chatModel = new BedrockTitanChatModel(titanApi,
|
||||
BedrockTitanChatOptions.builder()
|
||||
.withTemperature(0.6f)
|
||||
.withTopP(0.8f)
|
||||
.withMaxTokenCount(100)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Anthropic Function Calling
|
||||
|
||||
You can register custom Java functions with the `AnthropicChatClient` and have the Anthropic models intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `AnthropicChatModel` and have the Anthropic models intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The `claude-3-opus`, `claude-3-sonnet` and `claude-3-haiku` link:https://docs.anthropic.com/claude/docs/tool-use#tool-use-best-practices-and-limitations[models are trained to detect when a function should be called] and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -15,7 +15,7 @@ The `description` helps the model to understand when to call the function.
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and respond with the result back to the model.
|
||||
Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../anthropic-chat.html#_auto_configuration[AnthropicChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../anthropic-chat.html#_auto_configuration[AnthropicChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -70,7 +70,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -136,7 +136,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `AnthropicChatClient`.
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `AnthropicChatModel`.
|
||||
It also provides a description (2) and an optional response converter (3) to convert the response into a text as expected by the model.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
@@ -149,17 +149,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
AnthropicChatClient chatClient = ...
|
||||
AnthropicChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
AnthropicChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and produce the final response.
|
||||
|
||||
@@ -169,7 +169,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
AnthropicChatClient chatClient = ...
|
||||
AnthropicChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in Paris?");
|
||||
|
||||
@@ -180,12 +180,12 @@ var promptOptions = AnthropicChatOptions.builder()
|
||||
new MockWeatherService()))) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/anthropic/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `AnthropicChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/anthropic/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `AnthropicChatModel` and use it in a prompt request.
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Function calling lets developers create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.
|
||||
|
||||
You can register custom Java functions with the `AzureOpenAiChatClient` and have the model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `AzureOpenAiChatModel` and have the model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The Azure models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -16,7 +16,7 @@ In general, the custom functions need to provide a function `name`, `description
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and respond with the result back to the model.
|
||||
Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../azure-openai-chat.html#_auto_configuration[AzureOpenAiChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../azure-openai-chat.html#_auto_configuration[AzureOpenAiChatModelAuto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -70,7 +70,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -134,7 +134,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `AzureAiChatClient` and provides a description (2).
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `AzureAiChatModel` and provides a description (2).
|
||||
|
||||
NOTE: The default response converter does a JSON serialization of the Response object.
|
||||
|
||||
@@ -146,17 +146,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
AzureOpenAiChatClient chatClient = ...
|
||||
AzureOpenAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
AzureOpenAiChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
@@ -176,7 +176,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
AzureOpenAiChatClient chatClient = ...
|
||||
AzureOpenAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris? Use Multi-turn function calling.");
|
||||
|
||||
@@ -187,12 +187,12 @@ var promptOptions = AzureOpenAiChatOptions.builder()
|
||||
.build()))
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/azure/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `AzureOpenAiChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/azure/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `AzureOpenAiChatModel` and use it in a prompt request.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Function Calling
|
||||
|
||||
You can register custom Java functions with the `MiniMaxChatClient` and have the MiniMax model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `MiniMaxChatModel` and have the MiniMax model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The MiniMax models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -11,12 +11,12 @@ In general, the custom functions need to provide a function `name`, `descriptio
|
||||
|
||||
As a developer, you need to implement a functions that takes the function call arguments sent from the AI model, and respond with the result back to the model. Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatClient`.
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatModel`.
|
||||
|
||||
|
||||
== How it works
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../minimax-chat.html#_auto_configuration[MiniMaxChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../minimax-chat.html#_auto_configuration[MiniMaxChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -71,7 +71,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -136,7 +136,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party, `MockWeatherService` function and registers it as a `CurrentWeather` function with the `MiniMaxChatClient`.
|
||||
It wraps the 3rd party, `MockWeatherService` function and registers it as a `CurrentWeather` function with the `MiniMaxChatModel`.
|
||||
It also provides a description (2) and an optional response converter (3) to convert the response into a text as expected by the model.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
@@ -149,17 +149,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MiniMaxChatClient chatClient = ...
|
||||
MiniMaxChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
MiniMaxChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
@@ -179,7 +179,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MiniMaxChatClient chatClient = ...
|
||||
MiniMaxChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
@@ -190,18 +190,18 @@ var promptOptions = MiniMaxChatOptions.builder()
|
||||
new MockWeatherService()))) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/minimax/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `MiniMaxChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/minimax/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `MiniMaxChatModel` and use it in a prompt request.
|
||||
//
|
||||
// === Register Functions with Default Options
|
||||
//
|
||||
// You can programmatically register functions with the `MiniMaxChatClient` using the `MiniMaxChatOptions#withFunctionCallbacks`:
|
||||
// You can programmatically register functions with the `MiniMaxChatModel` using the `MiniMaxChatOptions#withFunctionCallbacks`:
|
||||
//
|
||||
// [source,java]
|
||||
// ----
|
||||
@@ -215,12 +215,12 @@ The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot
|
||||
// new MockWeatherService()))) // function code
|
||||
// .build();
|
||||
//
|
||||
// MiniMaxChatClient chatClient = new MiniMaxChatClient(miniMaxApi, defaultOptions);
|
||||
// MiniMaxChatModel chatModel = new MiniMaxChatModel(miniMaxApi, defaultOptions);
|
||||
//
|
||||
// UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
//
|
||||
// ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
// ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
// MiniMaxChatOptions.builder().withFunction("CurrentWeather").build())); // Enable the function
|
||||
// ----
|
||||
//
|
||||
// NOTE: Functions are registered when MiniMaxChatClient is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
// NOTE: Functions are registered when MiniMaxChatModel is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
@@ -1,6 +1,6 @@
|
||||
= Mistral AI Function Calling
|
||||
|
||||
You can register custom Java functions with the `MistralAiChatClient` and have the Mistral AI models intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `MistralAiChatModel` and have the Mistral AI models intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The `open-mixtral-8x22b`, `mistral_small_latest`, and `mistral_large_latest` models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -15,7 +15,7 @@ The `description` helps the model to understand when to call the function.
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and respond with the result back to the model.
|
||||
Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../mistralai-chat.html#_auto_configuration[MistralAiChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../mistralai-chat.html#_auto_configuration[MistralAiChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -70,7 +70,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -139,7 +139,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `MistralAiChatClient`.
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `MistralAiChatModel`.
|
||||
It also provides a description (2) and an optional response converter (3) to convert the response into a text as expected by the model.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
@@ -152,17 +152,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MistralAiChatClient chatClient = ...
|
||||
MistralAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
MistralAiChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and produce the final response.
|
||||
|
||||
@@ -172,7 +172,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MistralAiChatClient chatClient = ...
|
||||
MistralAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in Paris?");
|
||||
|
||||
@@ -183,14 +183,14 @@ var promptOptions = MistralAiChatOptions.builder()
|
||||
new MockWeatherService()))) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/mistralai/tool/PaymentStatusPromptIT.java[PaymentStatusPromptIT.java] integration test provides a complete example of how to register a function with the `MistralAiChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/mistralai/tool/PaymentStatusPromptIT.java[PaymentStatusPromptIT.java] integration test provides a complete example of how to register a function with the `MistralAiChatModel` and use it in a prompt request.
|
||||
|
||||
|
||||
== Appendices
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Function Calling
|
||||
|
||||
You can register custom Java functions with the `OpenAiChatClient` and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `OpenAiChatModel` and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The OpenAI models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -11,12 +11,12 @@ In general, the custom functions need to provide a function `name`, `descriptio
|
||||
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and respond with the result back to the model. Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatClient`.
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatModel`.
|
||||
|
||||
|
||||
== How it works
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../openai-chat.html#_auto_configuration[OpenAiChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../openai-chat.html#_auto_configuration[OpenAiChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -71,7 +71,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -136,7 +136,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `OpenAiChatClient`.
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `OpenAiChatModel`.
|
||||
It also provides a description (2) and an optional response converter (3) to convert the response into a text as expected by the model.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
@@ -149,17 +149,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
OpenAiChatClient chatClient = ...
|
||||
OpenAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
OpenAiChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
@@ -179,7 +179,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
OpenAiChatClient chatClient = ...
|
||||
OpenAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
@@ -190,18 +190,18 @@ var promptOptions = OpenAiChatOptions.builder()
|
||||
new MockWeatherService()))) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `OpenAiChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `OpenAiChatModel` and use it in a prompt request.
|
||||
//
|
||||
// === Register Functions with Default Options
|
||||
//
|
||||
// You can programmatically register functions with the `OpenAiChatClient` using the `OpenAiChatOptions#withFunctionCallbacks`:
|
||||
// You can programmatically register functions with the `OpenAiChatModel` using the `OpenAiChatOptions#withFunctionCallbacks`:
|
||||
//
|
||||
// [source,java]
|
||||
// ----
|
||||
@@ -215,24 +215,24 @@ The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot
|
||||
// new MockWeatherService()))) // function code
|
||||
// .build();
|
||||
//
|
||||
// OpenAiChatClient chatClient = new OpenAiChatClient(openaiApi, defaultOptions);
|
||||
// OpenAiChatModel chatModel = new OpenAiChatModel(openaiApi, defaultOptions);
|
||||
//
|
||||
// UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
//
|
||||
// ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
// ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
// OpenAiChatOptions.builder().withFunction("CurrentWeather").build())); // Enable the function
|
||||
// ----
|
||||
//
|
||||
// NOTE: Functions are registered when OpenAiChatClient is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
// NOTE: Functions are registered when OpenAiChatModel is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
|
||||
|
||||
== Appendices:
|
||||
|
||||
=== Spring AI Function Calling Flow [[spring-ai-function-calling-flow]]
|
||||
|
||||
The following diagram illustrates the flow of the OpenAiChatClient Function Calling:
|
||||
The following diagram illustrates the flow of the OpenAiChatModel Function Calling:
|
||||
|
||||
image:openai-chatclient-function-call.jpg[width=800, title="OpenAiChatClient Function Calling Flow"]
|
||||
image:openai-chatclient-function-call.jpg[width=800, title="OpenAiChatModel Function Calling Flow"]
|
||||
|
||||
=== OpenAI API Function Calling Flow
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ The parallel function calling is gone as well.
|
||||
|
||||
Function calling lets developers create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.
|
||||
|
||||
You can register custom Java functions with the `VertexAiGeminiChatClient` and have the Gemini Pro model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `VertexAiGeminiChatModel` and have the Gemini Pro model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The VertexAI Gemini Pro model is trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -18,12 +18,12 @@ In general, the custom functions need to provide a function `name`, `description
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and respond with the result back to the model.
|
||||
Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatClient`.
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatModel`.
|
||||
|
||||
== How it works
|
||||
|
||||
@@ -66,7 +66,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../vertexai-gemini-chat.html#_auto_configuration[VertexAiGeminiChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../vertexai-gemini-chat.html#_auto_configuration[VertexAiGeminiChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -74,7 +74,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -139,7 +139,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `VertexAiGeminiChatClient`.
|
||||
It wraps the 3rd party `MockWeatherService` function and registers it as a `CurrentWeather` function with the `VertexAiGeminiChatModel`.
|
||||
It also provides a description (2) and sets the Schema type to Open API type (3).
|
||||
|
||||
NOTE: The default response converter does a JSON serialization of the Response object.
|
||||
@@ -152,17 +152,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
VertexAiGeminiChatClient chatClient = ...
|
||||
VertexAiGeminiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
VertexAiGeminiChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
@@ -182,7 +182,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
VertexAiGeminiChatClient chatClient = ...
|
||||
VertexAiGeminiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris? Use Multi-turn function calling.");
|
||||
|
||||
@@ -194,12 +194,12 @@ var promptOptions = VertexAiGeminiChatOptions.builder()
|
||||
.build()))
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/gemini/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `VertexAiGeminiChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/gemini/tool/FunctionCallWithPromptFunctionIT.java[FunctionCallWithPromptFunctionIT.java] integration test provides a complete example of how to register a function with the `VertexAiGeminiChatModel` and use it in a prompt request.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Function Calling
|
||||
|
||||
You can register custom Java functions with the `ZhiPuAiChatClient` and have the ZhiPuAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the `ZhiPuAiChatModel` and have the ZhiPuAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The ZhiPuAI models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
|
||||
@@ -11,12 +11,12 @@ In general, the custom functions need to provide a function `name`, `descriptio
|
||||
|
||||
As a developer, you need to implement a functions that takes the function call arguments sent from the AI model, and respond with the result back to the model. Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallbackWrapper.java[FunctionCallbackWrapper.java] utility class to simplify the implementation and registration of Java callback functions.
|
||||
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatClient`.
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatModel`.
|
||||
|
||||
|
||||
== How it works
|
||||
@@ -62,7 +62,7 @@ public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../zhipuai-chat.html#_auto_configuration[ZhiPuAiChatClient Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
With the link:../zhipuai-chat.html#_auto_configuration[ZhiPuAiChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
@@ -71,7 +71,7 @@ We start with describing the most POJO friendly options.
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatClient` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallbackWrapper` wrapper that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
@@ -136,7 +136,7 @@ static class Config {
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party, `MockWeatherService` function and registers it as a `CurrentWeather` function with the `ZhiPuAiChatClient`.
|
||||
It wraps the 3rd party, `MockWeatherService` function and registers it as a `CurrentWeather` function with the `ZhiPuAiChatModel`.
|
||||
It also provides a description (2) and an optional response converter (3) to convert the response into a text as expected by the model.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
@@ -149,17 +149,17 @@ To let the model know and call your `CurrentWeather` function you need to enable
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ZhiPuAiChatClient chatClient = ...
|
||||
ZhiPuAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
ZhiPuAiChatOptions.builder().withFunction("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatClient` but only those enabled in the prompt request will be considered for the function calling.
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
@@ -179,7 +179,7 @@ In addition to the auto-configuration you can register callback functions, dynam
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ZhiPuAiChatClient chatClient = ...
|
||||
ZhiPuAiChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
@@ -190,18 +190,18 @@ var promptOptions = ZhiPuAiChatOptions.builder()
|
||||
new MockWeatherService()))) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage), promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/zhipuai/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `ZhiPuAiChatClient` and use it in a prompt request.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/zhipuai/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `ZhiPuAiChatModel` and use it in a prompt request.
|
||||
//
|
||||
// === Register Functions with Default Options
|
||||
//
|
||||
// You can programmatically register functions with the `ZhiPuAiChatClient` using the `ZhiPuAiChatOptions#withFunctionCallbacks`:
|
||||
// You can programmatically register functions with the `ZhiPuAiChatModel using the `ZhiPuAiChatOptions#withFunctionCallbacks`:
|
||||
//
|
||||
// [source,java]
|
||||
// ----
|
||||
@@ -215,12 +215,12 @@ The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot
|
||||
// new MockWeatherService()))) // function code
|
||||
// .build();
|
||||
//
|
||||
// ZhiPuAiChatClient chatClient = new ZhiPuAiChatClient(zhiPuAiApi, defaultOptions);
|
||||
// ZhiPuAiChatModel chatModel = new ZhiPuAiChatModel(zhiPuAiApi, defaultOptions);
|
||||
//
|
||||
// UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
//
|
||||
// ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
// ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
// ZhiPuAiChatOptions.builder().withFunction("CurrentWeather").build())); // Enable the function
|
||||
// ----
|
||||
//
|
||||
// NOTE: Functions are registered when ZhiPuAiChatClient is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
// NOTE: Functions are registered when ZhiPuAiChatModel is created, by you must enable in the Prompt the functions to be used in the request.
|
||||
@@ -27,7 +27,7 @@ export HUGGINGFACE_API_KEY=your_api_key_here
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Note, there is not yet a Spring Boot Starter for this client implementation.
|
||||
Note, there is not yet a Spring Boot Starter for this chat implementation.
|
||||
|
||||
Obtain the endpoint URL of the Inference Endpoint.
|
||||
You can find this on the Inference Endpoint's UI link:https://ui.endpoints.huggingface.co/[here].
|
||||
@@ -36,9 +36,9 @@ You can find this on the Inference Endpoint's UI link:https://ui.endpoints.huggi
|
||||
|
||||
[source,java]
|
||||
----
|
||||
HuggingfaceChatClient client = new HuggingfaceChatClient(apiKey, basePath);
|
||||
HuggingfaceChatModel chatModel = new HuggingfaceChatModel(apiKey, basePath);
|
||||
Prompt prompt = new Prompt("Your text here...");
|
||||
ChatResponse response = client.call(prompt);
|
||||
ChatResponse response = chatModel.call(prompt);
|
||||
System.out.println(response.getGeneration().getText());
|
||||
----
|
||||
|
||||
@@ -56,7 +56,7 @@ String mistral7bInstruct = """
|
||||
Just generate the JSON object without explanations:
|
||||
[/INST]""";
|
||||
Prompt prompt = new Prompt(mistral7bInstruct);
|
||||
ChatResponse aiResponse = huggingfaceChatClient.call(prompt);
|
||||
ChatResponse aiResponse = huggingfaceChatModel.call(prompt);
|
||||
System.out.println(response.getGeneration().getText());
|
||||
----
|
||||
Will produce the output
|
||||
|
||||
@@ -52,7 +52,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the MiniMax Chat client.
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the MiniMax chat model.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -81,13 +81,13 @@ The prefix `spring.ai.minimax` is used as the property prefix that lets you conn
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.minimax.chat` is the property prefix that lets you configure the chat client implementation for MiniMax.
|
||||
The prefix `spring.ai.minimax.chat` is the property prefix that lets you configure the chat model implementation for MiniMax.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.minimax.chat.enabled | Enable MiniMax chat client. | true
|
||||
| spring.ai.minimax.chat.enabled | Enable MiniMax chat model. | true
|
||||
| spring.ai.minimax.chat.base-url | Optional overrides the spring.ai.minimax.base-url to provide chat specific url | https://api.minimax.chat
|
||||
| spring.ai.minimax.chat.api-key | Optional overrides the spring.ai.minimax.api-key to provide chat specific api-key | -
|
||||
| spring.ai.minimax.chat.options.model | This is the MiniMax Chat model to use | `abab5.5-chat` (the `abab5.5s-chat`, `abab5.5-chat`, and `abab6-chat` point to the latest model versions)
|
||||
@@ -100,7 +100,7 @@ The prefix `spring.ai.minimax.chat` is the property prefix that lets you configu
|
||||
| spring.ai.minimax.chat.options.stop | The model will stop generating characters specified by stop, and currently only supports a single stop word in the format of ["stop_word1"] | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.minimax.base-url` and `spring.ai.minimax.api-key` for the `ChatClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.minimax.base-url` and `spring.ai.minimax.api-key` for the `ChatModel` implementations.
|
||||
The `spring.ai.minimax.chat.base-url` and `spring.ai.minimax.chat.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different MiniMax accounts for different models and different model endpoints.
|
||||
|
||||
@@ -110,14 +110,14 @@ TIP: All properties prefixed with `spring.ai.minimax.chat.options` can be overri
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-minimax/src/main/java/org/springframework/ai/minimax/MiniMaxChatOptions.java[MiniMaxChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `MiniMaxChatClient(api, options)` constructor or the `spring.ai.minimax.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `MiniMaxChatModel(api, options)` constructor or the `spring.ai.minimax.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
MiniMaxChatOptions.builder()
|
||||
@@ -133,7 +133,7 @@ TIP: In addition to the model specific link:https://github.com/spring-projects/s
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-minimax-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the MiniMax Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the MiniMax chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -144,37 +144,37 @@ spring.ai.minimax.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `api-key` with your MiniMax credentials.
|
||||
|
||||
This will create a `MiniMaxChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `MiniMaxChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final MiniMaxChatClient chatClient;
|
||||
private final MiniMaxChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(MiniMaxChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(MiniMaxChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
var prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-minimax/src/main/java/org/springframework/ai/minimax/MiniMaxChatClient.java[MiniMaxChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the MiniMax service.
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-minimax/src/main/java/org/springframework/ai/minimax/MiniMaxChatModel.java[MiniMaxChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the MiniMax service.
|
||||
|
||||
Add the `spring-ai-minimax` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -197,23 +197,23 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `MiniMaxChatClient` and use it for text generations:
|
||||
Next, create a `MiniMaxChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var miniMaxApi = new MiniMaxApi(System.getenv("MINIMAX_API_KEY"));
|
||||
|
||||
var chatClient = new MiniMaxChatClient(miniMaxApi, MiniMaxChatOptions.builder()
|
||||
var chatModel = new MiniMaxChatModel(miniMaxApi, MiniMaxChatOptions.builder()
|
||||
.withModel(MiniMaxApi.ChatModel.GLM_3_Turbo.getValue())
|
||||
.withTemperature(0.4f)
|
||||
.withMaxTokens(200)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> streamResponse = chatClient.stream(
|
||||
Flux<ChatResponse> streamResponse = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -51,7 +51,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the Mistral AI Chat client.
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the Mistral AI chat model.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -80,13 +80,13 @@ The prefix `spring.ai.mistralai` is used as the property prefix that lets you co
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.mistralai.chat` is the property prefix that lets you configure the chat client implementation for MistralAI.
|
||||
The prefix `spring.ai.mistralai.chat` is the property prefix that lets you configure the chat model implementation for MistralAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.mistralai.chat.enabled | Enable MistralAI chat client. | true
|
||||
| spring.ai.mistralai.chat.enabled | Enable MistralAI chat model. | true
|
||||
| spring.ai.mistralai.chat.base-url | Optional overrides the spring.ai.mistralai.base-url to provide chat specific url | -
|
||||
| spring.ai.mistralai.chat.api-key | Optional overrides the spring.ai.mistralai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.mistralai.chat.options.model | This is the MistralAI Chat model to use | `open-mistral-7b`, `open-mixtral-8x7b`, `mistral-small-latest`, `mistral-medium-latest`, `mistral-large-latest`
|
||||
@@ -100,10 +100,10 @@ The prefix `spring.ai.mistralai.chat` is the property prefix that lets you confi
|
||||
| spring.ai.mistralai.chat.options.tools | A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. | -
|
||||
| spring.ai.mistralai.chat.options.toolChoice | Controls which (if any) function is called by the model. none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function. Specifying a particular function via {"type: "function", "function": {"name": "my_function"}} forces the model to call that function. none is the default when no functions are present. auto is the default if functions are present. | -
|
||||
| spring.ai.mistralai.chat.options.functions | List of functions, identified by their names, to enable for function calling in a single prompt requests. Functions with those names must exist in the functionCallbacks registry. | -
|
||||
| spring.ai.mistralai.chat.options.functionCallbacks | MistralAI Tool Function Callbacks to register with the ChatClient. | -
|
||||
| spring.ai.mistralai.chat.options.functionCallbacks | MistralAI Tool Function Callbacks to register with the ChatModel. | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.mistralai.base-url` and `spring.ai.mistralai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.mistralai.base-url` and `spring.ai.mistralai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.mistralai.chat.base-url` and `spring.ai.mistralai.chat.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different MistralAI accounts for different models and different model endpoints.
|
||||
|
||||
@@ -113,14 +113,14 @@ TIP: All properties prefixed with `spring.ai.mistralai.chat.options` can be over
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/main/java/org/springframework/ai/mistralai/MistralAiChatOptions.java[MistralAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `MistralAiChatClient(api, options)` constructor or the `spring.ai.mistralai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `MistralAiChatModel(api, options)` constructor or the `spring.ai.mistralai.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
MistralAiChatOptions.builder()
|
||||
@@ -134,7 +134,7 @@ TIP: In addition to the model specific link:https://github.com/spring-projects/s
|
||||
|
||||
== Function Calling
|
||||
|
||||
You can register custom Java functions with the MistralAiChatClient and have the Mistral AI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the MistralAiChatModel and have the Mistral AI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This is a powerful technique to connect the LLM capabilities with external tools and APIs.
|
||||
Read more about xref:api/chat/functions/mistralai-chat-functions.adoc[Mistral AI Function Calling].
|
||||
|
||||
@@ -142,7 +142,7 @@ Read more about xref:api/chat/functions/mistralai-chat-functions.adoc[Mistral AI
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-mistral-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -153,37 +153,37 @@ spring.ai.mistralai.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `api-key` with your OpenAI credentials.
|
||||
|
||||
This will create a `MistralAiChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `MistralAiChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final MistralAiChatClient chatClient;
|
||||
private final MistralAiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(MistralAiChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(MistralAiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
var prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/main/java/org/springframework/ai/mistralai/MistralAiChatClient.java[MistralAiChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the MistralAI service.
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/main/java/org/springframework/ai/mistralai/MistralAiChatModel.java[MistralAiChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the MistralAI service.
|
||||
|
||||
Add the `spring-ai-mistral-ai` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -206,23 +206,23 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `MistralAiChatClient` and use it for text generations:
|
||||
Next, create a `MistralAiChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var mistralAiApi = new MistralAiApi(System.getenv("MISTRAL_AI_API_KEY"));
|
||||
|
||||
var chatClient = new MistralAiChatClient(mistralAiApi, MistralAiChatOptions.builder()
|
||||
var chatModel = new MistralAiChatModel(mistralAiApi, MistralAiChatOptions.builder()
|
||||
.withModel(MistralAiApi.ChatModel.LARGE.getValue())
|
||||
.withTemperature(0.4f)
|
||||
.withMaxToken(200)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
= Ollama Chat
|
||||
|
||||
With https://ollama.ai/[Ollama] you can run various Large Language Models (LLMs) locally and generate text from them.
|
||||
Spring AI supports the Ollama text generation with `OllamaChatClient`.
|
||||
Spring AI supports the Ollama text generation with `OllamaChatModel`.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
@@ -52,16 +52,16 @@ The prefix `spring.ai.ollama` is the property prefix to configure the connection
|
||||
| spring.ai.ollama.base-url | Base URL where Ollama API server is running. | `http://localhost:11434`
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.ollama.chat.options` is the property prefix that configures the Ollama chat client .
|
||||
The prefix `spring.ai.ollama.chat.options` is the property prefix that configures the Ollama chat model .
|
||||
It includes the Ollama request (advanced) parameters such as the `model`, `keep-alive`, and `format` as well as the Ollama model `options` properties.
|
||||
|
||||
Here are the advanced request parameter for the Ollama chat client:
|
||||
Here are the advanced request parameter for the Ollama chat model:
|
||||
|
||||
[cols="3,6,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.ollama.chat.enabled | Enable Ollama chat client. | true
|
||||
| spring.ai.ollama.chat.enabled | Enable Ollama chat model. | true
|
||||
| spring.ai.ollama.chat.options.model | The name of the https://github.com/ollama/ollama?tab=readme-ov-file#model-library[supported model] to use. | mistral
|
||||
| spring.ai.ollama.chat.options.format | The format to return a response in. Currently the only accepted value is `json` | -
|
||||
| spring.ai.ollama.chat.options.keep_alive | Controls how long the model will stay loaded into memory following the request | 5m
|
||||
@@ -110,14 +110,14 @@ TIP: All properties prefixed with `spring.ai.ollama.chat.options` can be overrid
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java[OllamaOptions.java] provides model configurations, such as the model to use, the temperature, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `OllamaChatClient(api, options)` constructor or the `spring.ai.ollama.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `OllamaChatModel(api, options)` constructor or the `spring.ai.ollama.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
OllamaOptions.create()
|
||||
@@ -141,7 +141,7 @@ The Ollama link:https://github.com/ollama/ollama/blob/main/docs/api.md#parameter
|
||||
Spring AI’s link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/messages/Message.java[Message] interface facilitates multimodal AI models by introducing the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/messages/Media.java[Media] type.
|
||||
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s `org.springframework.util.MimeType` and a `java.lang.Object` for the raw media data.
|
||||
|
||||
Below is a straightforward code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaChatClientMultimodalIT.java[OllamaChatClientMultimodalIT.java], illustrating the fusion of user text with an image.
|
||||
Below is a straightforward code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaChatModelMultimodalIT.java[OllamaChatModelMultimodalIT.java], illustrating the fusion of user text with an image.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -150,7 +150,7 @@ byte[] imageData = new ClassPathResource("/multimodal.test.png").getContentAsByt
|
||||
var userMessage = new UserMessage("Explain what do you see on this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)));
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(List.of(userMessage), OllamaOptions.create().withModel("llava")));
|
||||
|
||||
logger.info(response.getResult().getOutput().getContent());
|
||||
@@ -174,7 +174,7 @@ where fruits are being displayed, possibly for convenience or aesthetic purposes
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-ollama-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Ollama Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Ollama chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -185,30 +185,30 @@ spring.ai.ollama.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `base-url` with your Ollama server URL.
|
||||
|
||||
This will create a `OllamaChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `OllamaChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final OllamaChatClient chatClient;
|
||||
private final OllamaChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(OllamaChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(OllamaChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
|
||||
}
|
||||
@@ -216,8 +216,8 @@ public class ChatController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
If you don't want to use the Spring Boot auto-configuration, you can manually configure the `OllamaChatClient` in your application.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/OllamaChatClient.java[OllamaChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Ollama service.
|
||||
If you don't want to use the Spring Boot auto-configuration, you can manually configure the `OllamaChatModel` in your application.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/OllamaChatModel.java[OllamaChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Ollama service.
|
||||
|
||||
To use it add the `spring-ai-ollama` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -240,25 +240,25 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
TIP: The `spring-ai-ollama` dependency provides access also to the `OllamaEmbeddingClient`.
|
||||
For more information about the `OllamaEmbeddingClient` refer to the link:../embeddings/ollama-embeddings.html[Ollama Embedding Client] section.
|
||||
TIP: The `spring-ai-ollama` dependency provides access also to the `OllamaEmbeddingModel`.
|
||||
For more information about the `OllamaEmbeddingModel` refer to the link:../embeddings/ollama-embeddings.html[Ollama Embedding Client] section.
|
||||
|
||||
Next, create an `OllamaChatClient` instance and use it to text generations requests:
|
||||
Next, create an `OllamaChatModel` instance and use it to text generations requests:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var ollamaApi = new OllamaApi();
|
||||
|
||||
var chatClient = new OllamaChatClient(ollamaApi,
|
||||
var chatModel = new OllamaChatModel(ollamaApi,
|
||||
OllamaOptions.create()
|
||||
.withModel(OllamaOptions.DEFAULT_MODEL)
|
||||
.withTemperature(0.9f));
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
@@ -274,7 +274,7 @@ image::ollama-chat-completion-api.jpg[OllamaApi Chat Completion API Diagram, 800
|
||||
|
||||
Here is a simple snippet showing how to use the API programmatically:
|
||||
|
||||
NOTE: The `OllamaApi` is low level api and is not recommended for direct use. Use the `OllamaChatClient` instead.
|
||||
NOTE: The `OllamaApi` is low level api and is not recommended for direct use. Use the `OllamaChatModel` instead.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
|
||||
@@ -51,7 +51,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the OpenAI Chat client.
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the OpenAI chat model.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -81,13 +81,13 @@ The prefix `spring.ai.openai` is used as the property prefix that lets you conne
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.openai.chat` is the property prefix that lets you configure the chat client implementation for OpenAI.
|
||||
The prefix `spring.ai.openai.chat` is the property prefix that lets you configure the chat model implementation for OpenAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.openai.chat.enabled | Enable OpenAI chat client. | true
|
||||
| spring.ai.openai.chat.enabled | Enable OpenAI chat model. | true
|
||||
| spring.ai.openai.chat.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
|
||||
| spring.ai.openai.chat.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.openai.chat.options.model | This is the OpenAI Chat model to use | `gpt-3.5-turbo` (the `gpt-3.5-turbo`, `gpt-4`, and `gpt-4-32k` point to the latest model versions)
|
||||
@@ -107,7 +107,7 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
|
||||
| spring.ai.openai.chat.options.functions | List of functions, identified by their names, to enable for function calling in a single prompt requests. Functions with those names must exist in the functionCallbacks registry. | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.openai.base-url` and `spring.ai.openai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.openai.base-url` and `spring.ai.openai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.openai.chat.base-url` and `spring.ai.openai.chat.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
|
||||
|
||||
@@ -117,14 +117,14 @@ TIP: All properties prefixed with `spring.ai.openai.chat.options` can be overrid
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatOptions.java[OpenAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `OpenAiChatClient(api, options)` constructor or the `spring.ai.openai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `OpenAiChatModel(api, options)` constructor or the `spring.ai.openai.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
OpenAiChatOptions.builder()
|
||||
@@ -138,7 +138,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
== Function Calling
|
||||
|
||||
You can register custom Java functions with the OpenAiChatClient and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the OpenAiChatModel and have the OpenAI model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This is a powerful technique to connect the LLM capabilities with external tools and APIs.
|
||||
Read more about xref:api/chat/functions/openai-chat-functions.adoc[OpenAI Function Calling].
|
||||
|
||||
@@ -152,7 +152,7 @@ The OpenAI link:https://platform.openai.com/docs/api-reference/chat/create#chat-
|
||||
Spring AI’s link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/messages/Message.java[Message] interface facilitates multimodal AI models by introducing the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/messages/Media.java[Media] type.
|
||||
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s `org.springframework.util.MimeType` and a `java.lang.Object` for the raw media data.
|
||||
|
||||
Below is a code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/b3cfa2b900ea785e055e4ff71086eeb52f6578a3/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatClientIT.java[OpenAiChatClientIT.java], illustrating the fusion of user text with an image using the the `GPT_4_VISION_PREVIEW` model.
|
||||
Below is a code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/b3cfa2b900ea785e055e4ff71086eeb52f6578a3/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatModelIT.java[OpenAiChatModelIT.java], illustrating the fusion of user text with an image using the the `GPT_4_VISION_PREVIEW` model.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -161,7 +161,7 @@ byte[] imageData = new ClassPathResource("/multimodal.test.png").getContentAsByt
|
||||
var userMessage = new UserMessage("Explain what do you see on this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)));
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
OpenAiChatOptions.builder().withModel(OpenAiApi.ChatModel.GPT_4_VISION_PREVIEW.getValue()).build()));
|
||||
----
|
||||
|
||||
@@ -173,7 +173,7 @@ var userMessage = new UserMessage("Explain what do you see on this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG,
|
||||
"https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/_images/multimodal.test.png")));
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage),
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
|
||||
OpenAiChatOptions.builder().withModel(OpenAiApi.ChatModel.GPT_4_O.getValue()).build()));
|
||||
----
|
||||
|
||||
@@ -198,7 +198,7 @@ view of the fruit inside.
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-openai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -209,37 +209,37 @@ spring.ai.openai.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `api-key` with your OpenAI credentials.
|
||||
|
||||
This will create a `OpenAiChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `OpenAiChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final OpenAiChatClient chatClient;
|
||||
private final OpenAiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(OpenAiChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(OpenAiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatClient.java[OpenAiChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the OpenAI service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatModel.java[OpenAiChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the OpenAI service.
|
||||
|
||||
Add the `spring-ai-openai` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -262,7 +262,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `OpenAiChatClient` and use it for text generations:
|
||||
Next, create a `OpenAiChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -272,14 +272,14 @@ var openAiChatOptions = OpenAiChatOptions.builder()
|
||||
.withTemperature(0.4)
|
||||
.withMaxTokens(200)
|
||||
.build();
|
||||
var chatClient = new OpenAiChatClient(openAiApi, openAiChatOptions)
|
||||
var chatModel = new OpenAiChatModel(openAiApi, openAiChatOptions)
|
||||
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> response = chatClient.stream(
|
||||
Flux<ChatResponse> response = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -59,7 +59,7 @@ The prefix `spring.ai.vertex.ai.gemini` is used as the property prefix that lets
|
||||
| spring.ai.vertex.ai.gemini.transport | API transport. GRPC or REST. | GRPC
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.vertex.ai.gemini.chat` is the property prefix that lets you configure the chat client implementation for VertexAI Gemini Chat.
|
||||
The prefix `spring.ai.vertex.ai.gemini.chat` is the property prefix that lets you configure the chat model implementation for VertexAI Gemini Chat.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -84,14 +84,14 @@ TIP: All properties prefixed with `spring.ai.vertex.ai.gemini.chat.options` can
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatOptions.java[VertexAiGeminiChatOptions.java] provides model configurations, such as the temperature, the topK, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `VertexAiGeminiChatClient(api, options)` constructor or the `spring.ai.vertex.ai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `VertexAiGeminiChatModel(api, options)` constructor or the `spring.ai.vertex.ai.chat.options.*` properties.
|
||||
|
||||
At runtime you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
VertexAiPaLm2ChatOptions.builder()
|
||||
@@ -109,7 +109,7 @@ WARNING: As of 30th of April 2023, the Vertex AI `Gemini Pro` model has signific
|
||||
Apparently the Gemini Pro can not handle anymore the function name correctly.
|
||||
The parallel function calling is gone as well.
|
||||
|
||||
You can register custom Java functions with the VertexAiGeminiChatClient and have the Gemini Pro model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
You can register custom Java functions with the VertexAiGeminiChatModel and have the Gemini Pro model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This is a powerful technique to connect the LLM capabilities with external tools and APIs.
|
||||
Read more about xref:api/chat/functions/vertexai-gemini-chat-functions.adoc[Vertex AI Gemini Function Calling].
|
||||
|
||||
@@ -122,7 +122,7 @@ Google's Gemini AI models support this capability by comprehending and integrati
|
||||
Spring AI's `Message` interface supports multimodal AI models by introducing the Media type.
|
||||
This type contains data and information about media attachments in messages, using Spring's `org.springframework.util.MimeType` and a `java.lang.Object` for the raw media data.
|
||||
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/test/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatClientIT.java[VertexAiGeminiChatClientIT.java], demonstrating the combination of user text with an image.
|
||||
Below is a simple code example extracted from https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/test/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModelIT.java[VertexAiGeminiChatModelIT.java], demonstrating the combination of user text with an image.
|
||||
|
||||
|
||||
[source,java]
|
||||
@@ -132,14 +132,14 @@ byte[] data = new ClassPathResource("/vertex-test.png").getContentAsByteArray();
|
||||
var userMessage = new UserMessage("Explain what do you see o this picture?",
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, data)));
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage)));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage)));
|
||||
----
|
||||
|
||||
== Sample Controller
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-vertex-ai-palm2-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -151,37 +151,37 @@ spring.ai.vertex.ai.gemini.chat.options.temperature=0.5
|
||||
|
||||
TIP: replace the `api-key` with your VertexAI credentials.
|
||||
|
||||
This will create a `VertexAiGeminiChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `VertexAiGeminiChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final VertexAiGeminiChatClient chatClient;
|
||||
private final VertexAiGeminiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(VertexAiGeminiChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(VertexAiGeminiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatClient.java[VertexAiGeminiChatClient] implements the `ChatClient` and uses the `VertexAI` to connect to the Vertex AI Gemini service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/main/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModel.java[VertexAiGeminiChatModel] implements the `ChatModel` and uses the `VertexAI` to connect to the Vertex AI Gemini service.
|
||||
|
||||
Add the `spring-ai-vertex-ai-gemini` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -204,19 +204,19 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `VertexAiGeminiChatClient` and use it for text generations:
|
||||
Next, create a `VertexAiGeminiChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
VertexAI vertexApi = new VertexAI(projectId, location);
|
||||
|
||||
var chatClient = new VertexAiGeminiChatClient(vertexApi,
|
||||
var chatModel = new VertexAiGeminiChatModel(vertexApi,
|
||||
VertexAiGeminiChatOptions.builder()
|
||||
.withModel(ChatModel.GEMINI_PRO_1_5_PRO)
|
||||
.withTemperature(0.4)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -61,13 +61,13 @@ The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you co
|
||||
| spring.ai.vertex.ai.api-key | The API Key | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.vertex.ai.chat` is the property prefix that lets you configure the chat client implementation for VertexAI Chat.
|
||||
The prefix `spring.ai.vertex.ai.chat` is the property prefix that lets you configure the chat model implementation for VertexAI Chat.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.vertex.ai.chat.enabled | Enable Vertex AI PaLM API Chat client. | true
|
||||
| spring.ai.vertex.ai.chat.enabled | Enable Vertex AI PaLM API chat model. | true
|
||||
| spring.ai.vertex.ai.chat.model | This is the https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-chat[Vertex Chat model] to use | chat-bison-001
|
||||
| spring.ai.vertex.ai.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0], inclusive. A value closer to 1.0 will produce responses that are more varied, while a value closer to 0.0 will typically result in less surprising responses from the generative. This value specifies default to be used by the backend while making the call to the generative. | 0.7
|
||||
| spring.ai.vertex.ai.chat.options.topK | The maximum number of tokens to consider when sampling. The generative uses combined Top-k and nucleus sampling. Top-k sampling considers the set of topK most probable tokens. | 20
|
||||
@@ -81,14 +81,14 @@ TIP: All properties prefixed with `spring.ai.vertex.ai.chat.options` can be over
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-palm2/src/main/java/org/springframework/ai/vertexai/palm2/VertexAiPaLm2ChatOptions.java[VertexAiPaLm2ChatOptions.java] provides model configurations, such as the temperature, the topK, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `VertexAiPaLm2ChatClient(api, options)` constructor or the `spring.ai.vertex.ai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `VertexAiPaLm2ChatModel(api, options)` constructor or the `spring.ai.vertex.ai.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
VertexAiPaLm2ChatOptions.builder()
|
||||
@@ -103,7 +103,7 @@ TIP: In addition to the model specific `VertexAiPaLm2ChatOptions` you can use a
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-vertex-ai-palm2-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -114,37 +114,37 @@ spring.ai.vertex.ai.chat.options.temperature=0.5
|
||||
|
||||
TIP: replace the `api-key` with your VertexAI credentials.
|
||||
|
||||
This will create a `VertexAiPaLm2ChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `VertexAiPaLm2ChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final VertexAiPaLm2ChatClient chatClient;
|
||||
private final VertexAiPaLm2ChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(VertexAiPaLm2ChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(VertexAiPaLm2ChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
Prompt prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertexai/paml2/VertexAiPaLm2ChatClient.java[VertexAiPaLm2ChatClient] implements the `ChatClient` and uses the <<low-level-api>> to connect to the VertexAI service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertexai/paml2/VertexAiPaLm2ChatModel.java[VertexAiPaLm2ChatModel] implements the `ChatModel` and uses the <<low-level-api>> to connect to the VertexAI service.
|
||||
|
||||
Add the `spring-ai-vertex-ai-palm2` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -167,18 +167,18 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `VertexAiPaLm2ChatClient` and use it for text generations:
|
||||
Next, create a `VertexAiPaLm2ChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
VertexAiPaLm2Api vertexAiApi = new VertexAiPaLm2Api(< YOUR PALM_API_KEY>);
|
||||
|
||||
var chatClient = new VertexAiPaLm2ChatClient(vertexAiApi,
|
||||
var chatModel = new VertexAiPaLm2ChatModel(vertexAiApi,
|
||||
VertexAiPaLm2ChatOptions.builder()
|
||||
.withTemperature(0.4)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
= watsonx.ai Chat
|
||||
|
||||
With https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/overview-wx.html?context=wx&audience=wdp[watsonx.ai] you can run various Large Language Models (LLMs) locally and generate text from them.
|
||||
Spring AI supports the watsonx.ai text generation with `WatsonxAiChatClient`.
|
||||
Spring AI supports the watsonx.ai text generation with `WatsonxAiChatModel`.
|
||||
|
||||
|
||||
== Prerequisites
|
||||
@@ -53,13 +53,13 @@ The prefix `spring.ai.watsonx.ai` is used as the property prefix that lets you c
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.watsonx.ai.chat` is the property prefix that lets you configure the chat client implementation for Watsonx.AI.
|
||||
The prefix `spring.ai.watsonx.ai.chat` is the property prefix that lets you configure the chat model implementation for Watsonx.AI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.watsonx.ai.chat.enabled | Enable Watsonx.AI chat client. | true
|
||||
| spring.ai.watsonx.ai.chat.enabled | Enable Watsonx.AI chat model. | true
|
||||
| spring.ai.watsonx.ai.chat.options.temperature | The temperature of the model. Increasing the temperature will make the model answer more creatively. | 0.7
|
||||
| spring.ai.watsonx.ai.chat.options.top-p | Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.2) will generate more focused and conservative text. | 1.0
|
||||
| spring.ai.watsonx.ai.chat.options.top-k | Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. | 50
|
||||
@@ -76,14 +76,14 @@ The prefix `spring.ai.watsonx.ai.chat` is the property prefix that lets you conf
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-watsonx-ai/src/main/java/org/springframework/ai/watsonx/WatsonxAiChatOptions.java[WatsonxAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `WatsonxAiChatClient(api, options)` constructor or the `spring.ai.watsonxai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `WatsonxAiChatModel(api, options)` constructor or the `spring.ai.watsonxai.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
WatsonxAiChatOptions.builder()
|
||||
@@ -103,11 +103,11 @@ NOTE: For more information go to https://dataplatform.cloud.ibm.com/docs/content
|
||||
public class MyClass {
|
||||
|
||||
private final static String MODEL = "google/flan-ul2";
|
||||
private final WatsonxAiChatClient chat;
|
||||
private final WatsonxAiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
MyClass(WatsonxAiChatClient chat) {
|
||||
this.chat = chat;
|
||||
MyClass(WatsonxAiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
public String generate(String userInput) {
|
||||
@@ -119,7 +119,7 @@ public class MyClass {
|
||||
|
||||
Prompt prompt = new Prompt(new SystemMessage(userInput), options);
|
||||
|
||||
var results = chat.call(prompt);
|
||||
var results = chatModel.call(prompt);
|
||||
|
||||
var generatedText = results.getResult().getOutput().getContent();
|
||||
|
||||
@@ -135,7 +135,7 @@ public class MyClass {
|
||||
|
||||
Prompt prompt = new Prompt(new SystemMessage(userInput), options);
|
||||
|
||||
var results = chat.stream(prompt).collectList().block(); // wait till the stream is resolved (completed)
|
||||
var results = chatModel.stream(prompt).collectList().block(); // wait till the stream is resolved (completed)
|
||||
|
||||
var generatedText = results.stream()
|
||||
.map(generation -> generation.getResult().getOutput().getContent())
|
||||
|
||||
@@ -52,7 +52,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the ZhiPu AI Chat client.
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the ZhiPu AI chat model.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
@@ -81,13 +81,13 @@ The prefix `spring.ai.zhiPu` is used as the property prefix that lets you connec
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.zhipuai.chat` is the property prefix that lets you configure the chat client implementation for ZhiPuAI.
|
||||
The prefix `spring.ai.zhipuai.chat` is the property prefix that lets you configure the chat model implementation for ZhiPuAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.zhipuai.chat.enabled | Enable ZhiPuAI chat client. | true
|
||||
| spring.ai.zhipuai.chat.enabled | Enable ZhiPuAI chat model. | true
|
||||
| spring.ai.zhipuai.chat.base-url | Optional overrides the spring.ai.zhipuai.base-url to provide chat specific url | https://open.bigmodel.cn/api/paas
|
||||
| spring.ai.zhipuai.chat.api-key | Optional overrides the spring.ai.zhipuai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.zhipuai.chat.options.model | This is the ZhiPuAI Chat model to use | `GLM-3-Turbo` (the `GLM-3-Turbo`, `GLM-4`, and `GLM-4V` point to the latest model versions)
|
||||
@@ -101,7 +101,7 @@ The prefix `spring.ai.zhipuai.chat` is the property prefix that lets you configu
|
||||
| spring.ai.zhipuai.chat.options.user | A unique identifier representing your end-user, which can help ZhiPuAI to monitor and detect abuse. | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.zhipuai.base-url` and `spring.ai.zhipuai.api-key` for the `ChatClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.zhipuai.base-url` and `spring.ai.zhipuai.api-key` for the `ChatModel` implementations.
|
||||
The `spring.ai.zhipuai.chat.base-url` and `spring.ai.zhipuai.chat.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different ZhiPuAI accounts for different models and different model endpoints.
|
||||
|
||||
@@ -111,14 +111,14 @@ TIP: All properties prefixed with `spring.ai.zhipuai.chat.options` can be overri
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-zhipuai/src/main/java/org/springframework/ai/zhipuai/ZhiPuAiChatOptions.java[ZhiPuAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `ZhiPuAiChatClient(api, options)` constructor or the `spring.ai.zhipuai.chat.options.*` properties.
|
||||
On start-up, the default options can be configured with the `ZhiPuAiChatModel(api, options)` constructor or the `spring.ai.zhipuai.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
ZhiPuAiChatOptions.builder()
|
||||
@@ -134,7 +134,7 @@ TIP: In addition to the model specific link:https://github.com/spring-projects/s
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-zhipuai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the ZhiPuAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the ZhiPuAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -145,37 +145,37 @@ spring.ai.zhipuai.chat.options.temperature=0.7
|
||||
|
||||
TIP: replace the `api-key` with your ZhiPuAI credentials.
|
||||
|
||||
This will create a `ZhiPuAiChatClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `ZhiPuAiChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final ZhiPuAiChatClient chatClient;
|
||||
private final ZhiPuAiChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(ZhiPuAiChatClient chatClient) {
|
||||
this.chatClient = chatClient;
|
||||
public ChatController(ZhiPuAiChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", chatClient.call(message));
|
||||
return Map.of("generation", chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
var prompt = new Prompt(new UserMessage(message));
|
||||
return chatClient.stream(prompt);
|
||||
return chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-zhipuai/src/main/java/org/springframework/ai/zhipuai/ZhiPuAiChatClient.java[ZhiPuAiChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the ZhiPuAI service.
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-zhipuai/src/main/java/org/springframework/ai/zhipuai/ZhiPuAiChatModel.java[ZhiPuAiChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the ZhiPuAI service.
|
||||
|
||||
Add the `spring-ai-zhipuai` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -198,23 +198,23 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `ZhiPuAiChatClient` and use it for text generations:
|
||||
Next, create a `ZhiPuAiChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var zhiPuAiApi = new ZhiPuAiApi(System.getenv("ZHIPU_AI_API_KEY"));
|
||||
|
||||
var chatClient = new ZhiPuAiChatClient(zhiPuAiApi, ZhiPuAiChatOptions.builder()
|
||||
var chatModel = new ZhiPuAiChatModel(zhiPuAiApi, ZhiPuAiChatOptions.builder()
|
||||
.withModel(ZhiPuAiApi.ChatModel.GLM_3_Turbo.getValue())
|
||||
.withTemperature(0.4f)
|
||||
.withMaxTokens(200)
|
||||
.build());
|
||||
|
||||
ChatResponse response = chatClient.call(
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> streamResponse = chatClient.stream(
|
||||
Flux<ChatResponse> streamResponse = chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,27 +1,27 @@
|
||||
[[ChatClient]]
|
||||
= Chat Completion API
|
||||
[[ChatModel]]
|
||||
= Chat Model API
|
||||
|
||||
The Chat Completion API offers developers the ability to integrate AI-powered chat completion capabilities into their applications. It leverages pre-trained language models, such as GPT (Generative Pre-trained Transformer), to generate human-like responses to user inputs in natural language.
|
||||
The Chat Model API offers developers the ability to integrate AI-powered chat completion capabilities into their applications. It leverages pre-trained language models, such as GPT (Generative Pre-trained Transformer), to generate human-like responses to user inputs in natural language.
|
||||
|
||||
The API typically works by sending a prompt or partial conversation to the AI model, which then generates a completion or continuation of the conversation based on its training data and understanding of natural language patterns. The completed response is then returned to the application, which can present it to the user or use it for further processing.
|
||||
|
||||
The `Spring AI Chat Completion API` is designed to be a simple and portable interface for interacting with various xref:concepts.adoc#_models[AI Models], allowing developers to switch between different models with minimal code changes.
|
||||
The `Spring AI Chat Model API` is designed to be a simple and portable interface for interacting with various xref:concepts.adoc#_models[AI Models], allowing developers to switch between different models with minimal code changes.
|
||||
This design aligns with Spring's philosophy of modularity and interchangeability.
|
||||
|
||||
Also with the help of companion classes like `Prompt` for input encapsulation and `ChatResponse` for output handling, the Chat Completion API unifies the communication with AI Models.
|
||||
Also with the help of companion classes like `Prompt` for input encapsulation and `ChatResponse` for output handling, the Chat Model API unifies the communication with AI Models.
|
||||
It manages the complexity of request preparation and response parsing, offering a direct and simplified API interaction.
|
||||
|
||||
== API Overview
|
||||
|
||||
This section provides a guide to the Spring AI Chat Completion API interface and associated classes.
|
||||
This section provides a guide to the Spring AI Chat Model API interface and associated classes.
|
||||
|
||||
=== ChatClient
|
||||
=== ChatModel
|
||||
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatClient.java[ChatClient] interface definition:
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatModel.java[ChatModel] interface definition:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public interface ChatClient extends ModelClient<Prompt, ChatResponse> {
|
||||
public interface ChatModel extends Model<Prompt, ChatResponse> {
|
||||
|
||||
default String call(String message) {// implementation omitted
|
||||
}
|
||||
@@ -35,19 +35,19 @@ public interface ChatClient extends ModelClient<Prompt, ChatResponse> {
|
||||
The `call` method with a `String` parameter simplifies initial use, avoiding the complexities of the more sophisticated `Prompt` and `ChatResponse` classes.
|
||||
In real-world applications, it is more common to use the `call` method that takes a `Prompt` instance and returns an `ChatResponse`.
|
||||
|
||||
=== StreamingChatClient
|
||||
=== StreamingChatModel
|
||||
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/StreamingChatClient.java[StreamingChatClient] interface definition:
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/StreamingChatModel.java[StreamingChatModel] interface definition:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public interface StreamingChatClient extends StreamingModelClient<Prompt, ChatResponse> {
|
||||
public interface StreamingChatModel extends StreamingModel<Prompt, ChatResponse> {
|
||||
@Override
|
||||
Flux<ChatResponse> stream(Prompt prompt);
|
||||
}
|
||||
----
|
||||
|
||||
The `stream` method takes a `Prompt` request similar to `ChatClient` but it streams the responses using the reactive Flux API.
|
||||
The `stream` method takes a `Prompt` request similar to `ChatModel` but it streams the responses using the reactive Flux API.
|
||||
|
||||
=== Prompt
|
||||
|
||||
@@ -130,7 +130,7 @@ public interface ChatOptions extends ModelOptions {
|
||||
}
|
||||
----
|
||||
|
||||
Additionally, every model specific ChatClient/StreamingChatClient implementation can have its own options that can be passed to the AI model. For example, the OpenAI Chat Completion model has its own options like `presencePenalty`, `frequencyPenalty`, `bestOf` etc.
|
||||
Additionally, every model specific ChatModel/StreamingChatModel implementation can have its own options that can be passed to the AI model. For example, the OpenAI Chat Completion model has its own options like `presencePenalty`, `frequencyPenalty`, `bestOf` etc.
|
||||
|
||||
This is a powerful feature that allows developers to use model specific options when starting the application and then override them with at runtime using the Prompt request:
|
||||
|
||||
@@ -184,7 +184,7 @@ public class Generation implements ModelResult<AssistantMessage> {
|
||||
|
||||
== Available Implementations
|
||||
|
||||
The `ChatClient` and `StreamingChatClient` implementations are provided for the following Model providers:
|
||||
The `ChatModel` and `StreamingChatModel` implementations are provided for the following Model providers:
|
||||
|
||||
image::spring-ai-chat-completions-clients.jpg[align="center", width="800px"]
|
||||
|
||||
@@ -205,7 +205,7 @@ image::spring-ai-chat-completions-clients.jpg[align="center", width="800px"]
|
||||
|
||||
== Chat Model API
|
||||
|
||||
The Spring AI Chat Completion API is build on top of the Spring AI `Generic Model API` providing Chat specific abstractions and implementations. Following class diagram illustrates the main classes and interfaces of the Spring AI Chat Completion API.
|
||||
The Spring AI Chat Model API is build on top of the Spring AI `Generic Model API` providing Chat specific abstractions and implementations. Following class diagram illustrates the main classes and interfaces of the Spring AI Chat Model API.
|
||||
|
||||
image::spring-ai-chat-api.jpg[align="center", width="900px"]
|
||||
|
||||
|
||||
@@ -1,23 +1,23 @@
|
||||
[[EmbeddingClient]]
|
||||
= Embeddings API
|
||||
[[EmbeddingModel]]
|
||||
= Embeddings Model API
|
||||
|
||||
The `EmbeddingClient` interface is designed for straightforward integration with embedding models in AI and machine learning.
|
||||
The `EmbeddingModel` interface is designed for straightforward integration with embedding models in AI and machine learning.
|
||||
Its primary function is to convert text into numerical vectors, commonly referred to as embeddings.
|
||||
These embeddings are crucial for various tasks such as semantic analysis and text classification.
|
||||
|
||||
The design of the EmbeddingClient interface centers around two primary goals:
|
||||
The design of the EmbeddingModel interface centers around two primary goals:
|
||||
|
||||
* *Portability*: This interface ensures easy adaptability across various embedding models.
|
||||
It allows developers to switch between different embedding techniques or models with minimal code changes.
|
||||
This design aligns with Spring's philosophy of modularity and interchangeability.
|
||||
|
||||
* *Simplicity*: EmbeddingClient simplifies the process of converting text to embeddings.
|
||||
* *Simplicity*: EmbeddingModel simplifies the process of converting text to embeddings.
|
||||
By providing straightforward methods like `embed(String text)` and `embed(Document document)`, it takes the complexity out of dealing with raw text data and embedding algorithms. This design choice makes it easier for developers, especially those new to AI, to utilize embeddings in their applications without delving deep into the underlying mechanics.
|
||||
|
||||
== API Overview
|
||||
|
||||
The Embedding API is built on top of the generic https://github.com/spring-projects/spring-ai/tree/main/spring-ai-core/src/main/java/org/springframework/ai/model[Spring AI Model API], which is a part of the Spring AI library.
|
||||
As such, the EmbeddingClient interface extends the `ModelClient` interface, which provides a standard set of methods for interacting with AI models. The `EmbeddingRequest` and `EmbeddingResponse` classes extend from the `ModelRequest` and `ModelResponse` are used to encapsulate the input and output of the embedding models, respectively.
|
||||
The Embedding Model API is built on top of the generic https://github.com/spring-projects/spring-ai/tree/main/spring-ai-core/src/main/java/org/springframework/ai/model[Spring AI Model API], which is a part of the Spring AI library.
|
||||
As such, the EmbeddingModel interface extends the `Model` interface, which provides a standard set of methods for interacting with AI models. The `EmbeddingRequest` and `EmbeddingResponse` classes extend from the `ModelRequest` and `ModelResponse` are used to encapsulate the input and output of the embedding models, respectively.
|
||||
|
||||
The Embedding API in turn is used by higher-level components to implement Embedding Clients for specific embedding models, such as OpenAI, Titan, Azure OpenAI, Ollie, and others.
|
||||
|
||||
@@ -25,13 +25,13 @@ Following diagram illustrates the Embedding API and its relationship with the Sp
|
||||
|
||||
image:embeddings-api.jpg[title=Embeddings API,align=center,width=900]
|
||||
|
||||
=== EmbeddingClient
|
||||
=== EmbeddingModel
|
||||
|
||||
This section provides a guide to the `EmbeddingClient` interface and associated classes.
|
||||
This section provides a guide to the `EmbeddingModel` interface and associated classes.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public interface EmbeddingClient extends ModelClient<EmbeddingRequest, EmbeddingResponse> {
|
||||
public interface EmbeddingModel extends Model<EmbeddingRequest, EmbeddingResponse> {
|
||||
|
||||
@Override
|
||||
EmbeddingResponse call(EmbeddingRequest request);
|
||||
@@ -148,7 +148,7 @@ public class Embedding implements ModelResult<List<Double>> {
|
||||
|
||||
== Available Implementations [[available-implementations]]
|
||||
|
||||
Internally the various `EmbeddingClient` implementations use different low-level libraries and APIs to perform the embedding tasks. The following are some of the available implementations of the `EmbeddingClient` implementations:
|
||||
Internally the various `EmbeddingModel` implementations use different low-level libraries and APIs to perform the embedding tasks. The following are some of the available implementations of the `EmbeddingModel` implementations:
|
||||
|
||||
* xref:api/embeddings/openai-embeddings.adoc[Spring AI OpenAI Embeddings]
|
||||
* xref:api/embeddings/azure-openai-embeddings.adoc[Spring AI Azure OpenAI Embeddings]
|
||||
|
||||
@@ -66,13 +66,13 @@ The prefix `spring.ai.azure.openai` is the property prefix to configure the conn
|
||||
|====
|
||||
|
||||
|
||||
The prefix `spring.ai.azure.openai.embedding` is the property prefix that configures the `EmbeddingClient` implementation for Azure OpenAI
|
||||
The prefix `spring.ai.azure.openai.embedding` is the property prefix that configures the `EmbeddingModel` implementation for Azure OpenAI
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.azure.openai.embedding.enabled | Enable Azure OpenAI embedding client. | true
|
||||
| spring.ai.azure.openai.embedding.enabled | Enable Azure OpenAI embedding model. | true
|
||||
| spring.ai.azure.openai.embedding.metadata-mode | Document content extraction mode | EMBED
|
||||
| spring.ai.azure.openai.embedding.options.deployment-name | This is the value of the 'Deployment Name' as presented in the Azure AI Portal | text-embedding-ada-002
|
||||
| spring.ai.azure.openai.embedding.options.user | An identifier for the caller or end user of the operation. This may be used for tracking or rate-limiting purposes. | -
|
||||
@@ -85,14 +85,14 @@ TIP: All properties prefixed with `spring.ai.azure.openai.embedding.options` can
|
||||
The `AzureOpenAiEmbeddingOptions` provides the configuration information for the embedding requests.
|
||||
The `AzureOpenAiEmbeddingOptions` offers a builder to create the options.
|
||||
|
||||
At start time use the `AzureOpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At start time use the `AzureOpenAiEmbeddingModel` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, by passing a `AzureOpenAiEmbeddingOptions` instance with your to the `EmbeddingRequest` request.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
AzureOpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
@@ -102,8 +102,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Code
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -117,16 +117,16 @@ spring.ai.azure.openai.embedding.options.model=text-embedding-ada-002
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -134,7 +134,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
If you prefer not to use the Spring Boot auto-configuration, you can manually configure the `AzureOpenAiEmbeddingClient` in your application.
|
||||
If you prefer not to use the Spring Boot auto-configuration, you can manually configure the `AzureOpenAiEmbeddingModel` in your application.
|
||||
For this add the `spring-ai-azure-openai` dependency to your project's Maven `pom.xml` file:
|
||||
[source, xml]
|
||||
----
|
||||
@@ -155,9 +155,9 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-azure-openai` dependency also provide the access to the `AzureOpenAiEmbeddingClient`. For more information about the `AzureOpenAiChatClient` refer to the link:../embeddings/azure-openai-embeddings.html[Azure OpenAI Embeddings] section.
|
||||
NOTE: The `spring-ai-azure-openai` dependency also provide the access to the `AzureOpenAiEmbeddingModel`. For more information about the `AzureOpenAiChatModel` refer to the link:../embeddings/azure-openai-embeddings.html[Azure OpenAI Embeddings] section.
|
||||
|
||||
Next, create an `AzureOpenAiEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `AzureOpenAiEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -166,13 +166,13 @@ var openAIClient = OpenAIClientBuilder()
|
||||
.endpoint(System.getenv("AZURE_OPENAI_ENDPOINT"))
|
||||
.buildClient();
|
||||
|
||||
var embeddingClient = new AzureOpenAiEmbeddingClient(openAIClient)
|
||||
var embeddingModel = new AzureOpenAiEmbeddingModel(openAIClient)
|
||||
.withDefaultOptions(AzureOpenAiEmbeddingOptions.builder()
|
||||
.withModel("text-embedding-ada-002")
|
||||
.withUser("user-6")
|
||||
.build());
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -62,7 +62,7 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.cohere.embedding` (defined in `BedrockCohereEmbeddingProperties`) is the property prefix that configures the embedding client implementation for Cohere.
|
||||
The prefix `spring.ai.bedrock.cohere.embedding` (defined in `BedrockCohereEmbeddingProperties`) is the property prefix that configures the embedding model implementation for Cohere.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
@@ -83,14 +83,14 @@ TIP: All properties prefixed with `spring.ai.bedrock.cohere.embedding.options` c
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingOptions.java[BedrockCohereEmbeddingOptions.java] provides model configurations, such as `input-type` or `truncate`.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockCohereEmbeddingClient(api, options)` constructor or the `spring.ai.bedrock.cohere.embedding.options.*` properties.
|
||||
On start-up, the default options can be configured with the `BedrockCohereEmbeddingModel(api, options)` constructor or the `spring.ai.bedrock.cohere.embedding.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `EmbeddingRequest` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
BedrockCohereEmbeddingOptions.builder()
|
||||
.withInputType(InputType.SEARCH_DOCUMENT)
|
||||
@@ -115,24 +115,24 @@ spring.ai.bedrock.cohere.embedding.options.input-type=search-document
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockCohereEmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
This will create a `BedrockCohereEmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -140,7 +140,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingClient.java[BedrockCohereEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingModel.java[BedrockCohereEmbeddingModel] implements the `EmbeddingModel` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -163,7 +163,7 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingClient.java[BedrockCohereEmbeddingClient] and use it for text embeddings:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingModel.java[BedrockCohereEmbeddingModel] and use it for text embeddings:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@@ -172,9 +172,9 @@ var cohereEmbeddingApi =new CohereEmbeddingBedrockApi(
|
||||
EnvironmentVariableCredentialsProvider.create(), Region.US_EAST_1.id(), new ObjectMapper());
|
||||
|
||||
|
||||
var embeddingClient = new BedrockCohereEmbeddingClient(cohereEmbeddingApi);
|
||||
var embeddingModel = new BedrockCohereEmbeddingModel(cohereEmbeddingApi);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ The prefix `spring.ai.bedrock.aws` is the property prefix to configure the conne
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.titan.embedding` (defined in `BedrockTitanEmbeddingProperties`) is the property prefix that configures the embedding client implementation for Titan.
|
||||
The prefix `spring.ai.bedrock.titan.embedding` (defined in `BedrockTitanEmbeddingProperties`) is the property prefix that configures the embedding model implementation for Titan.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
@@ -84,14 +84,14 @@ Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/lat
|
||||
== Runtime Options [[embedding-options]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingOptions.java[BedrockTitanEmbeddingOptions.java] provides model configurations, such as `input-type`.
|
||||
On start-up, the default options can be configured with the `BedrockTitanEmbeddingClient(api).withInputType(type)` method or the `spring.ai.bedrock.titan.embedding.input-type` properties.
|
||||
On start-up, the default options can be configured with the `BedrockTitanEmbeddingModel(api).withInputType(type)` method or the `spring.ai.bedrock.titan.embedding.input-type` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `EmbeddingRequest` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
BedrockTitanEmbeddingOptions.builder()
|
||||
.withInputType(InputType.TEXT)
|
||||
@@ -116,23 +116,23 @@ spring.ai.bedrock.titan.embedding.enabled=true
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `EmbeddingController` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -140,7 +140,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the Bedrock Titan service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingModel.java[BedrockTitanEmbeddingModel] implements the `EmbeddingModel` and uses the <<low-level-api>> to connect to the Bedrock Titan service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -163,16 +163,16 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] and use it for text embeddings:
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingModel.java[BedrockTitanEmbeddingModel] and use it for text embeddings:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var titanEmbeddingApi = new TitanEmbeddingBedrockApi(
|
||||
TitanEmbeddingModel.TITAN_EMBED_IMAGE_V1.id(), Region.US_EAST_1.id());
|
||||
|
||||
var embeddingClient = new BedrockTitanEmbeddingClient(titanEmbeddingApi);
|
||||
var embeddingModel = new BedrockTitanEmbeddingModel(titanEmbeddingApi);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World")); // NOTE titan does not support batch embedding.
|
||||
----
|
||||
|
||||
|
||||
@@ -81,19 +81,19 @@ The prefix `spring.ai.minimax` is used as the property prefix that lets you conn
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.minimax.embedding` is property prefix that configures the `EmbeddingClient` implementation for MiniMax.
|
||||
The prefix `spring.ai.minimax.embedding` is property prefix that configures the `EmbeddingModel` implementation for MiniMax.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.minimax.embedding.enabled | Enable MiniMax embedding client. | true
|
||||
| spring.ai.minimax.embedding.enabled | Enable MiniMax embedding model. | true
|
||||
| spring.ai.minimax.embedding.base-url | Optional overrides the spring.ai.minimax.base-url to provide embedding specific url | -
|
||||
| spring.ai.minimax.embedding.api-key | Optional overrides the spring.ai.minimax.api-key to provide embedding specific api-key | -
|
||||
| spring.ai.minimax.embedding.options.model | The model to use | embo-01
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.minimax.base-url` and `spring.ai.minimax.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.minimax.base-url` and `spring.ai.minimax.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.minimax.embedding.base-url` and `spring.ai.minimax.embedding.api-key` properties if set take precedence over the common properties.
|
||||
Similarly, the `spring.ai.minimax.embedding.base-url` and `spring.ai.minimax.embedding.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different MiniMax accounts for different models and different model endpoints.
|
||||
@@ -106,14 +106,14 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mini
|
||||
|
||||
The default options can be configured using the `spring.ai.minimax.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `MiniMaxEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At start-time use the `MiniMaxEmbeddingModel` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `MiniMaxEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
MiniMaxEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
@@ -122,8 +122,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingC` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -136,16 +136,16 @@ spring.ai.minimax.embedding.options.model=embo-01
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -174,21 +174,21 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-minimax` dependency provides access also to the `MiniMaxChatClient`.
|
||||
For more information about the `MiniMaxChatClient` refer to the link:../chat/minimax-chat.html[MiniMax Chat Client] section.
|
||||
NOTE: The `spring-ai-minimax` dependency provides access also to the `MiniMaxChatModel`.
|
||||
For more information about the `MiniMaxChatModel refer to the link:../chat/minimax-chat.html[MiniMax Chat Client] section.
|
||||
|
||||
Next, create an `MiniMaxEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `MiniMaxEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var miniMaxApi = new MiniMaxApi(System.getenv("MINIMAX_API_KEY"));
|
||||
|
||||
var embeddingClient = new MiniMaxEmbeddingClient(miniMaxApi)
|
||||
var embeddingModel = new MiniMaxEmbeddingModel(miniMaxApi)
|
||||
.withDefaultOptions(MiniMaxChatOptions.build()
|
||||
.withModel("embo-01")
|
||||
.build());
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -80,13 +80,13 @@ The prefix `spring.ai.mistralai` is used as the property prefix that lets you co
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.mistralai.embedding` is property prefix that configures the `EmbeddingClient` implementation for MistralAI.
|
||||
The prefix `spring.ai.mistralai.embedding` is property prefix that configures the `EmbeddingModel` implementation for MistralAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.mistralai.embedding.enabled | Enable OpenAI embedding client. | true
|
||||
| spring.ai.mistralai.embedding.enabled | Enable OpenAI embedding model. | true
|
||||
| spring.ai.mistralai.embedding.base-url | Optional overrides the spring.ai.mistralai.base-url to provide embedding specific url | -
|
||||
| spring.ai.mistralai.embedding.api-key | Optional overrides the spring.ai.mistralai.api-key to provide embedding specific api-key | -
|
||||
| spring.ai.mistralai.embedding.metadata-mode | Document content extraction mode. | EMBED
|
||||
@@ -94,7 +94,7 @@ The prefix `spring.ai.mistralai.embedding` is property prefix that configures th
|
||||
| spring.ai.mistralai.embedding.options.encodingFormat | The format to return the embeddings in. Can be either float or base64. | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.mistralai.base-url` and `spring.ai.mistralai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.mistralai.base-url` and `spring.ai.mistralai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.mistralai.embedding.base-url` and `spring.ai.mistralai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
Similarly, the `spring.ai.mistralai.embedding.base-url` and `spring.ai.mistralai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different MistralAI accounts for different models and different model endpoints.
|
||||
@@ -107,14 +107,14 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mist
|
||||
|
||||
The default options can be configured using the `spring.ai.mistralai.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `MistralAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At start-time use the `MistralAiEmbeddingModel` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `MistralAiEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
MistralAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
@@ -123,8 +123,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -137,16 +137,16 @@ spring.ai.mistralai.embedding.options.model=mistral-embed
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
var embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
var embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -175,22 +175,22 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-mistral-ai` dependency provides access also to the `MistralAiChatClient`.
|
||||
For more information about the `MistralAiChatClient` refer to the link:../chat/mistralai-chat.html[MistralAI Chat Client] section.
|
||||
NOTE: The `spring-ai-mistral-ai` dependency provides access also to the `MistralAiChatModel`.
|
||||
For more information about the `MistralAiChatModel` refer to the link:../chat/mistralai-chat.html[MistralAI Chat Client] section.
|
||||
|
||||
Next, create an `MistralAiEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `MistralAiEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var mistralAiApi = new MistralAiApi(System.getenv("MISTRAL_AI_API_KEY"));
|
||||
|
||||
var embeddingClient = new MistralAiEmbeddingClient(mistralAiApi,
|
||||
var embeddingModel = new MistralAiEmbeddingModel(mistralAiApi,
|
||||
MistralAiEmbeddingOptions.builder()
|
||||
.withModel("mistral-embed")
|
||||
.withEncodingFormat("float")
|
||||
.build());
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
= Ollama Embeddings
|
||||
|
||||
With https://ollama.ai/[Ollama] you can run various Large Language Models (LLMs) locally and generate embeddings from them.
|
||||
Spring AI supports the Ollama text embeddings with `OllamaEmbeddingClient`.
|
||||
Spring AI supports the Ollama text embeddings with `OllamaEmbeddingModel`.
|
||||
|
||||
An embedding is a vector (list) of floating point numbers.
|
||||
The distance between two vectors measures their relatedness.
|
||||
@@ -47,7 +47,7 @@ dependencies {
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
The `spring.ai.ollama.embedding.options.*` properties are used to configure the default options used for all embedding requests.
|
||||
(It is used as `OllamaEmbeddingClient#withDefaultOptions()` instance).
|
||||
(It is used as `OllamaEmbeddingModel#withDefaultOptions()` instance).
|
||||
|
||||
=== Embedding Properties
|
||||
|
||||
@@ -60,13 +60,13 @@ The prefix `spring.ai.ollama` is the property prefix to configure the connection
|
||||
| spring.ai.ollama.base-url | Base URL where Ollama API server is running. | `http://localhost:11434`
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.ollama.embedding.options` is the property prefix that configures the `EmbeddingClient` implementation for Ollama.
|
||||
The prefix `spring.ai.ollama.embedding.options` is the property prefix that configures the `EmbeddingModel` implementation for Ollama.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.ollama.embedding.enabled | Enable Ollama embedding client. | true
|
||||
| spring.ai.ollama.embedding.enabled | Enable Ollama embedding model. | true
|
||||
| spring.ai.ollama.embedding.options.model | The name of the https://github.com/ollama/ollama?tab=readme-ov-file#model-library[supported model] to use. | mistral
|
||||
|====
|
||||
|
||||
@@ -115,14 +115,14 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-olla
|
||||
|
||||
The default options can be configured using the `spring.ai.ollama.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OllamaEmbeddingClient#withDefaultOptions()` to configure the default options used for all embedding requests.
|
||||
At start-time use the `OllamaEmbeddingModel#withDefaultOptions()` to configure the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OllamaOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OllamaOptions.create()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name"));
|
||||
@@ -130,24 +130,24 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -155,7 +155,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
If you are not using Spring Boot, you can manually configure the `OllamaEmbeddingClient`.
|
||||
If you are not using Spring Boot, you can manually configure the `OllamaEmbeddingModel`.
|
||||
For this add the spring-ai-ollama dependency to your project’s Maven pom.xml file:
|
||||
|
||||
[source,xml]
|
||||
@@ -177,21 +177,21 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-ollama` dependency provides access also to the `OllamaChatClient`.
|
||||
For more information about the `OllamaChatClient` refer to the link:../chat/ollama-chat.html[Ollama Chat Client] section.
|
||||
NOTE: The `spring-ai-ollama` dependency provides access also to the `OllamaChatModel`.
|
||||
For more information about the `OllamaChatModel` refer to the link:../chat/ollama-chat.html[Ollama Chat Client] section.
|
||||
|
||||
Next, create an `OllamaEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `OllamaEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var ollamaApi = new OllamaApi();
|
||||
|
||||
var embeddingClient = new OllamaEmbeddingClient(ollamaApi)
|
||||
var embeddingModel = new OllamaEmbeddingModel(ollamaApi)
|
||||
.withDefaultOptions(OllamaOptions.create()
|
||||
.withModel(OllamaOptions.DEFAULT_MODEL)
|
||||
.toMap());
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
= Transformers (ONNX) Embeddings
|
||||
|
||||
The `TransformersEmbeddingClient` is an `EmbeddingClient` implementation that locally computes https://www.sbert.net/examples/applications/computing-embeddings/README.html#sentence-embeddings-with-transformers[sentence embeddings] using a selected https://www.sbert.net/[sentence transformer].
|
||||
The `TransformersEmbeddingModel` is an `EmbeddingModel` implementation that locally computes https://www.sbert.net/examples/applications/computing-embeddings/README.html#sentence-embeddings-with-transformers[sentence embeddings] using a selected https://www.sbert.net/[sentence transformer].
|
||||
|
||||
It uses https://www.sbert.net/docs/pretrained_models.html[pre-trained] transformer models, serialized into the https://onnx.ai/[Open Neural Network Exchange (ONNX)] format.
|
||||
|
||||
@@ -25,7 +25,7 @@ source ./venv/bin/activate
|
||||
(venv) optimum-cli export onnx --generative sentence-transformers/all-MiniLM-L6-v2 onnx-output-folder
|
||||
----
|
||||
|
||||
The snippet exports the https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2[sentence-transformers/all-MiniLM-L6-v2] transformer into the `onnx-output-folder` folder. Later includes the `tokenizer.json` and `model.onnx` files used by the embedding client.
|
||||
The snippet exports the https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2[sentence-transformers/all-MiniLM-L6-v2] transformer into the `onnx-output-folder` folder. Later includes the `tokenizer.json` and `model.onnx` files used by the embedding model.
|
||||
|
||||
In place of the all-MiniLM-L6-v2 you can pick any huggingface transformer identifier or provide direct file path.
|
||||
|
||||
@@ -43,9 +43,9 @@ Add the `spring-ai-transformers` project to your maven dependencies:
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
then create a new `TransformersEmbeddingClient` instance and use the `setTokenizerResource(tokenizerJsonUri)` and `setModelResource(modelOnnxUri)` methods to set the URIs of the exported `tokenizer.json` and `model.onnx` files. (`classpath:`, `file:` or `https:` URI schemas are supported).
|
||||
then create a new `TransformersEmbeddingModel` instance and use the `setTokenizerResource(tokenizerJsonUri)` and `setModelResource(modelOnnxUri)` methods to set the URIs of the exported `tokenizer.json` and `model.onnx` files. (`classpath:`, `file:` or `https:` URI schemas are supported).
|
||||
|
||||
If the model is not explicitly set, `TransformersEmbeddingClient` defaults to https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2[sentence-transformers/all-MiniLM-L6-v2]:
|
||||
If the model is not explicitly set, `TransformersEmbeddingModel` defaults to https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2[sentence-transformers/all-MiniLM-L6-v2]:
|
||||
|
||||
[cols="2*"]
|
||||
|===
|
||||
@@ -55,53 +55,53 @@ If the model is not explicitly set, `TransformersEmbeddingClient` defaults to ht
|
||||
| Size | 80MB
|
||||
|===
|
||||
|
||||
The following snippet illustrates how to use the `TransformersEmbeddingClient` manually:
|
||||
The following snippet illustrates how to use the `TransformersEmbeddingModel` manually:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
TransformersEmbeddingClient embeddingClient = new TransformersEmbeddingClient();
|
||||
TransformersEmbeddingModel embeddingModel = new TransformersEmbeddingModel();
|
||||
|
||||
// (optional) defaults to classpath:/onnx/all-MiniLM-L6-v2/tokenizer.json
|
||||
embeddingClient.setTokenizerResource("classpath:/onnx/all-MiniLM-L6-v2/tokenizer.json");
|
||||
embeddingModel.setTokenizerResource("classpath:/onnx/all-MiniLM-L6-v2/tokenizer.json");
|
||||
|
||||
// (optional) defaults to classpath:/onnx/all-MiniLM-L6-v2/model.onnx
|
||||
embeddingClient.setModelResource("classpath:/onnx/all-MiniLM-L6-v2/model.onnx");
|
||||
embeddingModel.setModelResource("classpath:/onnx/all-MiniLM-L6-v2/model.onnx");
|
||||
|
||||
// (optional) defaults to ${java.io.tmpdir}/spring-ai-onnx-model
|
||||
// Only the http/https resources are cached by default.
|
||||
embeddingClient.setResourceCacheDirectory("/tmp/onnx-zoo");
|
||||
embeddingModel.setResourceCacheDirectory("/tmp/onnx-zoo");
|
||||
|
||||
// (optional) Set the tokenizer padding if you see an errors like:
|
||||
// "ai.onnxruntime.OrtException: Supplied array is ragged, ..."
|
||||
embeddingClient.setTokenizerOptions(Map.of("padding", "true"));
|
||||
embeddingModel.setTokenizerOptions(Map.of("padding", "true"));
|
||||
|
||||
embeddingClient.afterPropertiesSet();
|
||||
embeddingModel.afterPropertiesSet();
|
||||
|
||||
List<List<Double>> embeddings = embeddingClient.embed(List.of("Hello world", "World is big"));
|
||||
List<List<Double>> embeddings = embeddingModel.embed(List.of("Hello world", "World is big"));
|
||||
|
||||
----
|
||||
|
||||
NOTE: If you create an instance of `TransformersEmbeddingClient` manually, you must call the `afterPropertiesSet()` method after setting the properties and before using the client.
|
||||
NOTE: If you create an instance of `TransformersEmbeddingModel` manually, you must call the `afterPropertiesSet()` method after setting the properties and before using the client.
|
||||
|
||||
The first `embed()` call downloads the large ONNX model and caches it on the local file system.
|
||||
Therefore, the first call might take longer than usual.
|
||||
Use the `#setResourceCacheDirectory(<path>)` method to set the local folder where the ONNX models as stored.
|
||||
The default cache folder is `${java.io.tmpdir}/spring-ai-onnx-model`.
|
||||
|
||||
It is more convenient (and preferred) to create the TransformersEmbeddingClient as a `Bean`.
|
||||
It is more convenient (and preferred) to create the TransformersEmbeddingModel as a `Bean`.
|
||||
Then you don't have to call the `afterPropertiesSet()` manually.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
return new TransformersEmbeddingClient();
|
||||
public EmbeddingModel embeddingModel() {
|
||||
return new TransformersEmbeddingModel();
|
||||
}
|
||||
----
|
||||
|
||||
== Transformers Embedding Spring Boot Starter
|
||||
|
||||
You can bootstrap and autowire the `TransformersEmbeddingClient` with the following Spring Boot starter:
|
||||
You can bootstrap and autowire the `TransformersEmbeddingModel` with the following Spring Boot starter:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
|
||||
@@ -81,13 +81,13 @@ The prefix `spring.ai.openai` is used as the property prefix that lets you conne
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.openai.embedding` is property prefix that configures the `EmbeddingClient` implementation for OpenAI.
|
||||
The prefix `spring.ai.openai.embedding` is property prefix that configures the `EmbeddingModel` implementation for OpenAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.openai.embedding.enabled | Enable OpenAI embedding client. | true
|
||||
| spring.ai.openai.embedding.enabled | Enable OpenAI embedding model. | true
|
||||
| spring.ai.openai.embedding.base-url | Optional overrides the spring.ai.openai.base-url to provide embedding specific url | -
|
||||
| spring.ai.openai.embedding.api-key | Optional overrides the spring.ai.openai.api-key to provide embedding specific api-key | -
|
||||
| spring.ai.openai.embedding.metadata-mode | Document content extraction mode. | EMBED
|
||||
@@ -97,7 +97,7 @@ The prefix `spring.ai.openai.embedding` is property prefix that configures the `
|
||||
| spring.ai.openai.embedding.options.dimensions | The number of dimensions the resulting output embeddings should have. Only supported in `text-embedding-3` and later models. | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.openai.base-url` and `spring.ai.openai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.openai.base-url` and `spring.ai.openai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.openai.embedding.base-url` and `spring.ai.openai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
Similarly, the `spring.ai.openai.embedding.base-url` and `spring.ai.openai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
|
||||
@@ -110,14 +110,14 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-open
|
||||
|
||||
The default options can be configured using the `spring.ai.openai.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At start-time use the `OpenAiEmbeddingModel` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OpenAiEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
@@ -126,8 +126,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -140,16 +140,16 @@ spring.ai.openai.embedding.options.model=text-embedding-ada-002
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -178,16 +178,16 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-openai` dependency provides access also to the `OpenAiChatClient`.
|
||||
For more information about the `OpenAiChatClient` refer to the link:../chat/openai-chat.html[OpenAI Chat Client] section.
|
||||
NOTE: The `spring-ai-openai` dependency provides access also to the `OpenAiChatModel`.
|
||||
For more information about the `OpenAiChatModel` refer to the link:../chat/openai-chat.html[OpenAI Chat Client] section.
|
||||
|
||||
Next, create an `OpenAiEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `OpenAiEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var openAiApi = new OpenAiApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var embeddingClient = new OpenAiEmbeddingClient(
|
||||
var embeddingModel = new OpenAiEmbeddingModel(
|
||||
openAiApi,
|
||||
MetadataMode.EMBED,
|
||||
OpenAiEmbeddingOptions.builder()
|
||||
@@ -196,7 +196,7 @@ var embeddingClient = new OpenAiEmbeddingClient(
|
||||
.build(),
|
||||
RetryUtils.DEFAULT_RETRY_TEMPLATE);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -40,16 +40,16 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Use the `spring.ai.postgresml.embedding.options.*` properties to configure your `PostgresMlEmbeddingClient`. links
|
||||
Use the `spring.ai.postgresml.embedding.options.*` properties to configure your `PostgresMlEmbeddingModel`. links
|
||||
|
||||
=== Embedding Properties
|
||||
|
||||
The prefix `spring.ai.postgresml.embedding` is property prefix that configures the `EmbeddingClient` implementation for PostgresML embeddings.
|
||||
The prefix `spring.ai.postgresml.embedding` is property prefix that configures the `EmbeddingModel` implementation for PostgresML embeddings.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.postgresml.embedding.enabled | Enable PostgresML embedding client. | true
|
||||
| spring.ai.postgresml.embedding.enabled | Enable PostgresML embedding model. | true
|
||||
| spring.ai.postgresml.embedding.options.transformer | The Huggingface transformer model to use for the embedding. | distilbert-base-uncased
|
||||
| spring.ai.postgresml.embedding.options.kwargs | Additional transformer specific options. | empty map
|
||||
| spring.ai.postgresml.embedding.options.vectorType | PostgresML vector type to use for the embedding. Two options are supported: `PG_ARRAY` and `PG_VECTOR`. | PG_ARRAY
|
||||
@@ -61,10 +61,10 @@ TIP: All properties prefixed with `spring.ai.postgresml.embedding.options` can b
|
||||
|
||||
== Runtime Options [[embedding-options]]
|
||||
|
||||
Use the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/postgresml/PostgresMlEmbeddingOptions.java[PostgresMlEmbeddingOptions.java] to configure the `PostgresMlEmbeddingClient` with options, such as the model to use and etc.
|
||||
Use the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/postgresml/PostgresMlEmbeddingOptions.java[PostgresMlEmbeddingOptions.java] to configure the `PostgresMlEmbeddingModel` with options, such as the model to use and etc.
|
||||
|
||||
|
||||
On start you can pass a `PostgresMlEmbeddingOptions` to the `PostgresMlEmbeddingClient` constructor to configure the default options used for all embedding requests.
|
||||
On start you can pass a `PostgresMlEmbeddingOptions` to the `PostgresMlEmbeddingModel` constructor to configure the default options used for all embedding requests.
|
||||
|
||||
At run-time you can override the default options, using a `PostgresMlEmbeddingOptions` in your `EmbeddingRequest`.
|
||||
|
||||
@@ -73,7 +73,7 @@ For example to override the default model name for a specific request:
|
||||
[source,java]
|
||||
----
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
PostgresMlEmbeddingOptions.builder()
|
||||
.withTransformer("intfloat/e5-small")
|
||||
@@ -84,8 +84,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -100,16 +100,16 @@ spring.ai.postgresml.embedding.options.kwargs.device=cpu
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -117,7 +117,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual configuration
|
||||
|
||||
Instead of using the Spring Boot auto-configuration, you can create the `PostgresMlEmbeddingClient` manually.
|
||||
Instead of using the Spring Boot auto-configuration, you can create the `PostgresMlEmbeddingModel` manually.
|
||||
For this add the `spring-ai-postgresml` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source, xml]
|
||||
@@ -139,13 +139,13 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an `PostgresMlEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `PostgresMlEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var jdbcTemplate = new JdbcTemplate(dataSource); // your posgresml data source
|
||||
|
||||
PostgresMlEmbeddingClient embeddingClient = new PostgresMlEmbeddingClient(this.jdbcTemplate,
|
||||
PostgresMlEmbeddingModel embeddingModel = new PostgresMlEmbeddingModel(this.jdbcTemplate,
|
||||
PostgresMlEmbeddingOptions.builder()
|
||||
.withTransformer("distilbert-base-uncased") // huggingface transformer model name.
|
||||
.withVectorType(VectorType.PG_VECTOR) //vector type in PostgreSQL.
|
||||
@@ -153,21 +153,21 @@ PostgresMlEmbeddingClient embeddingClient = new PostgresMlEmbeddingClient(this.j
|
||||
.withMetadataMode(MetadataMode.EMBED) // Document metadata mode.
|
||||
.build());
|
||||
|
||||
embeddingClient.afterPropertiesSet(); // initialize the jdbc template and database.
|
||||
embeddingModel.afterPropertiesSet(); // initialize the jdbc template and database.
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
NOTE: When created manually, you must call the `afterPropertiesSet()` after setting the properties and before using the client.
|
||||
It is more convenient (and preferred) to create the PostgresMlEmbeddingClient as a `@Bean`.
|
||||
It is more convenient (and preferred) to create the PostgresMlEmbeddingModel as a `@Bean`.
|
||||
Then you don’t have to call the `afterPropertiesSet()` manually:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient(JdbcTemplate jdbcTemplate) {
|
||||
return new PostgresMlEmbeddingClient(jdbcTemplate,
|
||||
public EmbeddingModel embeddingModel(JdbcTemplate jdbcTemplate) {
|
||||
return new PostgresMlEmbeddingModel(jdbcTemplate,
|
||||
PostgresMlEmbeddingOptions.builder()
|
||||
....
|
||||
.build());
|
||||
|
||||
@@ -76,7 +76,7 @@ The prefix `spring.ai.vertex.ai.embedding` is the property prefix that lets you
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-vertex-ai-palm2-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -86,7 +86,7 @@ spring.ai.vertex.ai.embedding.model=embedding-gecko-001
|
||||
|
||||
TIP: replace the `api-key` with your VertexAI credentials.
|
||||
|
||||
This will create a `VertexAiPaLm2EmbeddingClient` implementation that you can inject into your class.
|
||||
This will create a `VertexAiPaLm2EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the embedding client for text generations.
|
||||
|
||||
[source,java]
|
||||
@@ -94,16 +94,16 @@ Here is an example of a simple `@Controller` class that uses the embedding clien
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -111,7 +111,7 @@ public class EmbeddingController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-palm2/src/main/java/org/springframework/ai/vertexai/palm2/VertexAiPaLm2EmbeddingClient.java[VertexAiPaLm2EmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the VertexAI service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-palm2/src/main/java/org/springframework/ai/vertexai/palm2/VertexAiPaLm2EmbeddingModel.java[VertexAiPaLm2EmbeddingModel] implements the `EmbeddingModel` and uses the <<low-level-api>> to connect to the VertexAI service.
|
||||
|
||||
Add the `spring-ai-vertex-ai` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -134,15 +134,15 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `VertexAiPaLm2EmbeddingClient` and use it for text generations:
|
||||
Next, create a `VertexAiPaLm2EmbeddingModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
VertexAiPaLm2Api vertexAiApi = new VertexAiPaLm2Api(< YOUR PALM_API_KEY>);
|
||||
|
||||
var embeddingClient = new VertexAiPaLm2EmbeddingClient(vertexAiApi);
|
||||
var embeddingModel = new VertexAiPaLm2EmbeddingModel(vertexAiApi);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -81,19 +81,19 @@ The prefix `spring.ai.zhipuai` is used as the property prefix that lets you conn
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
The prefix `spring.ai.zhipuai.embedding` is property prefix that configures the `EmbeddingClient` implementation for ZhiPuAI.
|
||||
The prefix `spring.ai.zhipuai.embedding` is property prefix that configures the `EmbeddingModel` implementation for ZhiPuAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.zhipuai.embedding.enabled | Enable ZhiPuAI embedding client. | true
|
||||
| spring.ai.zhipuai.embedding.enabled | Enable ZhiPuAI embedding model. | true
|
||||
| spring.ai.zhipuai.embedding.base-url | Optional overrides the spring.ai.zhipuai.base-url to provide embedding specific url | -
|
||||
| spring.ai.zhipuai.embedding.api-key | Optional overrides the spring.ai.zhipuai.api-key to provide embedding specific api-key | -
|
||||
| spring.ai.zhipuai.embedding.options.model | The model to use | embedding-2
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.zhipuai.base-url` and `spring.ai.zhipuai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
|
||||
NOTE: You can override the common `spring.ai.zhipuai.base-url` and `spring.ai.zhipuai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.
|
||||
The `spring.ai.zhipuai.embedding.base-url` and `spring.ai.zhipuai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
Similarly, the `spring.ai.zhipuai.embedding.base-url` and `spring.ai.zhipuai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different ZhiPuAI accounts for different models and different model endpoints.
|
||||
@@ -106,14 +106,14 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-zhip
|
||||
|
||||
The default options can be configured using the `spring.ai.zhipuai.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `ZhiPuAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At start-time use the `ZhiPuAiEmbeddingModel` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `ZhiPuAiEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
EmbeddingResponse embeddingResponse = embeddingModel.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
ZhiPuAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
@@ -122,8 +122,8 @@ EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
|
||||
== Sample Controller
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
This will create a `EmbeddingModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
@@ -136,16 +136,16 @@ spring.ai.zhipuai.embedding.options.model=embedding-2
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
private final EmbeddingModel embeddingModel;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
public EmbeddingController(EmbeddingModel embeddingModel) {
|
||||
this.embeddingModel = embeddingModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
@@ -174,21 +174,21 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
NOTE: The `spring-ai-zhipuai` dependency provides access also to the `ZhiPuAiChatClient`.
|
||||
For more information about the `ZhiPuAiChatClient` refer to the link:../chat/zhipuai-chat.html[ZhiPuAI Chat Client] section.
|
||||
NOTE: The `spring-ai-zhipuai` dependency provides access also to the `ZhiPuAiChatModel`.
|
||||
For more information about the `ZhiPuAiChatModel` refer to the link:../chat/zhipuai-chat.html[ZhiPuAI Chat Client] section.
|
||||
|
||||
Next, create an `ZhiPuAiEmbeddingClient` instance and use it to compute the similarity between two input texts:
|
||||
Next, create an `ZhiPuAiEmbeddingModel` instance and use it to compute the similarity between two input texts:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var zhiPuAiApi = new ZhiPuAiApi(System.getenv("ZHIPU_AI_API_KEY"));
|
||||
|
||||
var embeddingClient = new ZhiPuAiEmbeddingClient(zhiPuAiApi)
|
||||
var embeddingModel = new ZhiPuAiEmbeddingModel(zhiPuAiApi)
|
||||
.withDefaultOptions(ZhiPuAiChatOptions.build()
|
||||
.withModel("embedding-2")
|
||||
.build());
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
[[generic-model-api]]
|
||||
= Generic Model API
|
||||
|
||||
In order to provide a foundation for all AI Model clients, the Generic Model API was created.
|
||||
In order to provide a foundation for all AI Models, the Generic Model API was created.
|
||||
This makes it easy to contribute new AI Model support to Spring AI by following a common pattern.
|
||||
The following sections walk through this API.
|
||||
|
||||
@@ -9,15 +9,15 @@ The following sections walk through this API.
|
||||
|
||||
image::spring-ai-generic-model-api.jpg[width=900, align="center"]
|
||||
|
||||
== ModelClient
|
||||
== Model
|
||||
|
||||
The ModelClient interface provides a generic API for invoking AI models. It is designed to handle the interaction with various types of AI models by abstracting the process of sending requests and receiving responses. The interface uses Java generics to accommodate different types of requests and responses, enhancing flexibility and adaptability across different AI model implementations.
|
||||
The Model interface provides a generic API for invoking AI models. It is designed to handle the interaction with various types of AI models by abstracting the process of sending requests and receiving responses. The interface uses Java generics to accommodate different types of requests and responses, enhancing flexibility and adaptability across different AI model implementations.
|
||||
|
||||
The interface is defined below:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public interface ModelClient<TReq extends ModelRequest<?>, TRes extends ModelResponse<?>> {
|
||||
public interface Model<TReq extends ModelRequest<?>, TRes extends ModelResponse<?>> {
|
||||
|
||||
/**
|
||||
* Executes a method call to the AI model.
|
||||
@@ -29,13 +29,13 @@ public interface ModelClient<TReq extends ModelRequest<?>, TRes extends ModelRes
|
||||
}
|
||||
----
|
||||
|
||||
== StreamingModelClient
|
||||
== StreamingModel
|
||||
|
||||
The StreamingModelClient interface provides a generic API for invoking an AI model with streaming response. It abstracts the process of sending requests and receiving a streaming response. The interface uses Java generics to accommodate different types of requests and responses, enhancing flexibility and adaptability across different AI model implementations.
|
||||
The StreamingModel interface provides a generic API for invoking an AI model with streaming response. It abstracts the process of sending requests and receiving a streaming response. The interface uses Java generics to accommodate different types of requests and responses, enhancing flexibility and adaptability across different AI model implementations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public interface StreamingModelClient<TReq extends ModelRequest<?>, TResChunk extends ModelResponse<?>> {
|
||||
public interface StreamingModel<TReq extends ModelRequest<?>, TResChunk extends ModelResponse<?>> {
|
||||
|
||||
/**
|
||||
* Executes a method call to the AI model.
|
||||
|
||||
@@ -42,12 +42,12 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
=== Image Generation Properties
|
||||
|
||||
|
||||
The prefix `spring.ai.openai.image` is the property prefix that lets you configure the `ImageClient` implementation for OpenAI.
|
||||
The prefix `spring.ai.openai.image` is the property prefix that lets you configure the `ImageModel` implementation for OpenAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.openai.image.enabled | Enable OpenAI image client. | true
|
||||
| spring.ai.openai.image.enabled | Enable OpenAI image model. | true
|
||||
| spring.ai.openai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
|
||||
| spring.ai.openai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.openai.image.options.n | The number of images to generate. Must be between 1 and 10. For dall-e-3, only n=1 is supported. | -
|
||||
@@ -97,14 +97,14 @@ The prefix `spring.ai.retry` is used as the property prefix that lets you config
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiImageOptions.java[OpenAiImageOptions.java] provides model configurations, such as the model to use, the quality, the size, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `OpenAiImageClient(OpenAiImageApi openAiImageApi)` constructor and the `withDefaultOptions(OpenAiImageOptions defaultOptions)` method. Alternatively, use the `spring.ai.openai.image.options.*` properties described previously.
|
||||
On start-up, the default options can be configured with the `OpenAiImageModel(OpenAiImageApi openAiImageApi)` constructor and the `withDefaultOptions(OpenAiImageOptions defaultOptions)` method. Alternatively, use the `spring.ai.openai.image.options.*` properties described previously.
|
||||
|
||||
At runtime you can override the default options by adding new, request specific, options to the `ImagePrompt` call.
|
||||
For example to override the OpenAI specific options such as quality and the number of images to create, use the following code example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ImageResponse response = openaiImageClient.call(
|
||||
ImageResponse response = openaiImageModel.call(
|
||||
new ImagePrompt("A light cream colored mini golden doodle",
|
||||
OpenAiImageOptions.builder()
|
||||
.withQuality("hd")
|
||||
|
||||
@@ -51,13 +51,13 @@ The prefix `spring.ai.stabilityai` is used as the property prefix that lets you
|
||||
| spring.ai.stabilityai.api-key | The API Key | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.stabilityai.image` is the property prefix that lets you configure the `ImageClient` implementation for Stability AI.
|
||||
The prefix `spring.ai.stabilityai.image` is the property prefix that lets you configure the `ImageModel` implementation for Stability AI.
|
||||
|
||||
[cols="2,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.stabilityai.image.enabled | Enable Stability AI image client. | true
|
||||
| spring.ai.stabilityai.image.enabled | Enable Stability AI image model. | true
|
||||
| spring.ai.stabilityai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide a specific url | `https://api.stability.ai/v1`
|
||||
| spring.ai.stabilityai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide a specific api-key | -
|
||||
| spring.ai.stabilityai.image.option.n | The number of images to be generated. Must be between 1 and 10. | 1
|
||||
@@ -78,14 +78,14 @@ The prefix `spring.ai.stabilityai.image` is the property prefix that lets you co
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-stabilityai/src/main/java/org/springframework/ai/stabilityai/api/StabilityAiImageOptions.java[StabilityAiImageOptions.java] provides model configurations, such as the model to use, the style, the size, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `StabilityAiImageClient(StabilityAiApi stabilityAiApi, StabilityAiImageOptions options)` constructor. Alternatively, use the `spring.ai.openai.image.options.*` properties described previously.
|
||||
On start-up, the default options can be configured with the `StabilityAiImageModel(StabilityAiApi stabilityAiApi, StabilityAiImageOptions options)` constructor. Alternatively, use the `spring.ai.openai.image.options.*` properties described previously.
|
||||
|
||||
At runtime, you can override the default options by adding new, request specific, options to the `ImagePrompt` call.
|
||||
For example to override the Stability AI specific options such as quality and the number of images to create, use the following code example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ImageResponse response = openaiImageClient.call(
|
||||
ImageResponse response = openaiImageModel.call(
|
||||
new ImagePrompt("A light cream colored mini golden doodle",
|
||||
StabilityAiImageOptions.builder()
|
||||
.withStylePreset("cinematic")
|
||||
|
||||
@@ -48,12 +48,12 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
=== Image Generation Properties
|
||||
|
||||
The prefix `spring.ai.zhipuai.image` is the property prefix that lets you configure the `ImageClient` implementation for ZhiPuAI.
|
||||
The prefix `spring.ai.zhipuai.image` is the property prefix that lets you configure the `ImageModel` implementation for ZhiPuAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.zhipuai.image.enabled | Enable ZhiPuAI image client. | true
|
||||
| spring.ai.zhipuai.image.enabled | Enable ZhiPuAI image model. | true
|
||||
| spring.ai.zhipuai.image.base-url | Optional overrides the spring.ai.zhipuai.base-url to provide chat specific url | -
|
||||
| spring.ai.zhipuai.image.api-key | Optional overrides the spring.ai.zhipuai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.zhipuai.image.options.model | The model to use for image generation. | cogview-3
|
||||
@@ -96,14 +96,14 @@ The prefix `spring.ai.retry` is used as the property prefix that lets you config
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-zhipuai/src/main/java/org/springframework/ai/zhipuai/ZhiPuAiImageOptions.java[ZhiPuAiImageOptions.java] provides model configurations, such as the model to use, the quality, the size, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `ZhiPuAiImageClient(ZhiPuAiImageApi zhiPuAiImageApi)` constructor and the `withDefaultOptions(ZhiPuAiImageOptions defaultOptions)` method. Alternatively, use the `spring.ai.zhipuai.image.options.*` properties described previously.
|
||||
On start-up, the default options can be configured with the `ZhiPuAiImageModel(ZhiPuAiImageApi zhiPuAiImageApi)` constructor and the `withDefaultOptions(ZhiPuAiImageOptions defaultOptions)` method. Alternatively, use the `spring.ai.zhipuai.image.options.*` properties described previously.
|
||||
|
||||
At runtime you can override the default options by adding new, request specific, options to the `ImagePrompt` call.
|
||||
For example to override the ZhiPuAI specific options such as quality and the number of images to create, use the following code example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ImageResponse response = zhiPuAiImageClient.call(
|
||||
ImageResponse response = zhiPuAiImageModel.call(
|
||||
new ImagePrompt("A light cream colored mini golden doodle",
|
||||
ZhiPuAiImageOptions.builder()
|
||||
.withQuality("hd")
|
||||
|
||||
@@ -1,27 +1,27 @@
|
||||
[[ImageClient]]
|
||||
= Image Generation API
|
||||
[[ImageModel]]
|
||||
= Image Model API
|
||||
|
||||
|
||||
The `Spring Image Generation API` is designed to be a simple and portable interface for interacting with various xref:concepts.adoc#_models[AI Models] specialized in image generation, allowing developers to switch between different image-related models with minimal code changes.
|
||||
The `Spring Image Model API` is designed to be a simple and portable interface for interacting with various xref:concepts.adoc#_models[AI Models] specialized in image generation, allowing developers to switch between different image-related models with minimal code changes.
|
||||
This design aligns with Spring's philosophy of modularity and interchangeability, ensuring developers can quickly adapt their applications to different AI capabilities related to image processing.
|
||||
|
||||
Additionally, with the support of companion classes like `ImagePrompt` for input encapsulation and `ImageResponse` for output handling, the Image Generation API unifies the communication with AI Models dedicated to image generation.
|
||||
Additionally, with the support of companion classes like `ImagePrompt` for input encapsulation and `ImageResponse` for output handling, the Image Model API unifies the communication with AI Models dedicated to image generation.
|
||||
It manages the complexity of request preparation and response parsing, offering a direct and simplified API interaction for image-generation functionalities.
|
||||
|
||||
The Spring Image Generation API is built on top of the Spring AI `Generic Model API`, providing image-specific abstractions and implementations.
|
||||
The Spring Image Model API is built on top of the Spring AI `Generic Model API`, providing image-specific abstractions and implementations.
|
||||
|
||||
== API Overview
|
||||
|
||||
This section provides a guide to the Spring Image Generation API interface and associated classes.
|
||||
This section provides a guide to the Spring Image Model API interface and associated classes.
|
||||
|
||||
== Image Client
|
||||
== Image Model
|
||||
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/image/ImageClient.java[ImageClient] interface definition:
|
||||
Here is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/image/ImageModel.java[ImageModel] interface definition:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@FunctionalInterface
|
||||
public interface ImageClient extends ModelClient<ImagePrompt, ImageResponse> {
|
||||
public interface ImageModel extends Model<ImagePrompt, ImageResponse> {
|
||||
|
||||
ImageResponse call(ImagePrompt request);
|
||||
|
||||
@@ -68,7 +68,7 @@ public class ImageMessage {
|
||||
public Float getWeight() {...}
|
||||
|
||||
// constructors and utility methods omitted
|
||||
|
||||
}
|
||||
----
|
||||
|
||||
==== ImageOptions
|
||||
@@ -94,7 +94,7 @@ public interface ImageOptions extends ModelOptions {
|
||||
}
|
||||
----
|
||||
|
||||
Additionally, every model specific ImageClient implementation can have its own options that can be passed to the AI model. For example, the OpenAI Image Generation model has its own options like `quality`, `style`, etc.
|
||||
Additionally, every model specific ImageModel implementation can have its own options that can be passed to the AI model. For example, the OpenAI Image Generation model has its own options like `quality`, `style`, etc.
|
||||
|
||||
|
||||
This is a powerful feature that allows developers to use model specific options when starting the application and then override them with at runtime using the `ImagePrompt`.
|
||||
@@ -157,7 +157,7 @@ public class ImageGeneration implements ModelResult<Image> {
|
||||
|
||||
== Available Implementations
|
||||
|
||||
`ImageClient` implementations are provided for the following Model providers:
|
||||
`ImageModel` implementations are provided for the following Model providers:
|
||||
|
||||
* xref:api/image/openai-image.adoc[OpenAI Image Generation]
|
||||
* xref:api/image/stabilityai-image.adoc[StabilityAI Image Generation]
|
||||
|
||||
@@ -48,7 +48,7 @@ var userMessage = new UserMessage(
|
||||
"Explain what do you see in this picture?", // content
|
||||
List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData))); // media
|
||||
|
||||
ChatResponse response = chatClient.call(new Prompt(List.of(userMessage)));
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage)));
|
||||
----
|
||||
|
||||
and produce a response like:
|
||||
|
||||
@@ -11,7 +11,7 @@ Another analogy is a SQL statement that contain placeholders for certain express
|
||||
|
||||
As Spring AI evolves, it will introduce higher levels of abstraction for interacting with AI models.
|
||||
The foundational classes described in this section can be likened to JDBC in terms of their role and functionality.
|
||||
The `ChatClient` class, for instance, is analogous to the core JDBC library in the JDK.
|
||||
The `ChatModel` class, for instance, is analogous to the core JDBC library in the JDK.
|
||||
Building upon this, Spring AI can provide helper classes similar to `JdbcTemplate`, Spring Data Repositories, and eventually, more advanced constructs like ChatEngines and Agents that consider past interactions with the model.
|
||||
|
||||
The structure of prompts has evolved over time within the AI field.
|
||||
@@ -24,7 +24,7 @@ OpenAI have introduced even more structure to prompts by categorizing multiple m
|
||||
|
||||
=== Prompt
|
||||
|
||||
It is common to use the `call` method of `ChatClient` that takes a `Prompt` instance and returns an `ChatResponse`.
|
||||
It is common to use the `call` method of `ChatModel` that takes a `Prompt` instance and returns an `ChatResponse`.
|
||||
|
||||
The Prompt class functions as a container for an organized series of Message objects, with each one forming a segment of the overall prompt.
|
||||
Every Message embodies a unique role within the prompt, differing in its content and intent.
|
||||
@@ -152,7 +152,7 @@ The interfaces implemented by this class support different aspects of prompt cre
|
||||
|
||||
`PromptTemplateMessageActions` is tailored for prompt creation through the generation and manipulation of Message objects.
|
||||
|
||||
`PromptTemplateActions` is designed to return the Prompt object, which can be passed to ChatClient for generating a response.
|
||||
`PromptTemplateActions` is designed to return the Prompt object, which can be passed to ChatModel for generating a response.
|
||||
|
||||
While these interfaces might not be used extensively in many projects, they show the different approaches to prompt creation.
|
||||
|
||||
@@ -213,7 +213,7 @@ PromptTemplate promptTemplate = new PromptTemplate("Tell me a {adjective} joke a
|
||||
|
||||
Prompt prompt = promptTemplate.create(Map.of("adjective", adjective, "topic", topic));
|
||||
|
||||
return chatClient.call(prompt).getResult();
|
||||
return chatModel.call(prompt).getResult();
|
||||
```
|
||||
|
||||
Another example taken from the https://github.com/Azure-Samples/spring-ai-azure-workshop/blob/main/3-README-prompt-roles.md[AI Workshop on Roles] is shown below.
|
||||
@@ -237,13 +237,13 @@ Message systemMessage = systemPromptTemplate.createMessage(Map.of("name", name,
|
||||
|
||||
Prompt prompt = new Prompt(List.of(userMessage, systemMessage));
|
||||
|
||||
List<Generation> response = chatClient.call(prompt).getResults();
|
||||
List<Generation> response = chatModel.call(prompt).getResults();
|
||||
|
||||
```
|
||||
|
||||
This shows how you can build up the `Prompt` instance by using the `SystemPromptTemplate` to create a `Message` with the system role passing in placeholder values.
|
||||
The message with the role `user` is then combined with the message of the role `system` to form the prompt.
|
||||
The prompt is then passed to the ChatClient to get a generative response.
|
||||
The prompt is then passed to the ChatModel to get a generative response.
|
||||
|
||||
|
||||
=== Using resources instead of raw Strings
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[[Speech]]
|
||||
= Text-To-Speech (TTS) API
|
||||
= Speech Model API
|
||||
|
||||
Spring AI provides support for OpenAI's Speech API.
|
||||
When additional providers for Speech are implemented, a common `SpeechClient` and `StreamingSpeechClient` interface will be extracted.
|
||||
Spring AI provides support for OpenAI's Text-To-Speech (TTS) API.
|
||||
When additional providers for Speech are implemented, a common `SpeechModel` and `StreamingSpeechModel` interface will be extracted.
|
||||
@@ -68,7 +68,7 @@ OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.build();
|
||||
|
||||
SpeechPrompt speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
|
||||
SpeechResponse response = openAiAudioSpeechClient.call(speechPrompt);
|
||||
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
@@ -94,13 +94,13 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create an `OpenAiAudioSpeechClient`:
|
||||
Next, create an `OpenAiAudioSpeechModel`:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioSpeechClient = new OpenAiAudioSpeechClient(openAiAudioApi);
|
||||
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
|
||||
|
||||
var speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.withResponseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3)
|
||||
@@ -109,7 +109,7 @@ var speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.build();
|
||||
|
||||
var speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
|
||||
SpeechResponse response = openAiAudioSpeechClient.call(speechPrompt);
|
||||
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
|
||||
|
||||
// Accessing metadata (rate limit info)
|
||||
OpenAiAudioSpeechResponseMetadata metadata = response.getMetadata();
|
||||
@@ -125,7 +125,7 @@ The Speech API provides support for real-time audio streaming using chunk transf
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioSpeechClient = new OpenAiAudioSpeechClient(openAiAudioApi);
|
||||
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
|
||||
|
||||
OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
.withVoice(OpenAiAudioApi.SpeechRequest.Voice.ALLOY)
|
||||
@@ -136,9 +136,9 @@ OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
|
||||
|
||||
SpeechPrompt speechPrompt = new SpeechPrompt("Today is a wonderful day to build something people love!", speechOptions);
|
||||
|
||||
Flux<SpeechResponse> responseStream = openAiAudioSpeechClient.stream(speechPrompt);
|
||||
Flux<SpeechResponse> responseStream = openAiAudioSpeechModel.stream(speechPrompt);
|
||||
----
|
||||
|
||||
== Example Code
|
||||
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/speech/OpenAiSpeechClientIT.java[OpenAiSpeechClientIT.java] test provides some general examples of how to use the library.
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/speech/OpenAiSpeechModelIT.java[OpenAiSpeechModelIT.java] test provides some general examples of how to use the library.
|
||||
|
||||
@@ -123,7 +123,7 @@ String template = """
|
||||
{format}
|
||||
""";
|
||||
|
||||
Generation generation = chatClient.call(
|
||||
Generation generation = chatModel.call(
|
||||
new Prompt(new PromptTemplate(template, Map.of("actor", actor, "format", format)).createMessage())).getResult();
|
||||
|
||||
ActorsFilms actorsFilms = beanOutputConverter.convert(generation.getOutput().getContent());
|
||||
@@ -147,7 +147,7 @@ String template = """
|
||||
|
||||
Prompt prompt = new Prompt(new PromptTemplate(template, Map.of("format", format)).createMessage());
|
||||
|
||||
Generation generation = chatClient.call(prompt).getResult();
|
||||
Generation generation = chatModel.call(prompt).getResult();
|
||||
|
||||
List<ActorsFilmsRecord> actorsFilms = outputConverter.convert(generation.getOutput().getContent());
|
||||
----
|
||||
@@ -168,7 +168,7 @@ String template = """
|
||||
PromptTemplate promptTemplate = new PromptTemplate(template,
|
||||
Map.of("subject", "an array of numbers from 1 to 9 under they key name 'numbers'", "format", format));
|
||||
Prompt prompt = new Prompt(promptTemplate.createMessage());
|
||||
Generation generation = chatClient.call(prompt).getResult();
|
||||
Generation generation = chatModel.call(prompt).getResult();
|
||||
|
||||
Map<String, Object> result = mapOutputConverter.convert(generation.getOutput().getContent());
|
||||
----
|
||||
@@ -189,7 +189,7 @@ String template = """
|
||||
PromptTemplate promptTemplate = new PromptTemplate(template,
|
||||
Map.of("subject", "ice cream flavors", "format", format));
|
||||
Prompt prompt = new Prompt(promptTemplate.createMessage());
|
||||
Generation generation = this.chatClient.call(prompt).getResult();
|
||||
Generation generation = this.chatModel.call(prompt).getResult();
|
||||
|
||||
List<String> list = listOutputConverter.convert(generation.getOutput().getContent());
|
||||
----
|
||||
@@ -201,16 +201,16 @@ The following AI Models have been tested to support List, Map and Bean structure
|
||||
[cols="2,5"]
|
||||
|====
|
||||
| Model | Integration Tests / Samples
|
||||
| xref:api/chat/openai-chat.adoc[OpenAI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatClientIT.java[OpenAiChatClientIT]
|
||||
| xref:api/chat/anthropic-chat.adoc[Anthropic Claude 3] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic/AnthropicChatClientIT.java[AnthropicChatClientIT.java]
|
||||
| xref:api/chat/azure-openai-chat.adoc[Azure OpenAI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/test/java/org/springframework/ai/azure/openai/AzureOpenAiChatClientIT.java[AzureOpenAiChatClientIT.java]
|
||||
| xref:api/chat/mistralai-chat.adoc[Mistral AI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/test/java/org/springframework/ai/mistralai/MistralAiChatClientIT.java[MistralAiChatClientIT.java]
|
||||
| xref:api/chat/ollama-chat.adoc[Ollama] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaChatClientIT.java[OllamaChatClientIT.java]
|
||||
| xref:api/chat/vertexai-gemini-chat.adoc[Vertex AI Gemini] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/test/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatClientIT.java[VertexAiGeminiChatClientIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-anthropic.adoc[Bedrock Anthropic 2] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClientIT.java[BedrockAnthropicChatClientIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-anthropic3.adoc[Bedrock Anthropic 3] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatClientIT.java[BedrockAnthropic3ChatClientIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-cohere.adoc[Bedrock Cohere] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClientIT.java[BedrockCohereChatClientIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-llama.adoc[Bedrock Llama] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatClientIT.java[BedrockLlamaChatClientIT.java.java]
|
||||
| xref:api/chat/openai-chat.adoc[OpenAI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatModelIT.java[OpenAiChatModelIT]
|
||||
| xref:api/chat/anthropic-chat.adoc[Anthropic Claude 3] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-anthropic/src/test/java/org/springframework/ai/anthropic/AnthropicChatModelIT.java[AnthropicChatModelIT.java]
|
||||
| xref:api/chat/azure-openai-chat.adoc[Azure OpenAI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/test/java/org/springframework/ai/azure/openai/AzureOpenAiChatModelIT.java[AzureOpenAiChatModelIT.java]
|
||||
| xref:api/chat/mistralai-chat.adoc[Mistral AI] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/test/java/org/springframework/ai/mistralai/MistralAiChatModelIT.java[MistralAiChatModelIT.java]
|
||||
| xref:api/chat/ollama-chat.adoc[Ollama] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/test/java/org/springframework/ai/ollama/OllamaChatModelIT.java[OllamaChatModelIT.java]
|
||||
| xref:api/chat/vertexai-gemini-chat.adoc[Vertex AI Gemini] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai-gemini/src/test/java/org/springframework/ai/vertexai/gemini/VertexAiGeminiChatModelIT.java[VertexAiGeminiChatModelIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-anthropic.adoc[Bedrock Anthropic 2] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatModelIT.java[BedrockAnthropicChatModelIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-anthropic3.adoc[Bedrock Anthropic 3] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/anthropic3/BedrockAnthropic3ChatModelIT.java[BedrockAnthropic3ChatModelIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-cohere.adoc[Bedrock Cohere] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatModelIT.java[BedrockCohereChatModelIT.java]
|
||||
| xref:api/chat/bedrock/bedrock-llama.adoc[Bedrock Llama] | link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/test/java/org/springframework/ai/bedrock/llama/BedrockLlamaChatModelIT.java[BedrockLlamaChatModelIT.java.java]
|
||||
|====
|
||||
|
||||
== Build-in JSON mode
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[[Transcription]]
|
||||
= Transcription API
|
||||
= Transcription Model API
|
||||
|
||||
Spring AI provides support for OpenAI's Transcription API.
|
||||
When additional providers for Transcription are implemented, a common `AudioTranscriptionClient` interface will be extracted.
|
||||
Spring AI provides support for OpenAI's Transcription Model API.
|
||||
When additional providers for Transcription are implemented, a common `AudioTranscriptionModel` interface will be extracted.
|
||||
@@ -37,7 +37,7 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
=== Transcription Properties
|
||||
|
||||
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI Image client.
|
||||
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI image model.
|
||||
|
||||
[cols="3,5,2"]
|
||||
|====
|
||||
@@ -69,7 +69,7 @@ OpenAiAudioTranscriptionOptions transcriptionOptions = OpenAiAudioTranscriptionO
|
||||
.withResponseFormat(responseFormat)
|
||||
.build();
|
||||
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionClient.call(transcriptionRequest);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
@@ -95,13 +95,13 @@ dependencies {
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `OpenAiAudioTranscriptionClient`
|
||||
Next, create a `OpenAiAudioTranscriptionModel`
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
|
||||
|
||||
var openAiAudioTranscriptionClient = new OpenAiAudioTranscriptionClient(openAiAudioApi);
|
||||
var openAiAudioTranscriptionModel = new OpenAiAudioTranscriptionModel(openAiAudioApi);
|
||||
|
||||
var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
|
||||
.withResponseFormat(TranscriptResponseFormat.TEXT)
|
||||
@@ -111,8 +111,8 @@ var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
|
||||
var audioFile = new FileSystemResource("/path/to/your/resource/speech/jfk.flac");
|
||||
|
||||
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionClient.call(transcriptionRequest);
|
||||
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
|
||||
----
|
||||
|
||||
== Example Code
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionClientIT.java[OpenAiTranscriptionClientIT.java] test provides some general examples how to use the library.
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionModelIT.java[OpenAiTranscriptionModelIT.java] test provides some general examples how to use the library.
|
||||
@@ -72,7 +72,7 @@ It also contains metadata in the form of key-value pairs, including details such
|
||||
|
||||
Upon insertion into the vector database, the text content is transformed into a numerical array, or a `List<Double>`, known as vector embeddings, using an embedding model. Embedding models, such as https://en.wikipedia.org/wiki/Word2vec[Word2Vec], https://en.wikipedia.org/wiki/GloVe_(machine_learning)[GLoVE], and https://en.wikipedia.org/wiki/BERT_(language_model)[BERT], or OpenAI's `text-embedding-ada-002`, are used to convert words, sentences, or paragraphs into these vector embeddings.
|
||||
|
||||
The vector database's role is to store and facilitate similarity searches for these embeddings. It does not generate the embeddings itself. For creating vector embeddings, the `EmbeddingClient` should be utilized.
|
||||
The vector database's role is to store and facilitate similarity searches for these embeddings. It does not generate the embeddings itself. For creating vector embeddings, the `EmbeddingModel` should be utilized.
|
||||
|
||||
The `similaritySearch` methods in the interface allow for retrieving documents similar to a given query string. These methods can be fine-tuned by using the following parameters:
|
||||
|
||||
@@ -113,9 +113,9 @@ Information on each of the `VectorStore` implementations can be found in the sub
|
||||
|
||||
To compute the embeddings for a vector database, you need to pick an embedding model that matches the higher-level AI model being used.
|
||||
|
||||
For example, with OpenAI's ChatGPT, we use the `OpenAiEmbeddingClient` and a model named `text-embedding-ada-002`.
|
||||
For example, with OpenAI's ChatGPT, we use the `OpenAiEmbeddingModel` and a model named `text-embedding-ada-002`.
|
||||
|
||||
The Spring Boot starter's auto-configuration for OpenAI makes an implementation of `EmbeddingClient` available in the Spring application context for dependency injection.
|
||||
The Spring Boot starter's auto-configuration for OpenAI makes an implementation of `EmbeddingModel` available in the Spring application context for dependency injection.
|
||||
|
||||
The general usage of loading data into a vector store is something you would do in a batch-like job, by first loading data into Spring AI's `Document` class and then calling the `save` method.
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ It stands out from other HNSW Vector Similarity Search implementations by being
|
||||
|
||||
== Prerequisites
|
||||
|
||||
1. A `EmbeddingClient` instance to compute the document embeddings. This is usually configured as a Spring Bean. Several options are available:
|
||||
1. A `Embedding` instance to compute the document embeddings. This is usually configured as a Spring Bean. Several options are available:
|
||||
|
||||
- `Transformers Embedding` - computes the embedding in your local environment. The default is via ONNX and the all-MiniLM-L6-v2 Sentence Transformers. This just works.
|
||||
- If you want to use OpenAI's Embeddings` - uses the OpenAI embedding endpoint. You need to create an account at link:https://platform.openai.com/signup[OpenAI Signup] and generate the api-key token at link:https://platform.openai.com/account/api-keys[API Keys].
|
||||
@@ -82,11 +82,11 @@ Create a CassandraVectorStore instance connected to your Apache Cassandra databa
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(EmbeddingClient embeddingClient) {
|
||||
public VectorStore vectorStore(EmbeddingModel embeddingModel) {
|
||||
|
||||
CassandraVectorStoreConfig config = CassandraVectorStoreConfig.builder().build();
|
||||
|
||||
return new CassandraVectorStore(config, embeddingClient);
|
||||
return new CassandraVectorStore(config, embeddingModel);
|
||||
}
|
||||
----
|
||||
|
||||
@@ -189,7 +189,7 @@ Then configure the store like:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public CassandraVectorStore store(EmbeddingClient embeddingClient) {
|
||||
public CassandraVectorStore store(EmbeddingModel embeddingModel) {
|
||||
|
||||
List<SchemaColumn> partitionColumns = List.of(new SchemaColumn("wiki", DataTypes.TEXT),
|
||||
new SchemaColumn("language", DataTypes.TEXT), new SchemaColumn("title", DataTypes.TEXT));
|
||||
@@ -226,13 +226,13 @@ public CassandraVectorStore store(EmbeddingClient embeddingClient) {
|
||||
})
|
||||
.build();
|
||||
|
||||
return new CassandraVectorStore(conf, embeddingClient());
|
||||
return new CassandraVectorStore(conf, embeddingModel());
|
||||
}
|
||||
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// default is ONNX all-MiniLM-L6-v2 which is what we want
|
||||
return new TransformersEmbeddingClient();
|
||||
return new TransformersEmbeddingModel();
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -97,13 +97,13 @@ public SearchIndexClient searchIndexClient() {
|
||||
}
|
||||
----
|
||||
|
||||
To create a vector store, you can use the following code by injecting the `SearchIndexClient` bean created in the above sample along with an `EmbeddingClient` provided by the Spring AI library that implements the desired Embeddings interface.
|
||||
To create a vector store, you can use the following code by injecting the `SearchIndexClient` bean created in the above sample along with an `EmbeddingModel` provided by the Spring AI library that implements the desired Embeddings interface.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(SearchIndexClient searchIndexClient, EmbeddingClient embeddingClient) {
|
||||
return new AzureVectorStore(searchIndexClient, embeddingClient,
|
||||
public VectorStore vectorStore(SearchIndexClient searchIndexClient, EmbeddingModel embeddingModel) {
|
||||
return new AzureVectorStore(searchIndexClient, embeddingModel,
|
||||
// Define the metadata fields to be used
|
||||
// in the similarity search filters.
|
||||
List.of(MetadataField.text("country"),
|
||||
|
||||
@@ -8,8 +8,8 @@ link:https://docs.trychroma.com/[Chroma] is the open-source embedding database.
|
||||
|
||||
1. Access to ChromeDB. The <<Run Chroma Locally, setup local ChromaDB>> appendix shows how to set up a DB locally with a Docker container.
|
||||
|
||||
2. `EmbeddingClient` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `ChromaVectorStore`.
|
||||
2. `EmbeddingModel` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `ChromaVectorStore`.
|
||||
|
||||
On startup, the `ChromaVectorStore` creates the required collection if one is not provisioned already.
|
||||
|
||||
@@ -39,16 +39,16 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
// Can be any other EmbeddingClient implementation.
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other EmbeddingModel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
@@ -181,7 +181,7 @@ Add these dependencies to your project:
|
||||
</dependency>
|
||||
----
|
||||
|
||||
* OpenAI: Required for calculating embeddings. You can use any other embedding client implementation.
|
||||
* OpenAI: Required for calculating embeddings. You can use any other embedding model implementation.
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
@@ -218,8 +218,8 @@ Integrate with OpenAI's embeddings by adding the Spring Boot OpenAI starter to y
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore chromaVectorStore(EmbeddingClient embeddingClient, ChromaApi chromaApi) {
|
||||
return new ChromaVectorStore(embeddingClient, chromaApi, "TestCollection");
|
||||
public VectorStore chromaVectorStore(EmbeddingModel embeddingModel, ChromaApi chromaApi) {
|
||||
return new ChromaVectorStore(embeddingModel, chromaApi, "TestCollection");
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section t
|
||||
|
||||
Please have a look at the list of <<elasticsearchvector-properties,configuration parameters>> for the vector store to learn about the default values and configuration options.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
|
||||
Now you can auto-wire the `ElasticsearchVectorStore` as a vector store in your application.
|
||||
@@ -217,14 +217,14 @@ and then create the `ElasticsearchVectorStore` bean:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public ElasticsearchVectorStore vectorStore(EmbeddingClient embeddingClient, RestClient restClient) {
|
||||
return new ElasticsearchVectorStore( restClient, embeddingClient);
|
||||
public ElasticsearchVectorStore vectorStore(EmbeddingModel embeddingModel, RestClient restClient) {
|
||||
return new ElasticsearchVectorStore( restClient, embeddingModel);
|
||||
}
|
||||
|
||||
// This can be any EmbeddingClient implementation.
|
||||
// This can be any EmbeddingModel implementation.
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -65,8 +65,8 @@ public GemFireVectorStoreConfig gemFireVectorStoreConfig() {
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(GemFireVectorStoreConfig config, EmbeddingClient embeddingClient) {
|
||||
return new GemFireVectorStore(config, embeddingClient);
|
||||
public VectorStore vectorStore(GemFireVectorStoreConfig config, EmbeddingModel embeddingModel) {
|
||||
return new GemFireVectorStore(config, embeddingModel);
|
||||
}
|
||||
----
|
||||
- Create a Vector Index which will configure GemFire region.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
== Prerequisites
|
||||
|
||||
* You need a SAP HANA Cloud vector engine account - Refer xref:api/vectordbs/hanadb-provision-a-trial-account.adoc[SAP HANA Cloud vector engine - provision a trial account] guide to create a trial account.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the vector store.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the vector store.
|
||||
|
||||
|
||||
== Auto-configuration
|
||||
@@ -34,7 +34,7 @@ Please have a look at the list of xref:#_hanacloudvectorstore_properties[configu
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
== HanaCloudVectorStore properties
|
||||
|
||||
@@ -226,7 +226,7 @@ public class CricketWorldCupRepository implements HanaVectorRepository<CricketWo
|
||||
}
|
||||
----
|
||||
|
||||
* Now, create a REST Controller class `CricketWorldCupHanaController`, and autowire `ChatClient` and `VectorStore` as dependencies
|
||||
* Now, create a REST Controller class `CricketWorldCupHanaController`, and autowire `ChatModel` and `VectorStore` as dependencies
|
||||
In this controller class, create the following REST endpoints:
|
||||
|
||||
- `/ai/hana-vector-store/cricket-world-cup/purge-embeddings` - to purge all the embeddings from the Vector Store
|
||||
@@ -238,7 +238,7 @@ In this controller class, create the following REST endpoints:
|
||||
package com.interviewpedia.spring.ai.hana;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.springframework.ai.chat.ChatClient;
|
||||
import org.springframework.ai.chat.ChatModel;
|
||||
import org.springframework.ai.chat.messages.UserMessage;
|
||||
import org.springframework.ai.chat.prompt.Prompt;
|
||||
import org.springframework.ai.chat.prompt.SystemPromptTemplate;
|
||||
@@ -267,11 +267,11 @@ import java.util.stream.Collectors;
|
||||
@Slf4j
|
||||
public class CricketWorldCupHanaController {
|
||||
private final VectorStore hanaCloudVectorStore;
|
||||
private final ChatClient chatClient;
|
||||
private final ChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public CricketWorldCupHanaController(ChatClient chatClient, VectorStore hanaCloudVectorStore) {
|
||||
this.chatClient = chatClient;
|
||||
public CricketWorldCupHanaController(ChatModel chatModel, VectorStore hanaCloudVectorStore) {
|
||||
this.chatModel = chatModel;
|
||||
this.hanaCloudVectorStore = hanaCloudVectorStore;
|
||||
}
|
||||
|
||||
@@ -303,7 +303,7 @@ public class CricketWorldCupHanaController {
|
||||
|
||||
var userMessage = new UserMessage(message);
|
||||
Prompt prompt = new Prompt(List.of(similarDocsMessage, userMessage));
|
||||
String generation = chatClient.call(prompt).getResult().getOutput().getContent();
|
||||
String generation = chatModel.call(prompt).getResult().getOutput().getContent();
|
||||
log.info("Generation: {}", generation);
|
||||
return Map.of("generation", generation);
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@ link:https://milvus.io/[Milvus] is an open-source vector database that has garne
|
||||
* A running Milvus instance. The following options are available:
|
||||
** link:https://milvus.io/docs/install_standalone-docker.md[Milvus Standalone]: Docker, Operator, Helm,DEB/RPM, Docker Compose.
|
||||
** link:https://milvus.io/docs/install_cluster-milvusoperator.md[Milvus Cluster]: Operator, Helm.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `MilvusVectorStore`.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `MilvusVectorStore`.
|
||||
|
||||
== Dependencies
|
||||
|
||||
@@ -30,8 +30,8 @@ dependencies {
|
||||
}
|
||||
----
|
||||
|
||||
The Vector Store, also requires an `EmbeddingClient` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingClient Implementations].
|
||||
The Vector Store, also requires an `EmbeddingModel` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingModel Implementations].
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
@@ -99,14 +99,14 @@ To configure MilvusVectorStore in your application, you can use the following se
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(MilvusServiceClient milvusClient, EmbeddingClient embeddingClient) {
|
||||
public VectorStore vectorStore(MilvusServiceClient milvusClient, EmbeddingModel embeddingModel) {
|
||||
MilvusVectorStoreConfig config = MilvusVectorStoreConfig.builder()
|
||||
.withCollectionName("test_vector_store")
|
||||
.withDatabaseName("default")
|
||||
.withIndexType(IndexType.IVF_FLAT)
|
||||
.withMetricType(MetricType.COSINE)
|
||||
.build();
|
||||
return new MilvusVectorStore(milvusClient, embeddingClient, config);
|
||||
return new MilvusVectorStore(milvusClient, embeddingModel, config);
|
||||
}
|
||||
|
||||
@Bean
|
||||
|
||||
@@ -34,16 +34,16 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
// Can be any other EmbeddingClient implementation.
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other EmbeddingModel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Those indexes are powered by Lucene using a Hierarchical Navigable Small World G
|
||||
** link:https://neo4j.com/download/[Neo4j Desktop]
|
||||
** link:https://neo4j.com/cloud/aura-free/[Neo4j Aura]
|
||||
** link:https://neo4j.com/deployment-center/[Neo4j Server] instance
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `Neo4jVectorStore`.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `Neo4jVectorStore`.
|
||||
|
||||
== Dependencies
|
||||
|
||||
@@ -105,16 +105,16 @@ Please have a look at the list of xref:#_neo4jvectorstore_properties[configurati
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
// Can be any other EmbeddingClient implementation.
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other Embeddingmodel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ CREATE INDEX ON vector_store USING HNSW (embedding vector_cosine_ops);
|
||||
|
||||
TIP: replace the `1536` with the actual embedding dimension if you are using a different dimension.
|
||||
|
||||
Next if required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `PgVectorStore`.
|
||||
Next if required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `PgVectorStore`.
|
||||
|
||||
== Auto-Configuration
|
||||
|
||||
@@ -55,10 +55,10 @@ dependencies {
|
||||
}
|
||||
----
|
||||
|
||||
The Vector Store, also requires an `EmbeddingClient` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingClient Implementations].
|
||||
The Vector Store, also requires an `EmbeddingModel` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingModel Implementations].
|
||||
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingClient] add the following dependency to your project:
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingModel] add the following dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
@@ -131,7 +131,7 @@ You can use the following properties in your Spring Boot configuration to custom
|
||||
|
||||
|`spring.ai.vectorstore.pgvector.index-type`| Nearest neighbor search index type. Options are `NONE` - exact nearest neighbor search, `IVFFlat` - index divides vectors into lists, and then searches a subset of those lists that are closest to the query vector. It has faster build times and uses less memory than HNSW, but has lower query performance (in terms of speed-recall tradeoff). `HNSW` - creates a multilayer graph. It has slower build times and uses more memory than IVFFlat, but has better query performance (in terms of speed-recall tradeoff). There’s no training step like IVFFlat, so the index can be created without any data in the table.| HNSW
|
||||
|`spring.ai.vectorstore.pgvector.distance-type`| Search distance type. Defaults to `COSINE_DISTANCE`. But if vectors are normalized to length 1, you can use `EUCLIDEAN_DISTANCE` or `NEGATIVE_INNER_PRODUCT` for best performance.| COSINE_DISTANCE
|
||||
|`spring.ai.vectorstore.pgvector.dimensions`| Embeddings dimension. If not specified explicitly the PgVectorStore will retrieve the dimensions form the provided `EmbeddingClient`. Dimensions are set to the embedding column the on table creation. If you change the dimensions your would have to re-create the vector_store table as well. | -
|
||||
|`spring.ai.vectorstore.pgvector.dimensions`| Embeddings dimension. If not specified explicitly the PgVectorStore will retrieve the dimensions form the provided `EmbeddingModel`. Dimensions are set to the embedding column the on table creation. If you change the dimensions your would have to re-create the vector_store table as well. | -
|
||||
|`spring.ai.vectorstore.pgvector.remove-existing-vector-store-table` | Deletes the existing `vector_store` table on start up. | false
|
||||
|
||||
|===
|
||||
@@ -200,8 +200,8 @@ To configure PgVector in your application, you can use the following setup:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(JdbcTemplate jdbcTemplate, EmbeddingClient embeddingClient) {
|
||||
return new PgVectorStore(jdbcTemplate, embeddingClient);
|
||||
public VectorStore vectorStore(JdbcTemplate jdbcTemplate, EmbeddingModel embeddingModel) {
|
||||
return new PgVectorStore(jdbcTemplate, embeddingModel);
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -8,8 +8,8 @@ link:https://www.pinecone.io/[Pinecone] is a popular cloud-based vector database
|
||||
|
||||
1. Pinecone Account: Before you start, sign up for a link:https://app.pinecone.io/[Pinecone account].
|
||||
2. Pinecone Project: Once registered, create a new project, an index, and generate an API key. You'll need these details for configuration.
|
||||
3. `EmbeddingClient` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `PineconeVectorStore`.
|
||||
3. `EmbeddingModel` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `PineconeVectorStore`.
|
||||
|
||||
To set up `PineconeVectorStore`, gather the following details from your Pinecone account:
|
||||
|
||||
@@ -50,16 +50,16 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
// Can be any other EmbeddingClient implementation.
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other EmbeddingModel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
@@ -203,8 +203,8 @@ This provides you with an implementation of the Embeddings client:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(PineconeVectorStoreConfig config, EmbeddingClient embeddingClient) {
|
||||
return new PineconeVectorStore(config, embeddingClient);
|
||||
public VectorStore vectorStore(PineconeVectorStoreConfig config, EmbeddingModel embeddingModel) {
|
||||
return new PineconeVectorStore(config, embeddingModel);
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -7,12 +7,12 @@ link:https://www.qdrant.tech/[Qdrant] is an open-source, high-performance vector
|
||||
== Prerequisites
|
||||
|
||||
* Qdrant Instance: Set up a Qdrant instance by following the link:https://qdrant.tech/documentation/guides/installation/[installation instructions] in the Qdrant documentation.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `QdrantVectorStore`.
|
||||
* If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `QdrantVectorStore`.
|
||||
|
||||
To set up `QdrantVectorStore`, you'll need the following information from your Qdrant instance: `Host`, `GRPC Port`, `Collection Name`, and `API Key` (if required).
|
||||
|
||||
NOTE: It is recommended that the Qdrant collection is link:https://qdrant.tech/documentation/concepts/collections/#create-a-collection[created] in advance with the appropriate dimensions and configurations.
|
||||
If the collection is not created, the `QdrantVectorStore` will attempt to create one using the `Cosine` similarity and the dimension of the configured `EmbeddingClient`.
|
||||
If the collection is not created, the `QdrantVectorStore` will attempt to create one using the `Cosine` similarity and the dimension of the configured `EmbeddingModel`.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
@@ -35,10 +35,10 @@ dependencies {
|
||||
}
|
||||
----
|
||||
|
||||
The Vector Store, also requires an `EmbeddingClient` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingClient Implementations].
|
||||
The Vector Store, also requires an `EmbeddingModel` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingModel Implementations].
|
||||
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingClient] add the following dependency to your project:
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingModel] add the following dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
@@ -190,7 +190,7 @@ This provides you with an implementation of the Embeddings client:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public QdrantVectorStore vectorStore(EmbeddingClient embeddingClient, QdrantClient qdrantClient) {
|
||||
return new QdrantVectorStore(qdrantClient, "<QDRANT_COLLECTION_NAME>", embeddingClient);
|
||||
public QdrantVectorStore vectorStore(EmbeddingModel embeddingModel, QdrantClient qdrantClient) {
|
||||
return new QdrantVectorStore(qdrantClient, "<QDRANT_COLLECTION_NAME>", embeddingModel);
|
||||
}
|
||||
----
|
||||
|
||||
@@ -16,8 +16,8 @@ link:https://redis.io/docs/interact/search-and-query/[Redis Search and Query] ex
|
||||
- https://app.redislabs.com/#/[Redis Cloud] (recommended)
|
||||
- link:https://hub.docker.com/r/redis/redis-stack[Docker] image _redis/redis-stack:latest_
|
||||
|
||||
2. `EmbeddingClient` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] to generate the embeddings stored by the `RedisVectorStore`.
|
||||
2. `EmbeddingModel` instance to compute the document embeddings. Several options are available:
|
||||
- If required, an API key for the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] to generate the embeddings stored by the `RedisVectorStore`.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
@@ -45,16 +45,16 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Additionally, you will need a configured `EmbeddingClient` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingClient] section for more information.
|
||||
Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.
|
||||
|
||||
Here is an example of the needed bean:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public EmbeddingClient embeddingClient() {
|
||||
// Can be any other EmbeddingClient implementation.
|
||||
return new OpenAiEmbeddingClient(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
public EmbeddingModel embeddingModel() {
|
||||
// Can be any other EmbeddingModel implementation.
|
||||
return new OpenAiEmbeddingModel(new OpenAiApi(System.getenv("SPRING_AI_OPENAI_API_KEY")));
|
||||
}
|
||||
----
|
||||
|
||||
@@ -179,7 +179,7 @@ Then, create a `RedisVectorStore` bean in your Spring configuration:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public VectorStore vectorStore(EmbeddingClient embeddingClient) {
|
||||
public VectorStore vectorStore(EmbeddingModel embeddingModel) {
|
||||
RedisVectorStoreConfig config = RedisVectorStoreConfig.builder()
|
||||
.withURI("redis://localhost:6379")
|
||||
// Define the metadata fields to be used
|
||||
@@ -189,7 +189,7 @@ public VectorStore vectorStore(EmbeddingClient embeddingClient) {
|
||||
MetadataField.numeric("year"))
|
||||
.build();
|
||||
|
||||
return new RedisVectorStore(config, embeddingClient);
|
||||
return new RedisVectorStore(config, embeddingModel);
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ It provides tools to store document embeddings, content, and metadata and to sea
|
||||
|
||||
== Prerequisites
|
||||
|
||||
1. `EmbeddingClient` instance to compute the document embeddings. Several options are available:
|
||||
1. `EmbeddingModel` instance to compute the document embeddings. Several options are available:
|
||||
|
||||
- `Transformers Embedding` - computes the embedding in your local environment. Follow the ONNX Transformers Embedding instructions.
|
||||
- `OpenAI Embedding` - uses the OpenAI embedding endpoint. You need to create an account at link:https://platform.openai.com/signup[OpenAI Signup] and generate the api-key token at link:https://platform.openai.com/account/api-keys[API Keys].
|
||||
@@ -40,10 +40,10 @@ dependencies {
|
||||
}
|
||||
----
|
||||
|
||||
The Vector Store, also requires an `EmbeddingClient` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingClient Implementations].
|
||||
The Vector Store, also requires an `EmbeddingModel` instance to calculate embeddings for the documents.
|
||||
You can pick one of the available xref:api/embeddings.adoc#available-implementations[EmbeddingModel Implementations].
|
||||
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingClient] add the following dependency to your project:
|
||||
For example to use the xref:api/embeddings/openai-embeddings.adoc[OpenAI EmbeddingModel] add the following dependency to your project:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
@@ -230,13 +230,13 @@ This provides you with an implementation of the Embeddings client:
|
||||
[source,java]
|
||||
----
|
||||
@Bean
|
||||
public WeaviateVectorStore vectorStore(EmbeddingClient embeddingClient, WeaviateClient weaviateClient) {
|
||||
public WeaviateVectorStore vectorStore(EmbeddingModel embeddingModel, WeaviateClient weaviateClient) {
|
||||
|
||||
WeaviateVectorStoreConfig.Builder configBuilder = WeaviateVectorStore.WeaviateVectorStoreConfig.builder()
|
||||
.withObjectClass(<YOUR OBJECT CLASS>)
|
||||
.withConsistencyLevel(<YOUR CONSISTENCY LEVEL>);
|
||||
|
||||
return new WeaviateVectorStore(configBuilder.build(), embeddingClient, weaviateClient);
|
||||
return new WeaviateVectorStore(configBuilder.build(), embeddingModel, weaviateClient);
|
||||
}
|
||||
----
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ To contribute a new model, adhere to the following steps:
|
||||
you'll need to develop a low-level client API class. This often involves utilizing the
|
||||
`RestClient` class from the Spring Framework, similar to the `OpenAiApi` class.
|
||||
|
||||
. *Create a ModelClient implementation*
|
||||
. *Create a Model implementation*
|
||||
Ensure your client conforms to the link:https://docs.spring.io/spring-ai/reference/api/generic-model.html[Generic Model API].
|
||||
Use existing request and response classes if your model's inputs and outputs are supported.
|
||||
If not, create new classes for the Generic Model API and establish a new Java package.
|
||||
|
||||
@@ -57,9 +57,9 @@ to
|
||||
=== January 24, 2024 Update
|
||||
|
||||
* Moving the `prompt` and `messages` and `metadata` packages to subpackages of `org.sf.ai.chat`
|
||||
* New functionality is *text to image* clients. Classes are `OpenAiImageClient` and `StabilityAiImageClient`. See the integration tests for usage, docs are coming soon.
|
||||
* New functionality is *text to image* clients. Classes are `OpenAiImageModel` and `StabilityAiImageModel`. See the integration tests for usage, docs are coming soon.
|
||||
* A new package `model` that contains interfaces and base classes to support creating AI Model Clients for any input/output data type combination. At the moment the chat and image model packages implement this. We will be updating the embedding package to this new model soon.
|
||||
* A new "portable options" design pattern. We wanted to provide as much portability in the `ChatClient` as possible across different chat based AI Models. There is a common set of generation options and then those that are specific to a model provider. A sort of "duck typing" approach is used. `ModelOptions` in the model package is a marker interface indicating implementations of this class will provide the options for a model. See `ImageOptions`, a subinterface that defines portable options across all text->image `ImageClient` implementations. Then `StabilityAiImageOptions` and `OpenAiImageOptions` provide the options specific to each model provider. All options classes are created via a fluent API builder all can be passed into the portable `ImageClient` API. These option data types are using in autoconfiguration/configuration properties for the `ImageClient` implementations.
|
||||
* A new "portable options" design pattern. We wanted to provide as much portability in the `ModelCall` as possible across different chat based AI Models. There is a common set of generation options and then those that are specific to a model provider. A sort of "duck typing" approach is used. `ModelOptions` in the model package is a marker interface indicating implementations of this class will provide the options for a model. See `ImageOptions`, a subinterface that defines portable options across all text->image `ImageModel` implementations. Then `StabilityAiImageOptions` and `OpenAiImageOptions` provide the options specific to each model provider. All options classes are created via a fluent API builder all can be passed into the portable `ImageModel` API. These option data types are using in autoconfiguration/configuration properties for the `ImageModel` implementations.
|
||||
|
||||
=== January 13, 2024 Update
|
||||
|
||||
@@ -79,7 +79,7 @@ Merge SimplePersistentVectorStore and InMemoryVectorStore into SimpleVectorStore
|
||||
|
||||
Refactor the Ollama client and related classes and package names
|
||||
|
||||
* Replace the org.springframework.ai.ollama.client.OllamaClient by org.springframework.ai.ollama.OllamaChatClient.
|
||||
* Replace the org.springframework.ai.ollama.client.OllamaClient by org.springframework.ai.ollama.OllamaModelCall.
|
||||
* The OllamaChatClient method signatures have changed.
|
||||
* Rename the org.springframework.ai.autoconfigure.ollama.OllamaProperties into org.springframework.ai.autoconfigure.ollama.OllamaChatProperties and change the suffix to: `spring.ai.ollama.chat`. Some of the properties have changed as well.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user