Add Bedrock Cohere Embedding Options support
- Realign the Embeddings API docs strcture
This commit is contained in:
Christian Tzolov
Binary file not shown.
|
Before Width: | Height: | Size: 164 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 442 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 143 KiB |
@@ -8,10 +8,12 @@
|
||||
*** xref:api/embeddings/ollama-embeddings.adoc[]
|
||||
*** xref:api/embeddings/azure-openai-embeddings.adoc[]
|
||||
*** xref:api/embeddings/postgresml-embeddings.adoc[]
|
||||
*** xref:api/bedrock.adoc[Amazon Bedrock Embedding]
|
||||
**** xref:api/embeddings/bedrock-cohere-embedding.adoc[]
|
||||
** xref:api/chatclient.adoc[]
|
||||
*** xref:api/clients/openai-chat.adoc[]
|
||||
*** xref:api/clients/azure-openai-chat.adoc[]
|
||||
*** xref:api/clients/bedrock.adoc[]
|
||||
*** xref:api/bedrock.adoc[Amazon Bedrock Chat]
|
||||
**** xref:api/clients/bedrock/bedrock-anthropic.adoc[]
|
||||
**** xref:api/clients/bedrock/bedrock-llama2.adoc[]
|
||||
**** xref:api/clients/bedrock/bedrock-cohere.adoc[]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
= Amazon Bedrock Chat
|
||||
= Amazon Bedrock
|
||||
|
||||
link:https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock] is a managed service that provides foundation models from various AI providers, available through a unified API.
|
||||
|
||||
@@ -87,10 +87,5 @@ For more information, refer to the documentation below for each supported model.
|
||||
* xref:api/clients/bedrock/bedrock-anthropic.adoc[Spring AI Bedrock Anthropic Chat]: `spring.ai.bedrock.anthropic.chat.enabled=true`
|
||||
* xref:api/clients/bedrock/bedrock-llama2.adoc[Spring AI Bedrock Llama2 Chat]: `spring.ai.bedrock.llama2.chat.enabled=true`
|
||||
* xref:api/clients/bedrock/bedrock-cohere.adoc[Spring AI Bedrock Cohere Chat]: `spring.ai.bedrock.cohere.chat.enabled=true`
|
||||
|
||||
|
||||
// * [Spring AI Bedrock Cohere Chat](./README_COHERE_CHAT.md) - `spring.ai.bedrock.cohere.chat.enabled=true`
|
||||
// * [Spring AI Bedrock Cohere Embedding](./README_COHERE_EMBEDDING.md) - `spring.ai.bedrock.cohere.embedding.enabled=true`
|
||||
// * [Spring AI Bedrock Titan Chat](./README_TITAN_CHAT.md) - `spring.ai.bedrock.titan.chat.enabled=true`
|
||||
// * [Spring AI Bedrock Titan Embedding](./README_TITAN_EMBEDDING.md) - `spring.ai.bedrock.titan.embedding.enabled=true`
|
||||
// * (WIP) [Spring AI Bedrock Ai21 Jurassic2 Chat](./README_JURASSIC2_CHAT.md) - `spring.ai.bedrock.jurassic2.chat.enabled=true`
|
||||
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]: `spring.ai.bedrock.cohere.embedding.enabled=true`
|
||||
// * xref:api/clients/bedrock/bedrock-jurassic2-chat.adoc[(WIP)Spring AI Bedrock Jurassic Chat]: `spring.ai.bedrock.jurassic2.chat.enabled=true`
|
||||
@@ -11,7 +11,7 @@ The https://aws.amazon.com/bedrock/claude[AWS Bedrock Anthropic Model Page] and
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Refer to the xref:api/clients/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ The https://aws.amazon.com/bedrock/cohere-command-embed/[AWS Bedrock Cohere Mode
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Refer to the xref:api/clients/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
@@ -106,7 +106,7 @@ TIP: In addition to the model specific https://github.com/spring-projects/spring
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Anthropic Chat client:
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Cohere Chat client:
|
||||
|
||||
[source]
|
||||
----
|
||||
@@ -150,7 +150,7 @@ public class ChatController {
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClient.java[BedrockCohereChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Anthropic service.
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClient.java[BedrockCohereChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
@@ -203,7 +203,7 @@ The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedr
|
||||
|
||||
Following class diagram illustrates the CohereChatBedrockApi interface and building blocks:
|
||||
|
||||
image::bedrock/bedrock-cohere-chat-api.jpg[CohereChatBedrockApi Class Diagram]
|
||||
image::bedrock/bedrock-cohere-chat-low-level-api.jpg[align="center", width="800px"]
|
||||
|
||||
The CohereChatBedrockApi supports the `cohere.command-light-text-v14` and `cohere.command-text-v14` models for both synchronous (e.g. `chatCompletion()`) and streaming (e.g. `chatCompletionStream()`) requests.
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ The https://aws.amazon.com/bedrock/llama-2/[AWS Llama 2 Model Page] and https://
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Refer to the xref:api/clients/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
|
||||
@@ -69,6 +69,28 @@ The prefix `spring.ai.azure.openai.embeddings` is the property prefix that confi
|
||||
| spring.ai.azure.openai.embedding.options.user | An identifier for the caller or end user of the operation. This may be used for tracking or rate-limiting purposes. | -
|
||||
|====
|
||||
|
||||
TIP: All properties prefixed with `spring.ai.azure.openai.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.
|
||||
|
||||
=== Embedding Options [[embedding-options]]
|
||||
|
||||
The `AzureOpenAiEmbeddingOptions` provides the configuration information for the embedding requests.
|
||||
The `AzureOpenAiEmbeddingOptions` offers a builder to create the options.
|
||||
|
||||
At start time use the `AzureOpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, by passing a `AzureOpenAiEmbeddingOptions` instance with your to the `EmbeddingRequest` request.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
AzureOpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
.build()));
|
||||
----
|
||||
|
||||
|
||||
=== Sample Code
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
@@ -89,7 +111,7 @@ public class EmbeddingController {
|
||||
private final EmbeddingClient embeddingClient;
|
||||
|
||||
@Autowired
|
||||
public ChatController(EmbeddingClient embeddingClient) {
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
}
|
||||
|
||||
@@ -146,22 +168,3 @@ EmbeddingResponse embeddingResponse = embeddingClient
|
||||
|
||||
NOTE: the `text-embedding-ada-002` is actually the `Deployment Name` as presented in the Azure AI Portal.
|
||||
|
||||
=== Embedding Options
|
||||
|
||||
The `AzureOpenAiEmbeddingOptions` provides the configuration information for the embedding requests.
|
||||
The `AzureOpenAiEmbeddingOptions` offers a builder to create the options.
|
||||
|
||||
At start time use the `AzureOpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, by passing a `AzureOpenAiEmbeddingOptions` instance with your to the `EmbeddingRequest` request.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
AzureOpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
.build()));
|
||||
----
|
||||
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
= Cohere Embedding
|
||||
|
||||
Provides Bedrock Cohere Embedding client.
|
||||
Integrate generative AI capabilities into essential apps and workflows that improve business outcomes.
|
||||
|
||||
The https://aws.amazon.com/bedrock/cohere-command-embed/[AWS Bedrock Cohere Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
Add the `spring-ai-bedrock-ai-spring-boot-starter` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
|
||||
<version>0.8.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,gradle]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-bedrock-ai-spring-boot-starter:0.8.0-SNAPSHOT'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
=== Enable Cohere Embedding Support
|
||||
|
||||
By default the Cohere model is disabled.
|
||||
To enable it set the `spring.ai.bedrock.cohere.embedding.enabled` property to `true`.
|
||||
Exporting environment variable is one way to set this configuration property:
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_BEDROCK_COHERE_EMBEDDING_ENABLED=true
|
||||
----
|
||||
|
||||
=== Embedding Properties
|
||||
|
||||
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
|
||||
| spring.ai.bedrock.aws.access-key | AWS access key. | -
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.cohere.embedding` (defined in `BedrockCohereEmbeddingProperties`) is the property prefix that configures the embedding client implementation for Cohere.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.bedrock.cohere.embedding.enabled | Enable or disable support for Cohere | false
|
||||
| spring.ai.bedrock.cohere.embedding.model | The model id to use. See the https://github.com/spring-projects/spring-ai/blob/056b95a00efa5b014a1f488329fbd07a46c02378/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/api/CohereEmbeddingBedrockApi.java#L150[CohereEmbeddingModel] for the supported models. | cohere.embed-multilingual-v3
|
||||
| spring.ai.bedrock.cohere.embedding.options.input-type | Prepends special tokens to differentiate each type from one another. You should not mix different types together, except when mixing types for for search and retrieval. In this case, embed your corpus with the search_document type and embedded queries with type search_query type. | SEARCH_DOCUMENT
|
||||
| spring.ai.bedrock.cohere.embedding.options.truncate | Specifies how the API handles inputs longer than the maximum token length. If you specify LEFT or RIGHT, the model discards the input until the remaining input is exactly the maximum input token length for the model. | NONE
|
||||
|====
|
||||
|
||||
Look at the https://github.com/spring-projects/spring-ai/blob/056b95a00efa5b014a1f488329fbd07a46c02378/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/api/CohereEmbeddingBedrockApi.java#L150[CohereEmbeddingModel] for other model IDs.
|
||||
Supported values are: `cohere.embed-multilingual-v3` and `cohere.embed-english-v3`.
|
||||
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
|
||||
|
||||
TIP: All properties prefixed with `spring.ai.bedrock.cohere.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.
|
||||
|
||||
=== Embedding Options [[embedding-options]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingOptions.java[BedrockCohereEmbeddingOptions.java] provides model configurations, such as `input-type` or `truncate`.
|
||||
|
||||
On start-up, the default options can be configured with the `BedrockCohereEmbeddingClient(api, options)` constructor or the `spring.ai.bedrock.cohere.embedding.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `EmbeddingRequest` call.
|
||||
For example to override the default temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
BedrockCohereEmbeddingOptions.builder()
|
||||
.withInputType(InputType.SEARCH_DOCUMENT)
|
||||
.build()));
|
||||
----
|
||||
|
||||
=== Sample Controller (Auto-configuration)
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Cohere Embedding client:
|
||||
|
||||
[source]
|
||||
----
|
||||
spring.ai.bedrock.aws.region=eu-central-1
|
||||
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
|
||||
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}
|
||||
|
||||
spring.ai.bedrock.cohere.embedding.enabled=true
|
||||
spring.ai.bedrock.cohere.embedding.options.input-type=search-document
|
||||
----
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `BedrockCohereEmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingClient.java[BedrockCohereEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the Bedrock Cohere service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bedrock</artifactId>
|
||||
<version>0.8.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,gradle]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereEmbeddingClient.java[BedrockCohereEmbeddingClient] and use it for text embeddings:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var cohereEmbeddingApi =new CohereEmbeddingBedrockApi(
|
||||
CohereEmbeddingModel.COHERE_EMBED_MULTILINGUAL_V1.id(),
|
||||
EnvironmentVariableCredentialsProvider.create(), Region.US_EAST_1.id(), new ObjectMapper());
|
||||
|
||||
|
||||
var embeddingClient new BedrockCohereEmbeddingClient(cohereEmbeddingApi);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
----
|
||||
|
||||
== Low-level CohereEmbeddingBedrockApi Client [[low-level-api]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/cohere/api/CohereEmbeddingBedrockApi.java[CohereEmbeddingBedrockApi] provides is lightweight Java client on top of AWS Bedrock https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command.html[Cohere Command models].
|
||||
|
||||
Following class diagram illustrates the CohereEmbeddingBedrockApi interface and building blocks:
|
||||
|
||||
image::bedrock/bedrock-cohere-embedding-low-level-api.jpg[align="center", width="800px"]
|
||||
|
||||
The CohereEmbeddingBedrockApi supports the `cohere.embed-english-v3` and `cohere.embed-multilingual-v3` models for single and batch embedding computation.
|
||||
|
||||
Here is a simple snippet how to use the api programmatically:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
CohereEmbeddingBedrockApi api = new CohereEmbeddingBedrockApi(
|
||||
CohereEmbeddingModel.COHERE_EMBED_MULTILINGUAL_V1.id(),
|
||||
EnvironmentVariableCredentialsProvider.create(),
|
||||
Region.US_EAST_1.id(), new ObjectMapper());
|
||||
|
||||
CohereEmbeddingRequest request = new CohereEmbeddingRequest(
|
||||
List.of("I like to eat apples", "I like to eat oranges"),
|
||||
CohereEmbeddingRequest.InputType.search_document,
|
||||
CohereEmbeddingRequest.Truncate.NONE);
|
||||
|
||||
CohereEmbeddingResponse response = api.embedding(request);
|
||||
----
|
||||
|
||||
|
||||
@@ -78,7 +78,7 @@ The prefix `spring.ai.ollama.embedding.options` is the property prefix that conf
|
||||
| spring.ai.ollama.embedding.options.embedding-only | ??? | -
|
||||
| spring.ai.ollama.embedding.options.rope-frequency-base | ??? | -
|
||||
| spring.ai.ollama.embedding.options.rope-frequency-scale | ??? | -
|
||||
| spring.ai.ollama.chat.options.num-thread | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | -
|
||||
| spring.ai.ollama.embedding.options.num-thread | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | -
|
||||
| spring.ai.ollama.embedding.options.num-keep | ??? | -
|
||||
| spring.ai.ollama.embedding.options.seed | Sets the random number seed to use for generation. Setting this to a specific number will make the model generate the same text for the same prompt. | 0
|
||||
| spring.ai.ollama.embedding.options.num-predict | Maximum number of tokens to predict when generating text. (Default: 128, -1 = infinite generation, -2 = fill context) | 128
|
||||
@@ -100,7 +100,28 @@ The prefix `spring.ai.ollama.embedding.options` is the property prefix that conf
|
||||
|
||||
NOTE: The `spring.ai.ollama.embedding.options.*` properties are based on the https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values[Ollama Valid Parameters and Values] and https://github.com/jmorganca/ollama/blob/main/api/types.go[Ollama Types]
|
||||
|
||||
=== Sample Controller
|
||||
TIP: All properties prefixed with `spring.ai.ollama.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.
|
||||
|
||||
=== Embedding Options [[embedding-options]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java[OllamaOptions.java] provides the Ollama configurations, such as the model to use, the low level GPU and CPU tunning, etc.
|
||||
|
||||
The default options can be configured using the `spring.ai.ollama.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OllamaEmbeddingClient#withDefaultOptions()` to configure the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OllamaOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OllamaOptions.create()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name"));
|
||||
----
|
||||
|
||||
=== Sample Controller (Auto-configuration)
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
@@ -167,22 +188,3 @@ EmbeddingResponse embeddingResponse = embeddingClient
|
||||
----
|
||||
|
||||
The `OllamaOptions` provides the configuration information for all embedding requests.
|
||||
|
||||
=== Embedding Options
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java[OllamaOptions.java] provides the Ollama configurations, such as the model to use, the low level GPU and CPU tunning, etc.
|
||||
|
||||
The default options can be configured using the `spring.ai.ollama.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OllamaEmbeddingClient#withDefaultOptions()` to configure the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OllamaOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OllamaOptions.create()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name"));
|
||||
----
|
||||
|
||||
@@ -29,7 +29,7 @@ The snippet exports the https://huggingface.co/sentence-transformers/all-MiniLM-
|
||||
|
||||
In place of the all-MiniLM-L6-v2 you can pick any huggingface transformer identifier or provide direct file path.
|
||||
|
||||
== Using the ONNX models
|
||||
== Using the ONNX Transformers models
|
||||
|
||||
Add the `spring-ai-transformers` project to your maven dependencies:
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Spring AI supports the OpenAI's text embeddings models.
|
||||
OpenAI’s text embeddings measure the relatedness of text strings.
|
||||
An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness.
|
||||
|
||||
== Pre-requisites
|
||||
== Prerequisites
|
||||
|
||||
You will need to create an API with OpenAI to access OpenAI embeddings models.
|
||||
|
||||
@@ -72,7 +72,29 @@ The `spring.ai.openai.embedding.base-url` and `spring.ai.openai.embedding.api-ke
|
||||
Similarly, the `spring.ai.openai.embedding.base-url` and `spring.ai.openai.embedding.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
|
||||
|
||||
=== Sample Controller
|
||||
TIP: All properties prefixed with `spring.ai.openai.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.
|
||||
|
||||
=== Embedding Options [[embedding-options]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiEmbeddingOptions.java[OpenAiEmbeddingOptions.java] provides the OpenAI configurations, such as the model to use and etc.
|
||||
|
||||
The default options can be configured using the `spring.ai.openai.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OpenAiEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
.build()));
|
||||
----
|
||||
|
||||
=== Sample Controller (Auto-configuration)
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
@@ -147,23 +169,4 @@ EmbeddingResponse embeddingResponse = embeddingClient
|
||||
The `OpenAiEmbeddingOptions` provides the configuration information for the embedding requests.
|
||||
The options class offers a `builder()` for easy options creation.
|
||||
|
||||
=== Embedding Options
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiEmbeddingOptions.java[OpenAiEmbeddingOptions.java] provides the OpenAI configurations, such as the model to use and etc.
|
||||
|
||||
The default options can be configured using the `spring.ai.openai.embedding.options` properties as well.
|
||||
|
||||
At start-time use the `OpenAiEmbeddingClient` constructor to set the default options used for all embedding requests.
|
||||
At run-time you can override the default options, using a `OpenAiEmbeddingOptions` instance as part of your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
OpenAiEmbeddingOptions.builder()
|
||||
.withModel("Different-Embedding-Model-Deployment-Name")
|
||||
.build()));
|
||||
----
|
||||
|
||||
|
||||
@@ -38,28 +38,54 @@ Use the `spring.ai.postgresml.embedding.options.*` properties to configure your
|
||||
|
||||
=== Embedding Properties
|
||||
|
||||
The prefix `spring.ai.postgres.embedding` is property prefix that configures the `EmbeddingClient` implementation for PostgresML embeddings.
|
||||
The prefix `spring.ai.postgresml.embedding` is property prefix that configures the `EmbeddingClient` implementation for PostgresML embeddings.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.postgres.embedding.options.transformer | The Huggingface transformer model to use for the embedding. | distilbert-base-uncased
|
||||
| spring.ai.postgres.embedding.options.kwargs | Additional transformer specific options. | empty map
|
||||
| spring.ai.postgres.embedding.options.vectorType | PostgresML vector type to use for the embedding. Two options are supported: `PG_ARRAY` and `PG_VECTOR`. | PG_ARRAY
|
||||
| spring.ai.postgres.embedding.options.metadataMode | Document metadata aggregation mode | EMBED
|
||||
| spring.ai.postgresml.embedding.options.transformer | The Huggingface transformer model to use for the embedding. | distilbert-base-uncased
|
||||
| spring.ai.postgresml.embedding.options.kwargs | Additional transformer specific options. | empty map
|
||||
| spring.ai.postgresml.embedding.options.vectorType | PostgresML vector type to use for the embedding. Two options are supported: `PG_ARRAY` and `PG_VECTOR`. | PG_ARRAY
|
||||
| spring.ai.postgresml.embedding.options.metadataMode | Document metadata aggregation mode | EMBED
|
||||
|====
|
||||
|
||||
=== Sample Controller
|
||||
|
||||
TIP: All properties prefixed with `spring.ai.postgresml.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.
|
||||
|
||||
=== EmbeddingOptions [[embedding-options]]
|
||||
|
||||
Use the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/postgresml/PostgresMlEmbeddingOptions.java[PostgresMlEmbeddingOptions.java] to configure the `PostgresMlEmbeddingClient` with options, such as the model to use and etc.
|
||||
|
||||
|
||||
On start you can pass a `PostgresMlEmbeddingOptions` to the `PostgresMlEmbeddingClient` constructor to configure the default options used for all embedding requests.
|
||||
|
||||
At run-time you can override the default options, using a `PostgresMlEmbeddingOptions` in your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
PostgresMlEmbeddingOptions.builder()
|
||||
.withTransformer("intfloat/e5-small")
|
||||
.withVectorType(VectorType.PG_ARRAY)
|
||||
.withKwargs(Map.of("device", "gpu"))
|
||||
.build()));
|
||||
----
|
||||
|
||||
=== Sample Controller (Auto-configuration)
|
||||
|
||||
This will create a `EmbeddingClient` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the `EmbeddingClient` implementation.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
spring.ai.postgres.embedding.options.transformer=distilbert-base-uncased
|
||||
spring.ai.postgres.embedding.options.vectorType=PG_ARRAY
|
||||
spring.ai.postgres.embedding.options.metadataMode=EMBED
|
||||
spring.ai.postgres.embedding.options.kwargs.device=cpu
|
||||
spring.ai.postgresml.embedding.options.transformer=distilbert-base-uncased
|
||||
spring.ai.postgresml.embedding.options.vectorType=PG_ARRAY
|
||||
spring.ai.postgresml.embedding.options.metadataMode=EMBED
|
||||
spring.ai.postgresml.embedding.options.kwargs.device=cpu
|
||||
----
|
||||
|
||||
[source,java]
|
||||
@@ -140,27 +166,4 @@ public EmbeddingClient embeddingClient(JdbcTemplate jdbcTemplate) {
|
||||
}
|
||||
----
|
||||
|
||||
=== EmbeddingOptions
|
||||
|
||||
Use the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/postgresml/PostgresMlEmbeddingOptions.java[PostgresMlEmbeddingOptions.java] to configure the `PostgresMlEmbeddingClient` with options, such as the model to use and etc.
|
||||
|
||||
|
||||
On start you can pass a `PostgresMlEmbeddingOptions` to the `PostgresMlEmbeddingClient` constructor to configure the default options used for all embedding requests.
|
||||
|
||||
At run-time you can override the default options, using a `PostgresMlEmbeddingOptions` in your `EmbeddingRequest`.
|
||||
|
||||
For example to override the default model name for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient.call(
|
||||
new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
|
||||
PostgresMlEmbeddingOptions.builder()
|
||||
.withTransformer("intfloat/e5-small")
|
||||
.withVectorType(VectorType.PG_ARRAY)
|
||||
.withKwargs(Map.of("device", "gpu"))
|
||||
.build()));
|
||||
----
|
||||
|
||||
|
||||
|
||||
@@ -51,7 +51,7 @@ repositories {
|
||||
* xref:api/clients/openai-chat.adoc[OpenAI]
|
||||
* xref:api/clients/azure-openai-chat.adoc[Azure OpenAI]
|
||||
* xref:api/clients/huggingface.adoc[HuggingFace]
|
||||
* xref:api/clients/bedrock.adoc[Bedrock]
|
||||
* xref:api/bedrock.adoc[Bedrock]
|
||||
* xref:api/clients/ollama-chat.adoc[Ollama]
|
||||
|
||||
== Embedding Models
|
||||
|
||||
Reference in New Issue
Block a user