Document Bedrock Titan Embedding API and clients
- Remove outdated Bedrock README fiels and add references to the main docs.
This commit is contained in:
Binary file not shown.
|
After Width: | Height: | Size: 202 KiB |
@@ -10,6 +10,7 @@
|
||||
*** xref:api/embeddings/postgresml-embeddings.adoc[]
|
||||
*** xref:api/bedrock.adoc[Amazon Bedrock Embedding]
|
||||
**** xref:api/embeddings/bedrock-cohere-embedding.adoc[]
|
||||
**** xref:api/embeddings/bedrock-titan-embedding.adoc[]
|
||||
** xref:api/chatclient.adoc[]
|
||||
*** xref:api/clients/openai-chat.adoc[]
|
||||
*** xref:api/clients/azure-openai-chat.adoc[]
|
||||
|
||||
@@ -74,7 +74,7 @@ Here are the supported `<model>` and `<chat|embedding>` combinations:
|
||||
| llama2 | Yes | Yes | No
|
||||
| cohere | Yes | Yes | Yes
|
||||
| anthropic | Yes | Yes | No
|
||||
| jurassic2 | Yes | No | No
|
||||
| jurassic2 (WIP) | Yes | No | No
|
||||
| titan | Yes | Yes | Yes (however, no batch support)
|
||||
|====
|
||||
|
||||
@@ -89,5 +89,6 @@ For more information, refer to the documentation below for each supported model.
|
||||
* xref:api/clients/bedrock/bedrock-cohere.adoc[Spring AI Bedrock Cohere Chat]: `spring.ai.bedrock.cohere.chat.enabled=true`
|
||||
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]: `spring.ai.bedrock.cohere.embedding.enabled=true`
|
||||
* xref:api/clients/bedrock/bedrock-titan.adoc[Spring AI Bedrock Titan Chat]: `spring.ai.bedrock.titan.chat.enabled=true`
|
||||
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]: `spring.ai.bedrock.titan.embedding.enabled=true`
|
||||
|
||||
// * xref:api/clients/bedrock/bedrock-jurassic2-chat.adoc[(WIP)Spring AI Bedrock Jurassic Chat]: `spring.ai.bedrock.jurassic2.chat.enabled=true`
|
||||
|
||||
@@ -156,3 +156,4 @@ Internally the various `EmbeddingClient` implementations use different low-level
|
||||
* xref:api/embeddings/onnx.adoc[Spring AI Transformers (ONNX) Embeddings]
|
||||
* xref:api/embeddings/postgresml-embeddings.adoc[Spring AI PostgresML Embeddings]
|
||||
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]
|
||||
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
= Titan Embedding
|
||||
|
||||
Provides Bedrock Titan Embedding client.
|
||||
link:https://aws.amazon.com/bedrock/titan/[Amazon Titan] foundation models (FMs) provide customers with a breadth of high-performing image, multimodal, and text model choices, via a fully managed API.
|
||||
Amazon Titan models are created by AWS and pretrained on large datasets, making them powerful, general-purpose models built to support a variety of use cases, while also supporting the responsible use of AI.
|
||||
Use them as is or privately customize them with your own data.
|
||||
|
||||
NOTE: Bedrock Titan Embedding supports Text and Image embedding.
|
||||
|
||||
NOTE: Bedrock Titan Embedding does NOT support batch embedding.
|
||||
|
||||
The https://aws.amazon.com/bedrock/titan/[AWS Bedrock Titan Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
Add the `spring-ai-bedrock-ai-spring-boot-starter` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
|
||||
<version>0.8.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,gradle]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-bedrock-ai-spring-boot-starter:0.8.0-SNAPSHOT'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
=== Enable Titan Embedding Support
|
||||
|
||||
By default the Titan embedding model is disabled.
|
||||
To enable it set the `spring.ai.bedrock.titan.embedding.enabled` property to `true`.
|
||||
Exporting environment variable is one way to set this configuration property:
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_BEDROCK_TITAN_EMBEDDING_ENABLED=true
|
||||
----
|
||||
|
||||
=== Embedding Properties
|
||||
|
||||
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
|
||||
| spring.ai.bedrock.aws.access-key | AWS access key. | -
|
||||
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.bedrock.titan.embedding` (defined in `BedrockTitanEmbeddingProperties`) is the property prefix that configures the embedding client implementation for Titan.
|
||||
|
||||
[cols="3,4,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.bedrock.titan.embedding.enabled | Enable or disable support for Titan embedding | false
|
||||
| spring.ai.bedrock.titan.embedding.model | The model id to use. See the `TitanEmbeddingModel` for the supported models. | amazon.titan-embed-image-v1
|
||||
|====
|
||||
|
||||
Supported values are: `amazon.titan-embed-image-v1` and `amazon.titan-embed-text-v1`.
|
||||
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
|
||||
|
||||
|
||||
=== Sample Controller (Auto-configuration)
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Titan Embedding client:
|
||||
|
||||
[source]
|
||||
----
|
||||
spring.ai.bedrock.aws.region=eu-central-1
|
||||
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
|
||||
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}
|
||||
|
||||
spring.ai.bedrock.titan.embedding.enabled=true
|
||||
----
|
||||
|
||||
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
|
||||
|
||||
This will create a `EmbeddingController` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class EmbeddingController {
|
||||
|
||||
private final EmbeddingClient embeddingClient;
|
||||
|
||||
@Autowired
|
||||
public EmbeddingController(EmbeddingClient embeddingClient) {
|
||||
this.embeddingClient = embeddingClient;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/embedding")
|
||||
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
|
||||
return Map.of("embedding", embeddingResponse);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the Bedrock Titan service.
|
||||
|
||||
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source,xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-bedrock</artifactId>
|
||||
<version>0.8.0-SNAPSHOT</version>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,gradle]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
|
||||
|
||||
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] and use it for text embeddings:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var titanEmbeddingApi = new TitanEmbeddingBedrockApi(
|
||||
TitanEmbeddingModel.TITAN_EMBED_IMAGE_V1.id(), Region.US_EAST_1.id());
|
||||
|
||||
var embeddingClient new BedrockTitanEmbeddingClient(titanEmbeddingApi);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingClient
|
||||
.embedForResponse(List.of("Hello World")); // NOTE titan does not support batch embedding.
|
||||
----
|
||||
|
||||
== Low-level TitanEmbeddingBedrockApi Client [[low-level-api]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/api/TitanEmbeddingBedrockApi.java[TitanEmbeddingBedrockApi] provides is lightweight Java client on top of AWS Bedrock https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html[Titan Embedding models].
|
||||
|
||||
Following class diagram illustrates the TitanEmbeddingBedrockApi interface and building blocks:
|
||||
|
||||
image::bedrock/bedrock-titan-embedding-low-level-api.jpg[align="center", width="500px"]
|
||||
|
||||
The TitanEmbeddingBedrockApi supports the `amazon.titan-embed-image-v1` and `amazon.titan-embed-image-v1` models for single and batch embedding computation.
|
||||
|
||||
Here is a simple snippet how to use the api programmatically:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
TitanEmbeddingBedrockApi titanEmbedApi = new TitanEmbeddingBedrockApi(
|
||||
TitanEmbeddingModel.TITAN_EMBED_TEXT_V1.id(), Region.US_EAST_1.id());
|
||||
|
||||
TitanEmbeddingRequest request = TitanEmbeddingRequest.builder()
|
||||
.withInputText("I like to eat apples.")
|
||||
.build();
|
||||
|
||||
TitanEmbeddingResponse response = titanEmbedApi.embedding(request);
|
||||
----
|
||||
|
||||
To embed an image you need to convert it into `base64` format:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
TitanEmbeddingBedrockApi titanEmbedApi = new TitanEmbeddingBedrockApi(
|
||||
TitanEmbeddingModel.TITAN_EMBED_IMAGE_V1.id(), Region.US_EAST_1.id());
|
||||
|
||||
byte[] image = new DefaultResourceLoader()
|
||||
.getResource("classpath:/spring_framework.png")
|
||||
.getContentAsByteArray();
|
||||
|
||||
|
||||
TitanEmbeddingRequest request = TitanEmbeddingRequest.builder()
|
||||
.withInputImage(Base64.getEncoder().encodeToString(image))
|
||||
.build();
|
||||
|
||||
TitanEmbeddingResponse response = titanEmbedApi.embedding(request);
|
||||
----
|
||||
Reference in New Issue
Block a user