Document Bedrock Titan Embedding API and clients

- Remove outdated Bedrock README fiels and add references to the main docs.
This commit is contained in:
Christian Tzolov
2024-02-11 16:16:18 +01:00
parent ed52a3e0a1
commit d38afc50a3
18 changed files with 211 additions and 178 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 202 KiB

View File

@@ -10,6 +10,7 @@
*** xref:api/embeddings/postgresml-embeddings.adoc[]
*** xref:api/bedrock.adoc[Amazon Bedrock Embedding]
**** xref:api/embeddings/bedrock-cohere-embedding.adoc[]
**** xref:api/embeddings/bedrock-titan-embedding.adoc[]
** xref:api/chatclient.adoc[]
*** xref:api/clients/openai-chat.adoc[]
*** xref:api/clients/azure-openai-chat.adoc[]

View File

@@ -74,7 +74,7 @@ Here are the supported `<model>` and `<chat|embedding>` combinations:
| llama2 | Yes | Yes | No
| cohere | Yes | Yes | Yes
| anthropic | Yes | Yes | No
| jurassic2 | Yes | No | No
| jurassic2 (WIP) | Yes | No | No
| titan | Yes | Yes | Yes (however, no batch support)
|====
@@ -89,5 +89,6 @@ For more information, refer to the documentation below for each supported model.
* xref:api/clients/bedrock/bedrock-cohere.adoc[Spring AI Bedrock Cohere Chat]: `spring.ai.bedrock.cohere.chat.enabled=true`
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]: `spring.ai.bedrock.cohere.embedding.enabled=true`
* xref:api/clients/bedrock/bedrock-titan.adoc[Spring AI Bedrock Titan Chat]: `spring.ai.bedrock.titan.chat.enabled=true`
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]: `spring.ai.bedrock.titan.embedding.enabled=true`
// * xref:api/clients/bedrock/bedrock-jurassic2-chat.adoc[(WIP)Spring AI Bedrock Jurassic Chat]: `spring.ai.bedrock.jurassic2.chat.enabled=true`

View File

@@ -156,3 +156,4 @@ Internally the various `EmbeddingClient` implementations use different low-level
* xref:api/embeddings/onnx.adoc[Spring AI Transformers (ONNX) Embeddings]
* xref:api/embeddings/postgresml-embeddings.adoc[Spring AI PostgresML Embeddings]
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]

View File

@@ -0,0 +1,199 @@
= Titan Embedding
Provides Bedrock Titan Embedding client.
link:https://aws.amazon.com/bedrock/titan/[Amazon Titan] foundation models (FMs) provide customers with a breadth of high-performing image, multimodal, and text model choices, via a fully managed API.
Amazon Titan models are created by AWS and pretrained on large datasets, making them powerful, general-purpose models built to support a variety of use cases, while also supporting the responsible use of AI.
Use them as is or privately customize them with your own data.
NOTE: Bedrock Titan Embedding supports Text and Image embedding.
NOTE: Bedrock Titan Embedding does NOT support batch embedding.
The https://aws.amazon.com/bedrock/titan/[AWS Bedrock Titan Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
== Prerequisites
Refer to the xref:api/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
== Auto-configuration
Add the `spring-ai-bedrock-ai-spring-boot-starter` dependency to your project's Maven `pom.xml` file:
[source,xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,gradle]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-bedrock-ai-spring-boot-starter:0.8.0-SNAPSHOT'
}
----
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
=== Enable Titan Embedding Support
By default the Titan embedding model is disabled.
To enable it set the `spring.ai.bedrock.titan.embedding.enabled` property to `true`.
Exporting environment variable is one way to set this configuration property:
[source,shell]
----
export SPRING_AI_BEDROCK_TITAN_EMBEDDING_ENABLED=true
----
=== Embedding Properties
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
[cols="3,4,1"]
|====
| Property | Description | Default
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
| spring.ai.bedrock.aws.access-key | AWS access key. | -
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
|====
The prefix `spring.ai.bedrock.titan.embedding` (defined in `BedrockTitanEmbeddingProperties`) is the property prefix that configures the embedding client implementation for Titan.
[cols="3,4,1"]
|====
| Property | Description | Default
| spring.ai.bedrock.titan.embedding.enabled | Enable or disable support for Titan embedding | false
| spring.ai.bedrock.titan.embedding.model | The model id to use. See the `TitanEmbeddingModel` for the supported models. | amazon.titan-embed-image-v1
|====
Supported values are: `amazon.titan-embed-image-v1` and `amazon.titan-embed-text-v1`.
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
=== Sample Controller (Auto-configuration)
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-bedrock-ai-spring-boot-starter` to your pom (or gradle) dependencies.
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Titan Embedding client:
[source]
----
spring.ai.bedrock.aws.region=eu-central-1
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}
spring.ai.bedrock.titan.embedding.enabled=true
----
TIP: replace the `regions`, `access-key` and `secret-key` with your AWS credentials.
This will create a `EmbeddingController` implementation that you can inject into your class.
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
[source,java]
----
@RestController
public class EmbeddingController {
private final EmbeddingClient embeddingClient;
@Autowired
public EmbeddingController(EmbeddingClient embeddingClient) {
this.embeddingClient = embeddingClient;
}
@GetMapping("/ai/embedding")
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
return Map.of("embedding", embeddingResponse);
}
}
----
== Manual Configuration
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the Bedrock Titan service.
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
[source,xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bedrock</artifactId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,gradle]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
}
----
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
Next, create an https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingClient.java[BedrockTitanEmbeddingClient] and use it for text embeddings:
[source,java]
----
var titanEmbeddingApi = new TitanEmbeddingBedrockApi(
TitanEmbeddingModel.TITAN_EMBED_IMAGE_V1.id(), Region.US_EAST_1.id());
var embeddingClient new BedrockTitanEmbeddingClient(titanEmbeddingApi);
EmbeddingResponse embeddingResponse = embeddingClient
.embedForResponse(List.of("Hello World")); // NOTE titan does not support batch embedding.
----
== Low-level TitanEmbeddingBedrockApi Client [[low-level-api]]
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/api/TitanEmbeddingBedrockApi.java[TitanEmbeddingBedrockApi] provides is lightweight Java client on top of AWS Bedrock https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html[Titan Embedding models].
Following class diagram illustrates the TitanEmbeddingBedrockApi interface and building blocks:
image::bedrock/bedrock-titan-embedding-low-level-api.jpg[align="center", width="500px"]
The TitanEmbeddingBedrockApi supports the `amazon.titan-embed-image-v1` and `amazon.titan-embed-image-v1` models for single and batch embedding computation.
Here is a simple snippet how to use the api programmatically:
[source,java]
----
TitanEmbeddingBedrockApi titanEmbedApi = new TitanEmbeddingBedrockApi(
TitanEmbeddingModel.TITAN_EMBED_TEXT_V1.id(), Region.US_EAST_1.id());
TitanEmbeddingRequest request = TitanEmbeddingRequest.builder()
.withInputText("I like to eat apples.")
.build();
TitanEmbeddingResponse response = titanEmbedApi.embedding(request);
----
To embed an image you need to convert it into `base64` format:
[source,java]
----
TitanEmbeddingBedrockApi titanEmbedApi = new TitanEmbeddingBedrockApi(
TitanEmbeddingModel.TITAN_EMBED_IMAGE_V1.id(), Region.US_EAST_1.id());
byte[] image = new DefaultResourceLoader()
.getResource("classpath:/spring_framework.png")
.getContentAsByteArray();
TitanEmbeddingRequest request = TitanEmbeddingRequest.builder()
.withInputImage(Base64.getEncoder().encodeToString(image))
.build();
TitanEmbeddingResponse response = titanEmbedApi.embedding(request);
----