Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add VertexAi Embeddings Docs + clear READMEs

Browse Source
This commit is contained in:
Christian Tzolov
2024-02-11 18:32:55 +01:00
parent 7ea867d3cb
commit 21d7cdd5c0
21 changed files with 226 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
models/spring-ai-azure-openai/README.md
View File

@@ -1,3 +1,5 @@
# Azure OpenAI
Visit the Spring AI [Azure OpenAI Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/azure-openai-chat.html).
Visit the Spring AI
- [Azure OpenAI Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/azure-openai-chat.html).
- [Azure OpenAI Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/azure-openai-embeddigs.html).

5
models/spring-ai-huggingface/README.md Normal file
View File

@@ -0,0 +1,5 @@
# Huggingface Chat
Visit the Spring AI
- [Huggingface Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/huggingface.html).

4
models/spring-ai-ollama/README.md
View File

@@ -1,3 +1,5 @@
# Ollama Chat
Visit the Spring AI [Ollama Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/ollama-chat.html).
Visit the Spring AI
- [Ollama Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/ollama-chat.html).
- [Ollama Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/ollama-embeddings.html).

5
models/spring-ai-openai/README.md
View File

@@ -1,3 +1,6 @@
# OpenAI Chat
Visit the Spring AI [OpenAI Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/openai-chat.html).
Visit the Spring AI
- [OpenAI Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/openai-chat.html).
- [OpenAI Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/openai-embeddings.html).

4
models/spring-ai-postgresml/README.md Normal file
View File

@@ -0,0 +1,4 @@
# PostgresML Embeddings
Visit the Spring AI
- [PostgresML Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/postgresml-embeddings.html).

6
models/spring-ai-transformers/README.md Normal file
View File

@@ -0,0 +1,6 @@
# Transformers (ONNX) Embeddings
Visit the Spring AI:
- [Transformers Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/onnx.html).

39
models/spring-ai-vertex-ai/README.md
View File

@@ -1,37 +1,6 @@
# Vertex AI
# Vertex AI API client for the Generative Language model
Visit the Spring AI:
- [VertexAI Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/vertexai-chat.html).
- [VertexAI Embedding Documentation](https://docs.spring.io/spring-ai/reference/api/embeddings/vertexai-embeddings.html).
The [Generative Language](https://developers.generativeai.google/api/rest/generativelanguage) PaLM API allows developers to build generative AI applications using the PaLM model. Large Language Models (LLMs) are a powerful, versatile type of machine learning model that enables computers to comprehend and generate natural language through a series of prompts. The PaLM API is based on Google's next generation LLM, PaLM. It excels at a variety of different tasks like code generation, reasoning, and writing. You can use the PaLM API to build generative AI applications for use cases like content generation, dialogue agents, summarization and classification systems, and more.
Based on the [Models REST API](https://developers.generativeai.google/api/rest/generativelanguage/models).
## Prerequisite
To access the PaLM2 REST API you need to obtain an access API KEY form [makersuite](https://makersuite.google.com/app/apikey).
Note: Currently it is not available outside US, but you can use VPN for testing.
## PaLM API
The VertexAI, ChatClient and EmbeddingClient are built on top the [VertexAiApi.java](./src/main/java/org/springframework/ai/vertex/api/VertexAiApi.java) client library:
![PaLM API](./src/test/resources/Google%20Generative%20AI%20-%20PaLM2%20REST%20API.jpg)
Following snippets show how to use the `VertexAiApi` client directly:
```java
VertexAiApi vertexAiApi = new VertexAiApi(< YOUR PALM_API_KEY>);
// Generate
var prompt = new MessagePrompt(List.of(new Message("0", "Hello, how are you?")));
GenerateMessageRequest request = new GenerateMessageRequest(prompt);
GenerateMessageResponse response = vertexAiApi.generateMessage(request);
// Embed text
Embedding embedding = vertexAiApi.embedText("Hello, how are you?");
// Batch embedding
List<Embedding> embeddings = vertexAiApi.batchEmbedText(List.of("Hello, how are you?", "I am fine, thank you!"));
```

5
models/spring-ai-vertex-ai/src/test/java/org/springframework/ai/vertex/VertexAiEmbeddingClientIT.java
View File

@@ -1,18 +1,17 @@
package org.springframework.ai.vertex;
import java.util.List;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.springframework.ai.embedding.EmbeddingResponse;
import org.springframework.ai.vertex.VertexAiEmbeddingClient;
import org.springframework.ai.vertex.api.VertexAiApi;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.Bean;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
@SpringBootTest

5
spring-ai-docs/src/main/antora/modules/ROOT/nav.adoc
View File

@@ -3,14 +3,15 @@
* xref:getting-started.adoc[Getting Started]
* xref:api/index.adoc[]
** xref:api/embeddings.adoc[]
*** xref:api/embeddings/onnx.adoc[]
*** xref:api/embeddings/openai-embeddings.adoc[]
*** xref:api/embeddings/ollama-embeddings.adoc[]
*** xref:api/embeddings/azure-openai-embeddings.adoc[]
*** xref:api/embeddings/postgresml-embeddings.adoc[]
*** xref:api/bedrock.adoc[Amazon Bedrock Embedding]
*** xref:api/embeddings/vertexai-embeddings.adoc[]
*** xref:api/bedrock.adoc[Amazon Bedrock Embeddings]
**** xref:api/embeddings/bedrock-cohere-embedding.adoc[]
**** xref:api/embeddings/bedrock-titan-embedding.adoc[]
*** xref:api/embeddings/onnx.adoc[]
** xref:api/chatclient.adoc[]
*** xref:api/clients/openai-chat.adoc[]
*** xref:api/clients/azure-openai-chat.adoc[]

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/azure-openai-chat.adoc
View File

@@ -135,7 +135,7 @@ public class ChatController {
return Map.of("generation", chatClient.generate(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-anthropic.adoc
View File

@@ -142,7 +142,7 @@ public class ChatController {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-cohere.adoc
View File

@@ -140,7 +140,7 @@ public class ChatController {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-llama2.adoc
View File

@@ -138,7 +138,7 @@ public class ChatController {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-titan.adoc
View File

@@ -136,7 +136,7 @@ public class ChatController {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/ollama-chat.adoc
View File

@@ -153,7 +153,7 @@ public class ChatController {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

4
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/openai-chat.adoc
View File

@@ -134,12 +134,12 @@ public class ChatController {
this.chatClient = chatClient;
}
@GetMapping("/open-ai/generate")
@GetMapping("/ai/generate")
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);

16
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/vertexai-chat.adoc
View File

@@ -10,7 +10,7 @@ To access the PaLM2 REST API you need to obtain an access API KEY form link:http
NOTE: Currently the PaLM API it is not available outside US, but you can use VPN for testing.
The Spring AI project defines a configuration property named `spring.ai.vertex.ai.api-key` that you should set to the value of the `API Key` obtained from openai.com.
The Spring AI project defines a configuration property named `spring.ai.vertex.ai.api-key` that you should set to the value of the `API Key` obtained.
Exporting an environment variable is one way to set that configuration property:
[source,shell]
@@ -45,7 +45,7 @@ TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Ma
=== Chat Properties
The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you connect to OpenAI.
The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you connect to VertexAI.
[cols="3,5,1"]
|====
@@ -72,7 +72,7 @@ TIP: All properties prefixed with `spring.ai.vertex.ai.chat.options` can be over
=== Chat Options [[chat-options]]
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertex/VertexAiChatOptions.java[VertexAiChatOptions.java] provides model configurations, such as the temperature, the topK, etc.
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai/src/main/java/org/springframework/ai/vertex/VertexAiChatOptions.java[VertexAiChatOptions.java] provides model configurations, such as the temperature, the topK, etc.
On start-up, the default options can be configured with the `VertexAiChatClient(api, options)` constructor or the `spring.ai.vertex.ai.chat.options.*` properties.
@@ -94,7 +94,7 @@ TIP: In addition to the model specific `VertexAiChatOptions` you can use a porta
=== Sample Controller (Auto-configuration)
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-openai-spring-boot-starter` to your pom (or gradle) dependencies.
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-vertex-ai-spring-boot-starter` to your pom (or gradle) dependencies.
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi Chat client:
@@ -122,12 +122,12 @@ public class ChatController {
this.chatClient = chatClient;
}
@GetMapping("/open-ai/generate")
@GetMapping("/ai/generate")
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", chatClient.call(message));
}
@GetMapping("/open-ai/generateStream")
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return chatClient.stream(prompt);
@@ -137,7 +137,7 @@ public class ChatController {
== Manual Configuration
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertex/VertexAiChatClient.java[OpenAiChatClient] implements the `ChatClient` and uses the <<low-level-api>> to connect to the VertexAI service.
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertex/VertexAiChatClient.java[VertexAiChatClient] implements the `ChatClient` and uses the <<low-level-api>> to connect to the VertexAI service.
Add the `spring-ai-vertex-ai` dependency to your project's Maven `pom.xml` file:
@@ -181,7 +181,7 @@ The `VertexAiChatOptions.Builder` is fluent options builder.
=== Low-level VertexAiApi Client [[low-level-api]]
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/vertex/api/VertexAiApi.java[VertexAiApi] provides is lightweight Java client for VertexAiApi Chat API.
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai/src/main/java/org/springframework/ai/vertex/api/VertexAiApi.java[VertexAiApi] provides is lightweight Java client for VertexAiApi Chat API.
Following class diagram illustrates the `VertexAiApi` chat interfaces and building blocks:

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings.adoc
View File

@@ -157,3 +157,4 @@ Internally the various `EmbeddingClient` implementations use different low-level
* xref:api/embeddings/postgresml-embeddings.adoc[Spring AI PostgresML Embeddings]
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]
* xref:api/embeddings/vertexai-embeddings.adoc[Spring AI VertexAI Embeddings]

4
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/bedrock-cohere-embedding.adoc
View File

@@ -1,4 +1,4 @@
= Cohere Embedding
= Cohere Embeddings
Provides Bedrock Cohere Embedding client.
Integrate generative AI capabilities into essential apps and workflows that improve business outcomes.
@@ -168,7 +168,7 @@ var cohereEmbeddingApi =new CohereEmbeddingBedrockApi(
EnvironmentVariableCredentialsProvider.create(), Region.US_EAST_1.id(), new ObjectMapper());
var embeddingClient new BedrockCohereEmbeddingClient(cohereEmbeddingApi);
var embeddingClient = new BedrockCohereEmbeddingClient(cohereEmbeddingApi);
EmbeddingResponse embeddingResponse = embeddingClient
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/bedrock-titan-embedding.adoc
View File

@@ -1,4 +1,4 @@
= Titan Embedding
= Titan Embeddings
Provides Bedrock Titan Embedding client.
link:https://aws.amazon.com/bedrock/titan/[Amazon Titan] foundation models (FMs) provide customers with a breadth of high-performing image, multimodal, and text model choices, via a fully managed API.

172
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/vertexai-embeddings.adoc Normal file
View File

@@ -0,0 +1,172 @@
= VertexAI Embeddings
The link:https://developers.generativeai.google/api/rest/generativelanguage[Generative Language] PaLM API allows developers to build generative AI applications using the PaLM model. Large Language Models (LLMs) are a powerful, versatile type of machine learning model that enables computers to comprehend and generate natural language through a series of prompts. The PaLM API is based on Google's next generation LLM, PaLM. It excels at a variety of different tasks like code generation, reasoning, and writing. You can use the PaLM API to build generative AI applications for use cases like content generation, dialogue agents, summarization and classification systems, and more.
Based on the link:https://developers.generativeai.google/api/rest/generativelanguage/models[Models REST API].
== Prerequisites
To access the PaLM2 REST API you need to obtain an access API KEY form link:https://makersuite.google.com/app/apikey[makersuite].
NOTE: Currently the PaLM API it is not available outside US, but you can use VPN for testing.
The Spring AI project defines a configuration property named `spring.ai.vertex.ai.api-key` that you should set to the value of the `API Key` obtained.
Exporting an environment variable is one way to set that configuration property:
[source,shell]
----
export SPRING_AI_VERTEX_AI_API_KEY=<INSERT KEY HERE>
----
== Auto-configuration
Spring AI provides Spring Boot auto-configuration for the VertexAI Embedding Client.
To enable it add the following dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-vertex-ai-spring-boot-starter</artifactId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-vertex-ai-spring-boot-starter:0.8.0-SNAPSHOT'
}
----
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
=== Chat Properties
The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you connect to VertexAI.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.vertex.ai.ai.base-url | The URL to connect to | https://generativelanguage.googleapis.com/v1beta3
| spring.ai.vertex.ai.api-key | The API Key | -
|====
The prefix `spring.ai.vertex.ai.chat` is the property prefix that lets you configure the chat client implementation for VertexAI Chat.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.vertex.ai.chat.model | This is the https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-embeddings[Vertex Embedding model] to use | chat-bison-001
|====
=== Sample Controller (Auto-configuration)
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-vertex-ai-spring-boot-starter` to your pom (or gradle) dependencies.
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the VertexAi Chat client:
[source,application.properties]
----
spring.ai.vertex.ai.api-key=YOUR_API_KEY
spring.ai.vertex.ai.embedding.model=chat-bison-001
----
TIP: replace the `api-key` with your VertexAI credentials.
This will create a `VertexAiEmbeddingClient` implementation that you can inject into your class.
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
[source,java]
----
@RestController
public class EmbeddingController {
private final EmbeddingClient embeddingClient;
@Autowired
public EmbeddingController(EmbeddingClient embeddingClient) {
this.embeddingClient = embeddingClient;
}
@GetMapping("/ai/embedding")
public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
EmbeddingResponse embeddingResponse = this.embeddingClient.embedForResponse(List.of(message));
return Map.of("embedding", embeddingResponse);
}
}
----
== Manual Configuration
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai/src/main/java/org/springframework/ai/vertex/VertexAiEmbeddingClient.java[VertexAiEmbeddingClient] implements the `EmbeddingClient` and uses the <<low-level-api>> to connect to the VertexAI service.
Add the `spring-ai-vertex-ai` dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-vertex-ai</artifactId>
<version>0.8.0-SNAPSHOT</version>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-vertex-ai:0.8.0-SNAPSHOT'
}
----
TIP: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
Next, create a `VertexAiEmbeddingClient` and use it for text generations:
[source,java]
----
VertexAiApi vertexAiApi = new VertexAiApi(< YOUR PALM_API_KEY>);
var embeddingClient = new VertexAiEmbeddingClient(vertexAiApi);
EmbeddingResponse embeddingResponse = embeddingClient
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
----
=== Low-level VertexAiApi Client [[low-level-api]]
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai/src/main/java/org/springframework/ai/vertex/api/VertexAiApi.java[VertexAiApi] provides is lightweight Java client for VertexAiApi Chat API.
Following class diagram illustrates the `VertexAiApi` chat interfaces and building blocks:
image::vertex-ai-chat-low-level-api.jpg[w=800,align="center"]
Here is a simple snippet how to use the api programmatically:
[source,java]
----
VertexAiApi vertexAiApi = new VertexAiApi(< YOUR PALM_API_KEY>);
// Generate
var prompt = new MessagePrompt(List.of(new Message("0", "Hello, how are you?")));
GenerateMessageRequest request = new GenerateMessageRequest(prompt);
GenerateMessageResponse response = vertexAiApi.generateMessage(request);
// Embed text
Embedding embedding = vertexAiApi.embedText("Hello, how are you?");
// Batch embedding
List<Embedding> embeddings = vertexAiApi.batchEmbedText(List.of("Hello, how are you?", "I am fine, thank you!"));
----