Azure AI: Support OpenAI key and MS Entra auth
- Enable Authentication using OpenAI API key - Use TokenCredential for authorization if provided by the user - Add the spring.ai.azure.openai.openai-api-key for auth with OpenAI service - Update Azure Chat and Embedding docs - Add integration tests for OpenAI connection Resolves #260
This commit is contained in:
Christian Tzolov
committed by
Mark Pollack
Mark Pollack
parent
237feb3437
commit
13a4e76511
@@ -6,35 +6,57 @@ Azure offers Java developers the opportunity to leverage AI's full potential by
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Obtain your Azure OpenAI `endpoint` and `api-key` from the Azure OpenAI Service section on the link:https://portal.azure.com[Azure Portal].
|
||||
Spring AI defines a configuration property named `spring.ai.azure.openai.api-key` that you should set to the value of the `API Key` obtained from Azure.
|
||||
There is also a configuration property named `spring.ai.azure.openai.endpoint` that you should set to the endpoint URL obtained when provisioning your model in Azure.
|
||||
Exporting environment variables is one way to set these configuration properties:
|
||||
The Azure OpenAI client offers three options to connect: using an Azure API key or using an OpenAI API Key, or using Microsoft Entra ID.
|
||||
|
||||
=== Azure API Key & Endpoint
|
||||
|
||||
Obtain your Azure OpenAI `endpoint` and `api-key` from the Azure OpenAI Service section on the https://portal.azure.com[Azure Portal].
|
||||
|
||||
Spring AI defines two configuration properties:
|
||||
|
||||
|
||||
1. `spring.ai.azure.openai.api-key`: Set this to the value of the `API Key` obtained from Azure.
|
||||
2. `spring.ai.azure.openai.endpoint`: Set this to the endpoint URL obtained when provisioning your model in Azure.
|
||||
|
||||
You can set these configuration properties by exporting environment variables:
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT KEY HERE>
|
||||
export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT AZURE KEY HERE>
|
||||
export SPRING_AI_AZURE_OPENAI_ENDPOINT=<INSERT ENDPOINT URL HERE>
|
||||
----
|
||||
|
||||
|
||||
=== OpenAI Key
|
||||
|
||||
To authenticate with the OpenAI service (not Azure), provide an OpenAI API key. This will automatically set the endpoint to https://api.openai.com/v1.
|
||||
|
||||
When using this approach, set the `spring.ai.azure.openai.chat.options.deployment-name` property to the name of the https://platform.openai.com/docs/models[OpenAI model] you wish to use.
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_AZURE_OPENAI_OPENAI_API_KEY=<INSERT OPENAI KEY HERE>
|
||||
----
|
||||
|
||||
=== Microsoft Entra ID
|
||||
|
||||
To authenticate using Microsoft Entra ID (formerly Azure Active Directory), create a `TokenCredential` bean in your configuration.
|
||||
If this bean is available, an `OpenAIClient` instance will be created using the token credentials.
|
||||
bd
|
||||
=== Deployment Name
|
||||
|
||||
To use run Azure AI applications, create an Azure AI Deployment through the [Azure AI Portal](https://oai.azure.com/portal).
|
||||
|
||||
To use Azure AI applications, you need to create an Azure AI Deployment through the link:https://oai.azure.com/portal[Azure AI Portal].
|
||||
In Azure, each client must specify a `Deployment Name` to connect to the Azure OpenAI service.
|
||||
It's important to note that the `Deployment Name` is different from the model you choose to deploy.
|
||||
For example, a deployment named 'MyAiDeployment' could be configured to use either the GPT 3.5 Turbo model or the GPT 4.0 model.
|
||||
|
||||
It's essential to understand that the `Deployment Name` is different from the model you choose to deploy
|
||||
To get started, follow these steps to create a deployment with the default settings:
|
||||
|
||||
For instance, a deployment named 'MyAiDeployment' could be configured to use either the GPT 3.5 Turbo model or the GPT 4.0 model.
|
||||
Deployment Name: `gpt-35-turbo`
|
||||
Model Name: `gpt-35-turbo`
|
||||
|
||||
For now, to keep things simple, you can create a deployment using the following settings:
|
||||
|
||||
Deployment Name: `gpt-35-turbo`
|
||||
Model Name: `gpt-35-turbo`
|
||||
|
||||
This Azure configuration will align with the default configurations of the Spring Boot Azure AI Starter and its Autoconfiguration feature.
|
||||
|
||||
If you use a different Deployment Name, update the configuration property accordingly:
|
||||
This Azure configuration aligns with the default configurations of the Spring Boot Azure AI Starter and its Autoconfiguration feature.
|
||||
If you use a different Deployment Name, make sure to update the configuration property accordingly:
|
||||
|
||||
```
|
||||
spring.ai.azure.openai.chat.options.deployment-name=<my deployment name>
|
||||
@@ -45,6 +67,14 @@ This is because in OpenAI there is no `Deployment Name`, only a `Model Name`.
|
||||
|
||||
NOTE: The property `spring.ai.azure.openai.chat.options.model` has been renamed to `spring.ai.azure.openai.chat.options.deployment-name`.
|
||||
|
||||
NOTE: If you decide to connect to `OpenAI` instead of `Azure OpeanAI`, by setting the `spring.ai.azure.openai.openai-api-key=<Your OpenAI Key>` property,
|
||||
then the `spring.ai.azure.openai.chat.options.deployment-name` is treathed as an link:https://platform.openai.com/docs/models[OpenAI model] name.
|
||||
|
||||
==== Access the OpenAI Model
|
||||
|
||||
You can configure the client to use directly `OpenAI` instead of the `Azure OpenAI` deployed models.
|
||||
For this you need to set the `spring.ai.azure.openai.openai-api-key=<Your OpenAI Key>` instead of `spring.ai.azure.openai.api-key=<Yur Azure OpenAi Key>`.
|
||||
|
||||
=== Add Repositories and BOM
|
||||
|
||||
Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
|
||||
@@ -86,6 +116,9 @@ The prefix `spring.ai.azure.openai` is the property prefix to configure the conn
|
||||
|
||||
| spring.ai.azure.openai.api-key | The Key from Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` | -
|
||||
| spring.ai.azure.openai.endpoint | The endpoint from the Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` | -
|
||||
| spring.ai.azure.openai.openai-api-key | (non Azure) OpenAI API key. Used to authenticate with the OpenAI service, instead of Azure OpenAI.
|
||||
This automatically sets the endpoint to https://api.openai.com/v1. Use either `api-key` or `openai-api-key` property.
|
||||
With this configuraiton the `spring.ai.azure.openai.chat.options.deployment-name` is threated as an https://platform.openai.com/docs/models[OpenAi Model] name.| -
|
||||
|====
|
||||
|
||||
The prefix `spring.ai.azure.openai.chat` is the property prefix that configures the `ChatModel` implementation for Azure OpenAI.
|
||||
@@ -95,9 +128,11 @@ The prefix `spring.ai.azure.openai.chat` is the property prefix that configures
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.azure.openai.chat.enabled | Enable Azure OpenAI chat model. | true
|
||||
| spring.ai.azure.openai.chat.options.deployment-name | * In use with Azure, this refers to the "Deployment Name" of your model, which you can find at https://oai.azure.com/portal. It's important to note that within an Azure OpenAI deployment, the "Deployment Name" is distinct from the model itself. The confusion around these terms stems from the intention to make the Azure OpenAI client library compatible with the original OpenAI endpoint. The deployment structures offered by Azure OpenAI and Sam Altman's OpenAI differ significantly.
|
||||
Deployments model name to provide as part of this completions request.
|
||||
| gpt-35-turbo
|
||||
| spring.ai.azure.openai.chat.options.deployment-name | In use with Azure, this refers to the "Deployment Name" of your model, which you can find at https://oai.azure.com/portal.
|
||||
It's important to note that within an Azure OpenAI deployment, the "Deployment Name" is distinct from the model itself.
|
||||
The confusion around these terms stems from the intention to make the Azure OpenAI client library compatible with the original OpenAI endpoint.
|
||||
The deployment structures offered by Azure OpenAI and Sam Altman's OpenAI differ significantly.
|
||||
Deployments model name to provide as part of this completions request. | gpt-35-turbo
|
||||
| spring.ai.azure.openai.chat.options.maxTokens | The maximum number of tokens to generate. | -
|
||||
| spring.ai.azure.openai.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.7
|
||||
| spring.ai.azure.openai.chat.options.topP | An alternative to sampling with temperature called nucleus sampling. This value causes the model to consider the results of tokens with the provided probability mass. | -
|
||||
|
||||
@@ -10,18 +10,46 @@ The Azure OpenAI embeddings rely on `cosine similarity` to compute similarity be
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Obtain your Azure OpenAI `endpoint` and `api-key` from the Azure OpenAI Service section on the link:https://portal.azure.com[Azure Portal].
|
||||
|
||||
Spring AI defines a configuration property named `spring.ai.azure.openai.api-key` that you should set to the value of the `API Key` obtained from Azure.
|
||||
There is also a configuration property named `spring.ai.azure.openai.endpoint` that you should set to the endpoint URL obtained when provisioning your model in Azure.
|
||||
The Azure OpenAI client offers three options to connect: using an Azure API key or using an OpenAI API Key, or using Microsoft Entra ID.
|
||||
|
||||
|
||||
=== Azure API Key & Endpoint
|
||||
|
||||
Obtain your Azure OpenAI `endpoint` and `api-key` from the Azure OpenAI Service section on the https://portal.azure.com[Azure Portal].
|
||||
|
||||
Spring AI defines two configuration properties:
|
||||
|
||||
|
||||
1. `spring.ai.azure.openai.api-key`: Set this to the value of the `API Key` obtained from Azure.
|
||||
2. `spring.ai.azure.openai.endpoint`: Set this to the endpoint URL obtained when provisioning your model in Azure.
|
||||
|
||||
You can set these configuration properties by exporting environment variables:
|
||||
|
||||
Exporting environment variables is one way to set these configuration properties:
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT KEY HERE>
|
||||
export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT AZURE KEY HERE>
|
||||
export SPRING_AI_AZURE_OPENAI_ENDPOINT=<INSERT ENDPOINT URL HERE>
|
||||
----
|
||||
|
||||
|
||||
=== OpenAI Key
|
||||
|
||||
To authenticate with the OpenAI service (not Azure), provide an OpenAI API key. This will automatically set the endpoint to https://api.openai.com/v1.
|
||||
|
||||
When using this approach, set the `spring.ai.azure.openai.chat.options.deployment-name` property to the name of the https://platform.openai.com/docs/models[OpenAI model] you wish to use.
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_AZURE_OPENAI_OPENAI_API_KEY=<INSERT OPENAI KEY HERE>
|
||||
----
|
||||
|
||||
=== Microsoft Entra ID
|
||||
|
||||
To authenticate using Microsoft Entra ID (formerly Azure Active Directory), create a `TokenCredential` bean in your configuration.
|
||||
If this bean is available, an `OpenAIClient` instance will be created using the token credentials.
|
||||
|
||||
|
||||
=== Add Repositories and BOM
|
||||
|
||||
Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
|
||||
@@ -63,6 +91,9 @@ The prefix `spring.ai.azure.openai` is the property prefix to configure the conn
|
||||
|
||||
| spring.ai.azure.openai.api-key | The Key from Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` | -
|
||||
| spring.ai.azure.openai.endpoint | The endpoint from the Azure AI OpenAI `Keys and Endpoint` section under `Resource Management` | -
|
||||
| spring.ai.azure.openai.openai-api-key | (non Azure) OpenAI API key. Used to authenticate with the OpenAI service, instead of Azure OpenAI.
|
||||
This automatically sets the endpoint to https://api.openai.com/v1. Use either `api-key` or `openai-api-key` property.
|
||||
With this configuraiton the `spring.ai.azure.openai.embedding.options.deployment-name` is threated as an https://platform.openai.com/docs/models[OpenAi Model] name.| -
|
||||
|====
|
||||
|
||||
|
||||
|
||||
@@ -17,17 +17,14 @@ package org.springframework.ai.autoconfigure.azure.openai;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import com.azure.ai.openai.OpenAIClient;
|
||||
import com.azure.ai.openai.OpenAIClientBuilder;
|
||||
import com.azure.core.credential.AzureKeyCredential;
|
||||
import com.azure.core.util.ClientOptions;
|
||||
|
||||
import org.springframework.ai.azure.openai.AzureOpenAiChatModel;
|
||||
import org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel;
|
||||
import org.springframework.ai.azure.openai.AzureOpenAiImageModel;
|
||||
import org.springframework.ai.model.function.FunctionCallback;
|
||||
import org.springframework.ai.model.function.FunctionCallbackContext;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.boot.autoconfigure.AutoConfiguration;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
|
||||
@@ -36,6 +33,14 @@ import org.springframework.context.ApplicationContext;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.util.Assert;
|
||||
import org.springframework.util.CollectionUtils;
|
||||
import org.springframework.util.StringUtils;
|
||||
|
||||
import com.azure.ai.openai.OpenAIClient;
|
||||
import com.azure.ai.openai.OpenAIClientBuilder;
|
||||
import com.azure.core.credential.AzureKeyCredential;
|
||||
import com.azure.core.credential.KeyCredential;
|
||||
import com.azure.core.credential.TokenCredential;
|
||||
import com.azure.core.util.ClientOptions;
|
||||
|
||||
@AutoConfiguration
|
||||
@ConditionalOnClass({ OpenAIClientBuilder.class, AzureOpenAiChatModel.class })
|
||||
@@ -43,16 +48,46 @@ import org.springframework.util.CollectionUtils;
|
||||
AzureOpenAiConnectionProperties.class, AzureOpenAiImageOptionsProperties.class })
|
||||
public class AzureOpenAiAutoConfiguration {
|
||||
|
||||
private final static String APPLICATION_ID = "spring-ai";
|
||||
|
||||
@Bean
|
||||
@ConditionalOnMissingBean
|
||||
@ConditionalOnMissingBean({ OpenAIClient.class, TokenCredential.class })
|
||||
public OpenAIClient openAIClient(AzureOpenAiConnectionProperties connectionProperties) {
|
||||
|
||||
Assert.hasText(connectionProperties.getApiKey(), "API key must not be empty");
|
||||
if (StringUtils.hasText(connectionProperties.getApiKey())) {
|
||||
|
||||
Assert.hasText(connectionProperties.getEndpoint(), "Endpoint must not be empty");
|
||||
|
||||
return new OpenAIClientBuilder().endpoint(connectionProperties.getEndpoint())
|
||||
.credential(new AzureKeyCredential(connectionProperties.getApiKey()))
|
||||
.clientOptions(new ClientOptions().setApplicationId(APPLICATION_ID))
|
||||
.buildClient();
|
||||
}
|
||||
|
||||
// Connect to OpenAI (e.g. not the Azure OpenAI). The deploymentName property is
|
||||
// used as OpenAI model name.
|
||||
if (StringUtils.hasText(connectionProperties.getOpenAiApiKey())) {
|
||||
return new OpenAIClientBuilder().endpoint("https://api.openai.com/v1")
|
||||
.credential(new KeyCredential(connectionProperties.getOpenAiApiKey()))
|
||||
.clientOptions(new ClientOptions().setApplicationId(APPLICATION_ID))
|
||||
.buildClient();
|
||||
}
|
||||
|
||||
throw new IllegalArgumentException("Either API key or OpenAI API key must not be empty");
|
||||
}
|
||||
|
||||
@Bean
|
||||
@ConditionalOnMissingBean
|
||||
@ConditionalOnBean(TokenCredential.class)
|
||||
public OpenAIClient openAIClientWithTokenCredential(AzureOpenAiConnectionProperties connectionProperties,
|
||||
TokenCredential tokenCredential) {
|
||||
|
||||
Assert.notNull(tokenCredential, "TokenCredential must not be null");
|
||||
Assert.hasText(connectionProperties.getEndpoint(), "Endpoint must not be empty");
|
||||
|
||||
return new OpenAIClientBuilder().endpoint(connectionProperties.getEndpoint())
|
||||
.credential(new AzureKeyCredential(connectionProperties.getApiKey()))
|
||||
.clientOptions(new ClientOptions().setApplicationId("spring-ai"))
|
||||
.credential(tokenCredential)
|
||||
.clientOptions(new ClientOptions().setApplicationId(APPLICATION_ID))
|
||||
.buildClient();
|
||||
}
|
||||
|
||||
|
||||
@@ -28,6 +28,12 @@ public class AzureOpenAiConnectionProperties {
|
||||
*/
|
||||
private String apiKey;
|
||||
|
||||
/**
|
||||
* (non Azure) OpenAI API key. Used to authenticate with the OpenAI service, instead
|
||||
* of Azure OpenAI. This automatically sets the endpoint to https://api.openai.com/v1.
|
||||
*/
|
||||
private String openAiApiKey;
|
||||
|
||||
/**
|
||||
* Azure OpenAI API endpoint. From the Azure AI OpenAI `Keys and Endpoint` section
|
||||
* under `Resource Management`.
|
||||
@@ -35,7 +41,7 @@ public class AzureOpenAiConnectionProperties {
|
||||
private String endpoint;
|
||||
|
||||
public String getEndpoint() {
|
||||
return endpoint;
|
||||
return this.endpoint;
|
||||
}
|
||||
|
||||
public void setEndpoint(String endpoint) {
|
||||
@@ -47,7 +53,15 @@ public class AzureOpenAiConnectionProperties {
|
||||
}
|
||||
|
||||
public String getApiKey() {
|
||||
return apiKey;
|
||||
return this.apiKey;
|
||||
}
|
||||
|
||||
public String getOpenAiApiKey() {
|
||||
return this.openAiApiKey;
|
||||
}
|
||||
|
||||
public void setOpenAiApiKey(String openAiApiKey) {
|
||||
this.openAiApiKey = openAiApiKey;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
/*
|
||||
* Copyright 2024 - 2024 the original author or authors.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* https://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
package org.springframework.ai.autoconfigure.azure;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
|
||||
import org.springframework.ai.azure.openai.AzureOpenAiChatModel;
|
||||
import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
import reactor.core.publisher.Flux;
|
||||
|
||||
import org.springframework.ai.autoconfigure.azure.openai.AzureOpenAiAutoConfiguration;
|
||||
import org.springframework.ai.azure.openai.AzureOpenAiEmbeddingModel;
|
||||
import org.springframework.ai.chat.model.ChatResponse;
|
||||
import org.springframework.ai.chat.model.Generation;
|
||||
import org.springframework.ai.embedding.EmbeddingResponse;
|
||||
import org.springframework.ai.chat.prompt.Prompt;
|
||||
import org.springframework.ai.chat.prompt.SystemPromptTemplate;
|
||||
import org.springframework.ai.chat.messages.Message;
|
||||
import org.springframework.ai.chat.messages.UserMessage;
|
||||
import org.springframework.boot.autoconfigure.AutoConfigurations;
|
||||
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
/**
|
||||
* @author Christian Tzolov
|
||||
* @since 1.0.0
|
||||
*/
|
||||
@EnabledIfEnvironmentVariable(named = "OPENAI_API_KEY", matches = ".*")
|
||||
public class AzureOpenAiDirectOpenAiAutoConfigurationIT {
|
||||
|
||||
private static String CHAT_MODEL_NAME = "gpt-4o";
|
||||
|
||||
private static String EMBEDDING_MODEL_NAME = "text-embedding-ada-002";
|
||||
|
||||
private final ApplicationContextRunner contextRunner = new ApplicationContextRunner().withPropertyValues(
|
||||
// @formatter:off
|
||||
"spring.ai.azure.openai.openai-api-key=" + System.getenv("OPENAI_API_KEY"),
|
||||
|
||||
"spring.ai.azure.openai.chat.options.deployment-name=" + CHAT_MODEL_NAME,
|
||||
"spring.ai.azure.openai.chat.options.temperature=0.8",
|
||||
"spring.ai.azure.openai.chat.options.maxTokens=123",
|
||||
"spring.ai.azure.openai.embedding.options.deployment-name=" + EMBEDDING_MODEL_NAME
|
||||
// @formatter:on
|
||||
).withConfiguration(AutoConfigurations.of(AzureOpenAiAutoConfiguration.class));
|
||||
|
||||
private final Message systemMessage = new SystemPromptTemplate("""
|
||||
You are a helpful AI assistant. Your name is {name}.
|
||||
You are an AI assistant that helps people find information.
|
||||
Your name is {name}
|
||||
You should reply to the user's request with your name and also in the style of a {voice}.
|
||||
""").createMessage(Map.of("name", "Bob", "voice", "pirate"));
|
||||
|
||||
private final UserMessage userMessage = new UserMessage(
|
||||
"Tell me about 3 famous pirates from the Golden Age of Piracy and why they did.");
|
||||
|
||||
@Test
|
||||
public void chatCompletion() {
|
||||
contextRunner.run(context -> {
|
||||
AzureOpenAiChatModel chatModel = context.getBean(AzureOpenAiChatModel.class);
|
||||
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage, systemMessage)));
|
||||
assertThat(response.getResult().getOutput().getContent()).contains("Blackbeard");
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
public void chatCompletionStreaming() {
|
||||
contextRunner.run(context -> {
|
||||
|
||||
AzureOpenAiChatModel chatModel = context.getBean(AzureOpenAiChatModel.class);
|
||||
|
||||
Flux<ChatResponse> response = chatModel.stream(new Prompt(List.of(userMessage, systemMessage)));
|
||||
|
||||
List<ChatResponse> responses = response.collectList().block();
|
||||
assertThat(responses.size()).isGreaterThan(10);
|
||||
|
||||
String stitchedResponseContent = responses.stream()
|
||||
.map(ChatResponse::getResults)
|
||||
.flatMap(List::stream)
|
||||
.map(Generation::getOutput)
|
||||
.map(AssistantMessage::getContent)
|
||||
.collect(Collectors.joining());
|
||||
|
||||
assertThat(stitchedResponseContent).contains("Blackbeard");
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void embedding() {
|
||||
contextRunner.run(context -> {
|
||||
AzureOpenAiEmbeddingModel embeddingModel = context.getBean(AzureOpenAiEmbeddingModel.class);
|
||||
|
||||
EmbeddingResponse embeddingResponse = embeddingModel
|
||||
.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
|
||||
assertThat(embeddingResponse.getResults()).hasSize(2);
|
||||
assertThat(embeddingResponse.getResults().get(0).getOutput()).isNotEmpty();
|
||||
assertThat(embeddingResponse.getResults().get(0).getIndex()).isEqualTo(0);
|
||||
assertThat(embeddingResponse.getResults().get(1).getOutput()).isNotEmpty();
|
||||
assertThat(embeddingResponse.getResults().get(1).getIndex()).isEqualTo(1);
|
||||
|
||||
assertThat(embeddingModel.dimensions()).isEqualTo(1536);
|
||||
});
|
||||
}
|
||||
|
||||
}
|
||||
Reference in New Issue
Block a user