Add the support of Azure OpenAI image generation
* Add docs
This commit is contained in:
Benoit Moussaud
committed by
Mark Pollack
Mark Pollack
parent
574138a26a
commit
a30cc9d84d
@@ -44,6 +44,7 @@
|
||||
*** xref:api/embeddings/minimax-embeddings.adoc[MiniMax]
|
||||
*** xref:api/embeddings/zhipuai-embeddings.adoc[ZhiPu AI]
|
||||
** xref:api/imageclient.adoc[]
|
||||
*** xref:api/image/azure-openai-image.adoc[Azure OpenAI]
|
||||
*** xref:api/image/openai-image.adoc[OpenAI]
|
||||
*** xref:api/image/stabilityai-image.adoc[Stability]
|
||||
*** xref:api/image/zhipuai-image.adoc[ZhiPuAI]
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
= Azure OpenAI Image Generation
|
||||
|
||||
|
||||
Spring AI supports DALL-E, the Image generation model from Azure OpenAI.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
Obtain your Azure OpenAI `endpoint` and `api-key` from the Azure OpenAI Service section on the link:https://portal.azure.com[Azure Portal].
|
||||
Spring AI defines a configuration property named `spring.ai.azure.openai.api-key` that you should set to the value of the `API Key` obtained from Azure.
|
||||
There is also a configuration property named `spring.ai.azure.openai.endpoint` that you should set to the endpoint URL obtained when provisioning your model in Azure.
|
||||
Exporting environment variables is one way to set these configuration properties:
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT KEY HERE>
|
||||
export SPRING_AI_AZURE_OPENAI_ENDPOINT=<INSERT ENDPOINT URL HERE>
|
||||
----
|
||||
|
||||
=== Deployment Name
|
||||
|
||||
To use run Azure AI applications, create an Azure AI Deployment through the [Azure AI Portal](https://oai.azure.com/portal).
|
||||
|
||||
In Azure, each client must specify a `Deployment Name` to connect to the Azure OpenAI service.
|
||||
|
||||
It's essential to understand that the `Deployment Name` is different from the model you choose to deploy
|
||||
|
||||
For instance, a deployment named 'MyImgAiDeployment' could be configured to use either the `Dalle3` model or the `Dalle2` model.
|
||||
|
||||
For now, to keep things simple, you can create a deployment using the following settings:
|
||||
|
||||
Deployment Name: `MyImgAiDeployment`
|
||||
Model Name: `Dalle3`
|
||||
|
||||
This Azure configuration will align with the default configurations of the Spring Boot Azure AI Starter and its Autoconfiguration feature.
|
||||
|
||||
If you use a different Deployment Name, update the configuration property accordingly:
|
||||
|
||||
```
|
||||
spring.ai.azure.openai.image.options.deployment-name=<my deployment name>
|
||||
```
|
||||
|
||||
The different deployment structures of Azure OpenAI and OpenAI leads to a property in the Azure OpenAI client library named `deploymentOrModelName`.
|
||||
This is because in OpenAI there is no `Deployment Name`, only a `Model Name`.
|
||||
|
||||
=== Add Repositories and BOM
|
||||
|
||||
Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
|
||||
|
||||
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build system.
|
||||
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
Spring AI provides Spring Boot auto-configuration for the Azure OpenAI Chat Client.
|
||||
To enable it add the following dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source, xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-azure-openai-spring-boot-starter'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
=== Image Generation Properties
|
||||
|
||||
|
||||
The prefix `spring.ai.openai.image` is the property prefix that lets you configure the `ImageModel` implementation for OpenAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.azure.openai.image.enabled | Enable OpenAI image model. | true
|
||||
| spring.ai.azure.openai.image.options.n | The number of images to generate. Must be between 1 and 10. For dall-e-3, only n=1 is supported. | -
|
||||
| spring.ai.azure.openai.image.options.model | The model to use for image generation. | AzureOpenAiImageOptions.DEFAULT_IMAGE_MODEL
|
||||
| spring.ai.azure.openai.image.options.quality | The quality of the image that will be generated. HD creates images with finer details and greater consistency across the image. This parameter is only supported for dall-e-3. | -
|
||||
| spring.ai.azure.openai.image.options.response_format | The format in which the generated images are returned. Must be one of URL or b64_json. | -
|
||||
| `spring.ai.openai.image.options.size` | The size of the generated images. Must be one of 256x256, 512x512, or 1024x1024 for dall-e-2. Must be one of 1024x1024, 1792x1024, or 1024x1792 for dall-e-3 models. | -
|
||||
| `spring.ai.openai.image.options.size_width` | The width of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
|
||||
| `spring.ai.openai.image.options.size_height`| The height of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
|
||||
| `spring.ai.openai.image.options.style` | The style of the generated images. Must be one of vivid or natural. Vivid causes the model to lean towards generating hyper-real and dramatic images. Natural causes the model to produce more natural, less hyper-real looking images. This parameter is only supported for dall-e-3. | -
|
||||
| `spring.ai.openai.image.options.user` | A unique identifier representing your end-user, which can help Azure OpenAI to monitor and detect abuse. | -
|
||||
|====
|
||||
|
||||
==== Connection Properties
|
||||
|
||||
The prefix `spring.ai.openai` is used as the property prefix that lets you connect to Azure OpenAI.
|
||||
|
||||
[cols="3,5,1"]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
| spring.ai.azure.openai.endpoint | The URL to connect to | https://my-dalle3.openai.azure.com/
|
||||
| spring.ai.azure.openai.apiKey | The API Key | -
|
||||
|====
|
||||
|
||||
== Runtime Options [[image-options]]
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiImageOptions.java[OpenAiImageOptions.java] provides model configurations, such as the model to use, the quality, the size, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `AzureOpenAiImageModel(OpenAiImageApi openAiImageApi)` constructor and the `withDefaultOptions(OpenAiImageOptions defaultOptions)` method. Alternatively, use the `spring.ai.azure.openai.image.options.*` properties described previously.
|
||||
|
||||
At runtime you can override the default options by adding new, request specific, options to the `ImagePrompt` call.
|
||||
For example to override the OpenAI specific options such as quality and the number of images to create, use the following code example:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ImageResponse response = azureOpenaiImageModel.call(
|
||||
new ImagePrompt("A light cream colored mini golden doodle",
|
||||
OpenAiImageOptions.builder()
|
||||
.withQuality("hd")
|
||||
.withN(4)
|
||||
.withHeight(1024)
|
||||
.withWidth(1024).build())
|
||||
|
||||
);
|
||||
----
|
||||
|
||||
TIP: In addition to the model specific https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-azure-openai/src/main/java/org/springframework/ai/azure/openai/AzureOpenAiImageOptions.java[AzureOpenAiImageOptions] you can use a portable https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/image/ImageOptions.java[ImageOptions] instance, created with the https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/image/ImageOptionsBuilder.java[ImageOptionsBuilder#builder()].
|
||||
Reference in New Issue
Block a user