Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Enhance OpenAI Authentication and Configuration

Browse Source 
- Add org-id and project-id properties with unified merging logic
- Update autoconfig and docs for all OpenAI models
- Introduce OpenAiChatOptions#httpHeaders option
- Add integration test for httpHeaders and update docs

Resolves #1141
This commit is contained in:
Christian Tzolov
2024-08-01 14:31:21 +02:00
parent 4c1347db17
commit 3978e8ecc2
17 changed files with 863 additions and 733 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/audio/speech/openai-speech.adoc
View File

@@ -1,4 +1,4 @@
= OpenAI Text-to-Speech (TTS) Integration
= OpenAI Text-to-Speech (TTS)
== Introduction
@@ -37,7 +37,25 @@ dependencies {
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
=== TTS Properties
== Speech Properties
=== Connection Properties
The prefix `spring.ai.openai` is used as the property prefix that lets you connect to OpenAI.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.project-id | Optionally, you can specify which project is used for an API request. | -
|====
TIP: For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), optionally, you can specify which organization and project is used for an API request.
Usage from these API requests will count as usage for the specified organization and project.
=== Configuraiton Properties
The prefix `spring.ai.openai.audio.speech` is used as the property prefix that lets you configure the OpenAI Text-to-Speech client.
@@ -45,12 +63,22 @@ The prefix `spring.ai.openai.audio.speech` is used as the property prefix that l
|====
| Property | Description | Default
| spring.ai.openai.audio.speech.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.audio.speech.api-key | The API Key | -
| spring.ai.openai.audio.speech.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.audio.speech.project-id | Optionally, you can specify which project is used for an API request. | -
| spring.ai.openai.audio.speech.options.model | ID of the model to use. Only tts-1 is currently available. | tts-1
| spring.ai.openai.audio.speech.options.voice | The voice to use for the TTS output. Available options are: alloy, echo, fable, onyx, nova, and shimmer. | alloy
| spring.ai.openai.audio.speech.options.response-format | The format of the audio output. Supported formats are mp3, opus, aac, flac, wav, and pcm. | mp3
| spring.ai.openai.audio.speech.options.speed | The speed of the voice synthesis. The acceptable range is from 0.0 (slowest) to 1.0 (fastest). | 1.0
|====
NOTE: You can override the common `spring.ai.openai.base-url`, `spring.ai.openai.api-key`, `spring.ai.openai.organization-id` and `spring.ai.openai.project-id` properties.
The `spring.ai.openai.audio.speech.base-url`, `spring.ai.openai.audio.speech.api-key`, `spring.ai.openai.audio.speech.organization-id` and `spring.ai.openai.audio.speech.project-id` properties if set take precedence over the common properties.
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
TIP: All properties prefixed with `spring.ai.openai.image.options` can be overridden at runtime.
== Runtime Options [[speech-options]]
The `OpenAiAudioSpeechOptions` class provides the options to use when making a text-to-speech request.

43
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/audio/transcriptions/openai-transcriptions.adoc
View File

@@ -1,20 +1,22 @@
= OpenAI Transcriptions
== OpenAI Transcriptions
Spring AI supports https://platform.openai.com/docs/api-reference/audio/createTranscription[OpenAI's Transcription model].
== Prerequisites
You will need to create an API key with OpenAI to access ChatGPT models.
Create an account at https://platform.openai.com/signup[OpenAI signup page] and generate the token on the https://platform.openai.com/account/api-keys[API Keys page].
The Spring AI project defines a configuration property named `spring.ai.openai.api-key` that you should set to the value of the `API Key` obtained from openai.com.
Exporting an environment variable is one way to set that configuration property:
== Auto-configuration
Spring AI provides Spring Boot auto-configuration for the OpenAI Image Generation Client.
To enable it, add the following dependency to your project's Maven `pom.xml` file:
To enable it add the following dependency to your project's Maven `pom.xml` file:
[source,xml]
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
@@ -35,12 +37,34 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
=== Transcription Properties
==== Connection Properties
The prefix `spring.ai.openai` is used as the property prefix that lets you connect to OpenAI.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.project-id | Optionally, you can specify which project is used for an API request. | -
|====
TIP: For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), optionally, you can specify which organization and project is used for an API request.
Usage from these API requests will count as usage for the specified organization and project.
==== Configuraiton Properties
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI image model.
[cols="3,5,2"]
|====
| Property | Description | Default
| spring.ai.openai.audio.transcription.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.audio.transcription.api-key | The API Key | -
| spring.ai.openai.audio.transcription.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.audio.transcription.project-id | Optionally, you can specify which project is used for an API request. | -
| spring.ai.openai.audio.transcription.options.model | ID of the model to use. Only whisper-1 (which is powered by our open source Whisper V2 model) is currently available. | whisper-1
| spring.ai.openai.audio.transcription.options.response-format | The format of the transcript output, in one of these options: json, text, srt, verbose_json, or vtt. | json
| spring.ai.openai.audio.transcription.options.prompt | An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language. |
@@ -49,10 +73,16 @@ The prefix `spring.ai.openai.audio.transcription` is used as the property prefix
| spring.ai.openai.audio.transcription.options.timestamp_granularities | The timestamp granularities to populate for this transcription. response_format must be set verbose_json to use timestamp granularities. Either or both of these options are supported: word, or segment. Note: There is no additional latency for segment timestamps, but generating word timestamps incurs additional latency. | segment
|====
NOTE: You can override the common `spring.ai.openai.base-url`, `spring.ai.openai.api-key`, `spring.ai.openai.organization-id` and `spring.ai.openai.project-id` properties.
The `spring.ai.openai.audio.transcription.base-url`, `spring.ai.openai.audio.transcription.api-key`, `spring.ai.openai.audio.transcription.organization-id` and `spring.ai.openai.audio.transcription.project-id` properties if set take precedence over the common properties.
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
TIP: All properties prefixed with `spring.ai.openai.image.options` can be overridden at runtime.
== Runtime Options [[image-options]]
The `OpenAiAudioTranscriptionOptions` class provides the options to use when making a transcription.
On start-up, the options specified by `spring.ai.openai.audio.transcription` are used, but you can override these at runtime.
On start-up, the options specified by `spring.ai.openai.audio.transcription` are used but you can override these at runtime.
For example:
@@ -74,7 +104,7 @@ AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptio
Add the `spring-ai-openai` dependency to your project's Maven `pom.xml` file:
[source,xml]
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
@@ -113,5 +143,4 @@ AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptio
----
== Example Code
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionModelIT.java[OpenAiTranscriptionModelIT.java] test provides some general examples how to use the library.
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionModelIT.java[OpenAiTranscriptionModelIT.java] test provides some general examples how to use the library.

11
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat/openai-chat.adoc
View File

@@ -74,10 +74,14 @@ The prefix `spring.ai.openai` is used as the property prefix that lets you conne
|====
| Property | Description | Default
| spring.ai.openai.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.project-id | Optionally, you can specify which project is used for an API request. | -
|====
TIP: For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), optionally, you can specify which organization and project is used for an API request.
Usage from these API requests will count as usage for the specified organization and project.
==== Configuration Properties
@@ -91,6 +95,8 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
| spring.ai.openai.chat.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.chat.completions-path | The path to append to the base-url | `/v1/chat/completions`
| spring.ai.openai.chat.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.chat.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.chat.project-id | Optionally, you can specify which project is used for an API request. | -
| spring.ai.openai.chat.options.model | This is the OpenAI Chat model to use. `gpt-4o`, `gpt-4-turbo`, `gpt-4-turbo-2024-04-09`, `gpt-4-0125-preview`, `gpt-4-turbo-preview`, `gpt-3.5-turbo`, `gpt-3.5-turbo-0125`, `gpt-3.5-turbo-1106`. See the https://platform.openai.com/docs/models[models] page for more information. | `gpt-3.5-turbo`
| spring.ai.openai.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.8
| spring.ai.openai.chat.options.frequencyPenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | 0.0f
@@ -108,6 +114,7 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
| spring.ai.openai.chat.options.functions | List of functions, identified by their names, to enable for function calling in a single prompt requests. Functions with those names must exist in the functionCallbacks registry. | -
| spring.ai.openai.chat.options.stream-usage | (For streaming only) Set to add an additional chunk with token usage statistics for the entire request. The `choices` field for this chunk is an empty array and all other chunks will also include a usage field, but with a null value. | false
| spring.ai.openai.chat.options.parallel-tool-calls | Whether to enable link:https://platform.openai.com/docs/guides/function-calling/parallel-function-calling[parallel function calling] during tool use. | true
| spring.ai.openai.chat.options.http-headers | Optional HTTP headers to be added to the chat completion request. To override the api-key you need to use a `Authorization` header key and you have to prefix the key value with the `Bearer ` prefix. | -
|====
NOTE: You can override the common `spring.ai.openai.base-url` and `spring.ai.openai.api-key` for the `ChatModel` and `EmbeddingModel` implementations.

7
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/openai-embeddings.adoc
View File

@@ -77,8 +77,13 @@ The prefix `spring.ai.openai` is used as the property prefix that lets you conne
| spring.ai.openai.base-url | The URL to connect to | +https://api.openai.com+
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.project-id | Optionally, you can specify which project is used for an API request. | -
|====
TIP: For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), optionally, you can specify which organization and project is used for an API request.
Usage from these API requests will count as usage for the specified organization and project.
==== Configuration Properties
The prefix `spring.ai.openai.embedding` is property prefix that configures the `EmbeddingModel` implementation for OpenAI.
@@ -91,6 +96,8 @@ The prefix `spring.ai.openai.embedding` is property prefix that configures the `
| spring.ai.openai.embedding.base-url | Optional overrides the spring.ai.openai.base-url to provide embedding specific url | -
| spring.ai.openai.chat.embeddings-path | The path to append to the base-url | `/v1/embeddings`
| spring.ai.openai.embedding.api-key | Optional overrides the spring.ai.openai.api-key to provide embedding specific api-key | -
| spring.ai.openai.embedding.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.embedding.project-id | Optionally, you can specify which project is used for an API request. | -
| spring.ai.openai.embedding.metadata-mode | Document content extraction mode. | EMBED
| spring.ai.openai.embedding.options.model | The model to use | text-embedding-ada-002 (other options: text-embedding-3-large, text-embedding-3-small)
| spring.ai.openai.embedding.options.encodingFormat | The format to return the embeddings in. Can be either float or base64. | -

53
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/image/openai-image.adoc
View File

@@ -41,26 +41,6 @@ TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Man
=== Image Generation Properties
The prefix `spring.ai.openai.image` is the property prefix that lets you configure the `ImageModel` implementation for OpenAI.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.image.enabled | Enable OpenAI image model. | true
| spring.ai.openai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.image.options.n | The number of images to generate. Must be between 1 and 10. For dall-e-3, only n=1 is supported. | -
| spring.ai.openai.image.options.model | The model to use for image generation. | OpenAiImageApi.DEFAULT_IMAGE_MODEL
| spring.ai.openai.image.options.quality | The quality of the image that will be generated. HD creates images with finer details and greater consistency across the image. This parameter is only supported for dall-e-3. | -
| spring.ai.openai.image.options.response_format | The format in which the generated images are returned. Must be one of URL or b64_json. | -
| `spring.ai.openai.image.options.size` | The size of the generated images. Must be one of 256x256, 512x512, or 1024x1024 for dall-e-2. Must be one of 1024x1024, 1792x1024, or 1024x1792 for dall-e-3 models. | -
| `spring.ai.openai.image.options.size_width` | The width of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
| `spring.ai.openai.image.options.size_height`| The height of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
| `spring.ai.openai.image.options.style` | The style of the generated images. Must be one of vivid or natural. Vivid causes the model to lean towards generating hyper-real and dramatic images. Natural causes the model to produce more natural, less hyper-real looking images. This parameter is only supported for dall-e-3. | -
| `spring.ai.openai.image.options.user` | A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. | -
|====
==== Connection Properties
The prefix `spring.ai.openai` is used as the property prefix that lets you connect to OpenAI.
@@ -70,9 +50,12 @@ The prefix `spring.ai.openai` is used as the property prefix that lets you conne
| Property | Description | Default
| spring.ai.openai.base-url | The URL to connect to | https://api.openai.com
| spring.ai.openai.api-key | The API Key | -
| spring.ai.openai.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.project-id | Optionally, you can specify which project is used for an API request. | -
|====
==== Configuration Properties
TIP: For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), optionally, you can specify which organization and project is used for an API request.
Usage from these API requests will count as usage for the specified organization and project.
==== Retry Properties
@@ -92,6 +75,34 @@ The prefix `spring.ai.retry` is used as the property prefix that lets you config
| spring.ai.retry.on-http-codes | List of HTTP status codes that should trigger a retry (e.g. to throw TransientAiException). | empty
|====
==== Configuration Properties
The prefix `spring.ai.openai.image` is the property prefix that lets you configure the `ImageModel` implementation for OpenAI.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.image.enabled | Enable OpenAI image model. | true
| spring.ai.openai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.image.organization-id | Optionally you can specify which organization used for an API request. | -
| spring.ai.openai.image.project-id | Optionally, you can specify which project is used for an API request. | -
| spring.ai.openai.image.options.n | The number of images to generate. Must be between 1 and 10. For dall-e-3, only n=1 is supported. | -
| spring.ai.openai.image.options.model | The model to use for image generation. | OpenAiImageApi.DEFAULT_IMAGE_MODEL
| spring.ai.openai.image.options.quality | The quality of the image that will be generated. HD creates images with finer details and greater consistency across the image. This parameter is only supported for dall-e-3. | -
| spring.ai.openai.image.options.response_format | The format in which the generated images are returned. Must be one of URL or b64_json. | -
| `spring.ai.openai.image.options.size` | The size of the generated images. Must be one of 256x256, 512x512, or 1024x1024 for dall-e-2. Must be one of 1024x1024, 1792x1024, or 1024x1792 for dall-e-3 models. | -
| `spring.ai.openai.image.options.size_width` | The width of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
| `spring.ai.openai.image.options.size_height`| The height of the generated images. Must be one of 256, 512, or 1024 for dall-e-2. | -
| `spring.ai.openai.image.options.style` | The style of the generated images. Must be one of vivid or natural. Vivid causes the model to lean towards generating hyper-real and dramatic images. Natural causes the model to produce more natural, less hyper-real looking images. This parameter is only supported for dall-e-3. | -
| `spring.ai.openai.image.options.user` | A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. | -
|====
NOTE: You can override the common `spring.ai.openai.base-url`, `spring.ai.openai.api-key`, `spring.ai.openai.organization-id` and `spring.ai.openai.project-id` properties.
The `spring.ai.openai.image.base-url`, `spring.ai.openai.image.api-key`, `spring.ai.openai.image.organization-id` and `spring.ai.openai.image.project-id` properties if set take precedence over the common properties.
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
TIP: All properties prefixed with `spring.ai.openai.image.options` can be overridden at runtime.
== Runtime Options [[image-options]]

144
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/speech/openai-speech.adoc
View File

@@ -1,144 +0,0 @@
= OpenAI Text-to-Speech (TTS) Integration
== Introduction
The Audio API provides a speech endpoint based on OpenAI's TTS (text-to-speech) model, enabling users to:
- Narrate a written blog post.
- Produce spoken audio in multiple languages.
- Give real-time audio output using streaming.
== Prerequisites
. Create an OpenAI account and obtain an API key. You can sign up at the https://platform.openai.com/signup[OpenAI signup page] and generate an API key on the https://platform.openai.com/account/api-keys[API Keys page].
. Add the `spring-ai-openai` dependency to your project's build file. For more information, refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section.
== Auto-configuration
Spring AI provides Spring Boot auto-configuration for the OpenAI Text-to-Speech Client.
To enable it add the following dependency to your project's Maven `pom.xml` file:
[source,xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file:
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-openai-spring-boot-starter'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
=== TTS Properties
The prefix `spring.ai.openai.audio.speech` is used as the property prefix that lets you configure the OpenAI Text-to-Speech client.
[cols="3,5,2"]
|====
| Property | Description | Default
| spring.ai.openai.audio.speech.options.model | ID of the model to use. Only tts-1 is currently available. | tts-1
| spring.ai.openai.audio.speech.options.voice | The voice to use for the TTS output. Available options are: alloy, echo, fable, onyx, nova, and shimmer. | alloy
| spring.ai.openai.audio.speech.options.response-format | The format of the audio output. Supported formats are mp3, opus, aac, flac, wav, and pcm. | mp3
| spring.ai.openai.audio.speech.options.speed | The speed of the voice synthesis. The acceptable range is from 0.0 (slowest) to 1.0 (fastest). | 1.0
|====
== Runtime Options [[speech-options]]
The `OpenAiAudioSpeechOptions` class provides the options to use when making a text-to-speech request.
On start-up, the options specified by `spring.ai.openai.audio.speech` are used but you can override these at runtime.
For example:
[source,java]
----
OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
.withModel("tts-1")
.withVoice(OpenAiAudioApi.SpeechRequest.Voice.ALLOY)
.withResponseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3)
.withSpeed(1.0f)
.build();
SpeechPrompt speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
----
== Manual Configuration
Add the `spring-ai-openai` dependency to your project's Maven `pom.xml` file:
[source,xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file:
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-openai'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
Next, create an `OpenAiAudioSpeechModel`:
[source,java]
----
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
var speechOptions = OpenAiAudioSpeechOptions.builder()
.withResponseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3)
.withSpeed(1.0f)
.withModel(OpenAiAudioApi.TtsModel.TTS_1.value)
.build();
var speechPrompt = new SpeechPrompt("Hello, this is a text-to-speech example.", speechOptions);
SpeechResponse response = openAiAudioSpeechModel.call(speechPrompt);
// Accessing metadata (rate limit info)
OpenAiAudioSpeechResponseMetadata metadata = response.getMetadata();
byte[] responseAsBytes = response.getResult().getOutput();
----
== Streaming Real-time Audio
The Speech API provides support for real-time audio streaming using chunk transfer encoding. This means that the audio is able to be played before the full file has been generated and made accessible.
[source,java]
----
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
var openAiAudioSpeechModel = new OpenAiAudioSpeechModel(openAiAudioApi);
OpenAiAudioSpeechOptions speechOptions = OpenAiAudioSpeechOptions.builder()
.withVoice(OpenAiAudioApi.SpeechRequest.Voice.ALLOY)
.withSpeed(1.0f)
.withResponseFormat(OpenAiAudioApi.SpeechRequest.AudioResponseFormat.MP3)
.withModel(OpenAiAudioApi.TtsModel.TTS_1.value)
.build();
SpeechPrompt speechPrompt = new SpeechPrompt("Today is a wonderful day to build something people love!", speechOptions);
Flux<SpeechResponse> responseStream = openAiAudioSpeechModel.stream(speechPrompt);
----
== Example Code
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/speech/OpenAiSpeechModelIT.java[OpenAiSpeechModelIT.java] test provides some general examples of how to use the library.

118
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/transcriptions/openai-transcriptions.adoc
View File

@@ -1,118 +0,0 @@
== OpenAI Transcriptions
Spring AI supports https://platform.openai.com/docs/api-reference/audio/createTranscription[OpenAI's Transcription model].
== Prerequisites
You will need to create an API key with OpenAI to access ChatGPT models.
Create an account at https://platform.openai.com/signup[OpenAI signup page] and generate the token on the https://platform.openai.com/account/api-keys[API Keys page].
The Spring AI project defines a configuration property named `spring.ai.openai.api-key` that you should set to the value of the `API Key` obtained from openai.com.
Exporting an environment variable is one way to set that configuration property:
== Auto-configuration
Spring AI provides Spring Boot auto-configuration for the OpenAI Image Generation Client.
To enable it add the following dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-openai-spring-boot-starter'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
=== Transcription Properties
The prefix `spring.ai.openai.audio.transcription` is used as the property prefix that lets you configure the retry mechanism for the OpenAI image model.
[cols="3,5,2"]
|====
| Property | Description | Default
| spring.ai.openai.audio.transcription.options.model | ID of the model to use. Only whisper-1 (which is powered by our open source Whisper V2 model) is currently available. | whisper-1
| spring.ai.openai.audio.transcription.options.response-format | The format of the transcript output, in one of these options: json, text, srt, verbose_json, or vtt. | json
| spring.ai.openai.audio.transcription.options.prompt | An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language. |
| spring.ai.openai.audio.transcription.options.language | The language of the input audio. Supplying the input language in ISO-639-1 format will improve accuracy and latency. |
| spring.ai.openai.audio.transcription.options.temperature | The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit. | 0
| spring.ai.openai.audio.transcription.options.timestamp_granularities | The timestamp granularities to populate for this transcription. response_format must be set verbose_json to use timestamp granularities. Either or both of these options are supported: word, or segment. Note: There is no additional latency for segment timestamps, but generating word timestamps incurs additional latency. | segment
|====
== Runtime Options [[image-options]]
The `OpenAiAudioTranscriptionOptions` class provides the options to use when making a transcription.
On start-up, the options specified by `spring.ai.openai.audio.transcription` are used but you can override these at runtime.
For example:
[source,java]
----
OpenAiAudioApi.TranscriptResponseFormat responseFormat = OpenAiAudioApi.TranscriptResponseFormat.VTT;
OpenAiAudioTranscriptionOptions transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
.withLanguage("en")
.withPrompt("Ask not this, but ask that")
.withTemperature(0f)
.withResponseFormat(responseFormat)
.build();
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
----
== Manual Configuration
Add the `spring-ai-openai` dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-openai'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
Next, create a `OpenAiAudioTranscriptionModel`
[source,java]
----
var openAiAudioApi = new OpenAiAudioApi(System.getenv("OPENAI_API_KEY"));
var openAiAudioTranscriptionModel = new OpenAiAudioTranscriptionModel(openAiAudioApi);
var transcriptionOptions = OpenAiAudioTranscriptionOptions.builder()
.withResponseFormat(TranscriptResponseFormat.TEXT)
.withTemperature(0f)
.build();
var audioFile = new FileSystemResource("/path/to/your/resource/speech/jfk.flac");
AudioTranscriptionPrompt transcriptionRequest = new AudioTranscriptionPrompt(audioFile, transcriptionOptions);
AudioTranscriptionResponse response = openAiTranscriptionModel.call(transcriptionRequest);
----
== Example Code
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/audio/transcription/OpenAiTranscriptionModelIT.java[OpenAiTranscriptionModelIT.java] test provides some general examples how to use the library.