OpenAi: Add support for structured outputs and JSON schema
- Added support for OpenAI's structured outputs feature, which allows specifying a JSON schema for the model to match - Introduced new record to configure the desired response format - Added support for configuring the response format via application properties or the chat options builder - Extend teh BeanOutputConverter to help generate JSON schema from a target domain object and convert the response. - Added comprehensive tests to cover the new response format functionality Resolves #1196
This commit is contained in:
Christian Tzolov
committed by
Mark Pollack
Mark Pollack
parent
866b262cdd
commit
91afed5ae5
@@ -97,14 +97,18 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
|
||||
| spring.ai.openai.chat.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
|
||||
| spring.ai.openai.chat.organization-id | Optionally you can specify which organization used for an API request. | -
|
||||
| spring.ai.openai.chat.project-id | Optionally, you can specify which project is used for an API request. | -
|
||||
| spring.ai.openai.chat.options.model | This is the OpenAI Chat model to use. `gpt-4o`, `gpt-4-turbo`, `gpt-4-turbo-2024-04-09`, `gpt-4-0125-preview`, `gpt-4-turbo-preview`, `gpt-3.5-turbo`, `gpt-3.5-turbo-0125`, `gpt-3.5-turbo-1106`. See the https://platform.openai.com/docs/models[models] page for more information. | `gpt-3.5-turbo`
|
||||
| spring.ai.openai.chat.options.model | Name of the the OpenAI Chat model to use. You can select between models such as: `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `gpt-3.5-turbo` ... See the https://platform.openai.com/docs/models[models] page for more information. | `gpt-4o`
|
||||
| spring.ai.openai.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.8
|
||||
| spring.ai.openai.chat.options.frequencyPenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | 0.0f
|
||||
| spring.ai.openai.chat.options.logitBias | Modify the likelihood of specified tokens appearing in the completion. | -
|
||||
| spring.ai.openai.chat.options.maxTokens | The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. | -
|
||||
| spring.ai.openai.chat.options.n | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs. | 1
|
||||
| spring.ai.openai.chat.options.presencePenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | -
|
||||
| spring.ai.openai.chat.options.responseFormat | An object specifying the format that the model must output. Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.| -
|
||||
| spring.ai.openai.chat.options.responseFormat.type | Compatible with `GPT-4o`, `GPT-4o mini`, `GPT-4 Turbo` and all `GPT-3.5 Turbo` models newer than `gpt-3.5-turbo-1106`. The `JSON_OBJECT` type enables JSON mode, which guarantees the message the model generates is valid JSON.
|
||||
The `JSON_SCHEMA` type enables link:https://platform.openai.com/docs/guides/structured-outputs[Structured Outputs] which guarantees the model will match your supplied JSON schema. The JSON_SCHEMA type requires setting the `responseFormat.schema` property as well. | -
|
||||
| spring.ai.openai.chat.options.responseFormat.name | Reponse format schema name. Applicable only for `responseFormat.type=JSON_SCHEMA` | custom_response_format_schema
|
||||
| spring.ai.openai.chat.options.responseFormat.schema | Reponse format JSON schema. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
|
||||
| spring.ai.openai.chat.options.responseFormat.strict | Reponse format JSON schema adheranse strictens. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
|
||||
| spring.ai.openai.chat.options.seed | This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. | -
|
||||
| spring.ai.openai.chat.options.stop | Up to 4 sequences where the API will stop generating further tokens. | -
|
||||
| spring.ai.openai.chat.options.topP | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. | -
|
||||
@@ -155,7 +159,7 @@ Read more about xref:api/chat/functions/openai-chat-functions.adoc[OpenAI Functi
|
||||
== Multimodal
|
||||
|
||||
Multimodality refers to a model's ability to simultaneously understand and process information from various sources, including text, images, audio, and other data formats.
|
||||
Presently, the OpenAI `gpt-4-visual-preview` and `gpt-4o` models offers multimodal support.
|
||||
Presently, the OpenAI `gpt-4o` and `gpt-4o-mini` models offers multimodal support.
|
||||
Refer to the link:https://platform.openai.com/docs/guides/vision[Vision] guide for more information.
|
||||
|
||||
The OpenAI link:https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages[User Message API] can incorporate a list of base64-encoded images or image urls with the message.
|
||||
@@ -206,6 +210,108 @@ for carrying. The bowl is placed on a flat surface with a neutral-colored backgr
|
||||
view of the fruit inside.
|
||||
----
|
||||
|
||||
== Structured Output
|
||||
|
||||
In addition to existing, model agnostic, xref::api/structured-output-converter.adoc[Structured Output Converter] utilities,
|
||||
OpenAi provides custom https://platform.openai.com/docs/guides/structured-outputs[Structured Outputs] API that guarantees
|
||||
the model will always generate responses that adhere to your supplied https://json-schema.org/overview/what-is-jsonschema[JSON Schema].
|
||||
|
||||
Spring AI offers flexible options to configure your response-format either manually using the OpenAiChatOptions builder or using application properties.
|
||||
|
||||
=== with chat options builder
|
||||
|
||||
The OpenAiChatOptions builder allow you to set the response format programmatically like this:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
|
||||
var jsonSchema = """
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"explanation": { "type": "string" },
|
||||
"output": { "type": "string" }
|
||||
},
|
||||
"required": ["explanation", "output"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"final_answer": { "type": "string" }
|
||||
},
|
||||
"required": ["steps", "final_answer"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
""";
|
||||
|
||||
Prompt prompt = new Prompt("how can I solve 8x + 7 = -23",
|
||||
OpenAiChatOptions.builder()
|
||||
.withModel(ChatModel.GPT_4_O_MINI)
|
||||
.withResponseFormat(new ResponseFormat(ResponseFormat.Type.JSON_SCHEMA, jsonSchema))
|
||||
.build());
|
||||
|
||||
ChatResponse response = this.openAiChatModel.call(prompt);
|
||||
|
||||
----
|
||||
|
||||
You can combine this with the existing xref::api/structured-output-converter.adoc#_bean_output_converter[BeanOutputConverter] utilities to
|
||||
generate the JSON schema from your domain objects and convert the response into domain instances:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
record MathReasoning(
|
||||
@JsonProperty(required = true, value = "steps") Steps steps,
|
||||
@JsonProperty(required = true, value = "final_answer") String finalAnswer) {
|
||||
|
||||
record Steps(
|
||||
@JsonProperty(required = true, value = "items") Items[] items) {
|
||||
|
||||
record Items(
|
||||
@JsonProperty(required = true, value = "explanation") String explanation,
|
||||
@JsonProperty(required = true, value = "output") String output) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var outputConverter = new BeanOutputConverter<>(MathReasoning.class);
|
||||
|
||||
var jsonSchema = outputConverter.getJsonSchema();
|
||||
|
||||
Prompt prompt = new Prompt("how can I solve 8x + 7 = -23",
|
||||
OpenAiChatOptions.builder()
|
||||
.withModel(ChatModel.GPT_4_O_MINI)
|
||||
.withResponseFormat(new ResponseFormat(ResponseFormat.Type.JSON_SCHEMA, jsonSchema))
|
||||
.build());
|
||||
|
||||
ChatResponse response = this.openAiChatModel.call(prompt);
|
||||
String content = response.getResult().getOutput().getContent();
|
||||
|
||||
MathReasoning mathReasoning = outputConverter.convert(content);
|
||||
----
|
||||
|
||||
NOTE: You must use the `@JsonProperty(required = true,..)` annotation to ensure the generated schema produces the `required[...]` field.
|
||||
Although optional for JSON Schema, OpenAI requires this filed for the structured response to work.
|
||||
|
||||
=== with application properties
|
||||
|
||||
You can use the `spring.ai.openai.chat.options.response-format.*` application properties to configure your desired response format.
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
spring.ai.openai.api-key=YOUR_API_KEY
|
||||
spring.ai.openai.chat.options.model=gpt-4o-mini
|
||||
|
||||
spring.ai.openai.chat.options.response-format.type=JSON_SCHEMA
|
||||
spring.ai.openai.chat.options.response-format.name=MySchemaName
|
||||
spring.ai.openai.chat.options.response-format.schema=`{"type":"object","properties":{"steps":{"type":"array","items":{"type":"object","properties":{"explanation":{"type":"string"},"output":{"type":"string"}},"required":["explanation","output"],"additionalProperties":false}},"final_answer":{"type":"string"}},"required":["steps","final_answer"],"additionalProperties":false}`
|
||||
spring.ai.openai.chat.options.response-format.strict=true
|
||||
|
||||
----
|
||||
|
||||
== Sample Controller
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-openai-spring-boot-starter` to your pom (or gradle) dependencies.
|
||||
|
||||
Reference in New Issue
Block a user