Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add activation/deactivation auto-config property to all Model clients

Browse Source 
Property is of format `*.enabled=` and is true by default.
 Note: the Bedrock clients are disabled by default.

 - Add deactivation support for Vertex AI PaLM2 clients
 - Add deactivation support for Azure OpenAI clients
 - Add deactivation support for huggingface chat client
 - Add deactivation support for Ollama clients
 - Add deactivation support for OpenAI clients
 - Add deactivation support for Stability Image and PostgresML clients
 - Add deactivation support for ONNX Transformers Embedding client

 Resolves #363
This commit is contained in:
Christian Tzolov
2024-02-26 16:29:50 +01:00
parent 288ef449d6
commit 4e270ac75a
42 changed files with 557 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/azure-openai-chat.adoc
View File

@@ -94,6 +94,7 @@ The prefix `spring.ai.azure.openai.chat` is the property prefix that configures
|====
| Property | Description | Default
| spring.ai.azure.openai.chat.enabled | Enable Azure OpenAI chat client. | true
| spring.ai.azure.openai.chat.options.model | * In use with Azure, this actually refers to the "Deployment Name" of your model, which you can find at https://oai.azure.com/portal. It's important to note that within an Azure OpenAI deployment, the "Deployment Name" is distinct from the model itself. The confusion around these terms stems from the intention to make the Azure OpenAI client library compatible with the original OpenAI endpoint. The deployment structures offered by Azure OpenAI and Sam Altman's OpenAI differ significantly. To clarify this distinction, we plan to rename this attribute to `deployment-name` in future updates.
Deployments model name to provide as part of this completions request.
| gpt-35-turbo

5
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/image/openai-image.adoc
View File

@@ -55,8 +55,9 @@ The prefix `spring.ai.openai.image` is the property prefix that lets you configu
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.chat.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.chat.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.image.enabled | Enable OpenAI image client. | true
| spring.ai.openai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.image.options.n | The number of images to generate. Must be between 1 and 10. For dall-e-3, only n=1 is supported. | -
| spring.ai.openai.image.options.model | The model to use for image generation. | OpenAiImageApi.DEFAULT_IMAGE_MODEL
| spring.ai.openai.image.options.quality | The quality of the image that will be generated. HD creates images with finer details and greater consistency across the image. This parameter is only supported for dall-e-3. | -

27
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/image/stabilityai-image.adoc
View File

@@ -57,19 +57,20 @@ The prefix `spring.ai.stabilityai.image` is the property prefix that lets you co
|====
| Property | Description | Default
| `spring.ai.stabilityai.image.base-url` | Optional overrides the spring.ai.openai.base-url to provide a specific url | `https://api.stability.ai/v1`
| `spring.ai.stabilityai.image.api-key` | Optional overrides the spring.ai.openai.api-key to provide a specific api-key | -
| `spring.ai.stabilityai.image.option.n` | The number of images to be generated. Must be between 1 and 10. | 1
| `spring.ai.stabilityai.image.option.model` | The engine/model to use in Stability AI. The model is passed in the URL as a path parameter. | `stable-diffusion-v1-6`
| `spring.ai.stabilityai.image.option.width` | Width of the image to generate, in pixels, in an increment divisible by 64. Engine-specific dimension validation applies. | 512
| `spring.ai.stabilityai.image.option.height` | Height of the image to generate, in pixels, in an increment divisible by 64. Engine-specific dimension validation applies.| 512
| `spring.ai.stabilityai.image.option.responseFormat` | The format in which the generated images are returned. Must be "application/json" or "image/png". | -
| `spring.ai.stabilityai.image.option.cfg_scale` | The strictness level of the diffusion process adherence to the prompt text. Range: 0 to 35. | 7
| `spring.ai.stabilityai.image.option.clip_guidance_preset` | Pass in a style preset to guide the image model towards a particular style. This list of style presets is subject to change. | `NONE`
| `spring.ai.stabilityai.image.option.sampler` | Which sampler to use for the diffusion process. If this value is omitted, an appropriate sampler will be automatically selected. | -
| `spring.ai.stabilityai.image.option.seed` | Random noise seed (omit this option or use 0 for a random seed). Valid range: 0 to 4294967295. | 0
| `spring.ai.stabilityai.image.option.steps` | Number of diffusion steps to run. Valid range: 10 to 50. | 30
| `spring.ai.stabilityai.image.option.style_preset` | Pass in a style preset to guide the image model towards a particular style. This list of style presets is subject to change. | -
| spring.ai.stabilityai.image.enabled | Enable Stability AI image client. | true
| spring.ai.stabilityai.image.base-url | Optional overrides the spring.ai.openai.base-url to provide a specific url | `https://api.stability.ai/v1`
| spring.ai.stabilityai.image.api-key | Optional overrides the spring.ai.openai.api-key to provide a specific api-key | -
| spring.ai.stabilityai.image.option.n | The number of images to be generated. Must be between 1 and 10. | 1
| spring.ai.stabilityai.image.option.model | The engine/model to use in Stability AI. The model is passed in the URL as a path parameter. | `stable-diffusion-v1-6`
| spring.ai.stabilityai.image.option.width | Width of the image to generate, in pixels, in an increment divisible by 64. Engine-specific dimension validation applies. | 512
| spring.ai.stabilityai.image.option.height | Height of the image to generate, in pixels, in an increment divisible by 64. Engine-specific dimension validation applies.| 512
| spring.ai.stabilityai.image.option.responseFormat | The format in which the generated images are returned. Must be "application/json" or "image/png". | -
| spring.ai.stabilityai.image.option.cfg_scale | The strictness level of the diffusion process adherence to the prompt text. Range: 0 to 35. | 7
| spring.ai.stabilityai.image.option.clip_guidance_preset | Pass in a style preset to guide the image model towards a particular style. This list of style presets is subject to change. | `NONE`
| spring.ai.stabilityai.image.option.sampler | Which sampler to use for the diffusion process. If this value is omitted, an appropriate sampler will be automatically selected. | -
| spring.ai.stabilityai.image.option.seed | Random noise seed (omit this option or use 0 for a random seed). Valid range: 0 to 4294967295. | 0
| spring.ai.stabilityai.image.option.steps | Number of diffusion steps to run. Valid range: 10 to 50. | 30
| spring.ai.stabilityai.image.option.style_preset | Pass in a style preset to guide the image model towards a particular style. This list of style presets is subject to change. | -
|====

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/ollama-chat.adoc
View File

@@ -60,6 +60,7 @@ NOTE: The `options` properties are based on the link:https://github.com/jmorganc
|====
| Property | Description | Default
| spring.ai.ollama.chat.enabled | Enable Ollama chat client. | true
| spring.ai.ollama.chat.model (DEPRECATED) | The name of the model to use. Deprecated use the `spring.ai.ollama.chat.options.model` instead | mistral
| spring.ai.ollama.chat.options.model | The name of the https://github.com/ollama/ollama?tab=readme-ov-file#model-library[supported models] to use. | mistral
| spring.ai.ollama.chat.options.numa | Whether to use NUMA. | false

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/openai-chat.adoc
View File

@@ -65,6 +65,7 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
|====
| Property | Description | Default
| spring.ai.openai.chat.enabled | Enable OpenAI chat client. | true
| spring.ai.openai.chat.base-url | Optional overrides the spring.ai.openai.base-url to provide chat specific url | -
| spring.ai.openai.chat.api-key | Optional overrides the spring.ai.openai.api-key to provide chat specific api-key | -
| spring.ai.openai.chat.options.model | This is the OpenAI Chat model to use | `gpt-3.5-turbo` (the `gpt-3.5-turbo`, `gpt-4`, and `gpt-4-32k` point to the latest model versions)

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/vertexai-chat.adoc
View File

@@ -67,6 +67,7 @@ The prefix `spring.ai.vertex.ai.chat` is the property prefix that lets you confi
|====
| Property | Description | Default
| spring.ai.vertex.ai.chat.enabled | Enable Vertex AI PaLM API Chat client. | true
| spring.ai.vertex.ai.chat.model | This is the https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-chat[Vertex Chat model] to use | chat-bison-001
| spring.ai.vertex.ai.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0], inclusive. A value closer to 1.0 will produce responses that are more varied, while a value closer to 0.0 will typically result in less surprising responses from the generative. This value specifies default to be used by the backend while making the call to the generative. | 0.7
| spring.ai.vertex.ai.chat.options.topK | The maximum number of tokens to consider when sampling. The generative uses combined Top-k and nucleus sampling. Top-k sampling considers the set of topK most probable tokens. | 20

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/azure-openai-embeddings.adoc
View File

@@ -72,6 +72,7 @@ The prefix `spring.ai.azure.openai.embeddings` is the property prefix that confi
|====
| Property | Description | Default
| spring.ai.azure.openai.embedding.enabled | Enable Azure OpenAI embedding client. | true
| spring.ai.azure.openai.embedding.metadata-mode | Document content extraction mode | EMBED
| spring.ai.azure.openai.embedding.options.model | This is the value of the 'Deployment Name' as presented in the Azure AI Portal | text-embedding-ada-002
| spring.ai.azure.openai.embedding.options.user | An identifier for the caller or end user of the operation. This may be used for tracking or rate-limiting purposes. | -

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/ollama-embeddings.adoc
View File

@@ -66,6 +66,7 @@ The prefix `spring.ai.ollama.embedding.options` is the property prefix that conf
|====
| Property | Description | Default
| spring.ai.ollama.embedding.enabled | Enable Ollama embedding client. | true
| spring.ai.ollama.embedding.model (DEPRECATED) | The name of the model to use. Deprecated use the `spring.ai.ollama.embedding.options.model` instead | mistral
| spring.ai.ollama.embedding.options.model | The name of the https://github.com/ollama/ollama?tab=readme-ov-file#model-library[supported models] to use. | mistral
| spring.ai.ollama.embedding.options.numa | Whether to use NUMA. | false

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc
View File

@@ -127,6 +127,8 @@ The complete list of supported properties are:
[cols="3*"]
|===
| Property | Description | Default
| spring.ai.embedding.transformer.enabled | Enable the Transformer Embedding client. | true
| spring.ai.embedding.transformer.tokenizer.uri | URI of a pre-trained HuggingFaceTokenizer created by the ONNX engine (e.g. tokenizer.json). | onnx/all-MiniLM-L6-v2/tokenizer.json
| spring.ai.embedding.transformer.tokenizer.options | HuggingFaceTokenizer options such as '`addSpecialTokens`', '`modelMaxLength`', '`truncation`', '`padding`', '`maxLength`', '`stride`', '`padToMultipleOf`'. Leave empty to fallback to the defaults. | empty
| spring.ai.embedding.transformer.cache.enabled | Enable remote Resource caching. | true

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/openai-embeddings.adoc
View File

@@ -65,6 +65,8 @@ The prefix `spring.ai.openai.embedding` is property prefix that configures the `
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.openai.embedding.enabled | Enable OpenAI embedding client. | true
| spring.ai.openai.embedding.base-url | Optional overrides the spring.ai.openai.base-url to provide embedding specific url | -
| spring.ai.openai.embedding.api-key | Optional overrides the spring.ai.openai.api-key to provide embedding specific api-key | -
| spring.ai.openai.embedding.metadata-mode | Document content extraction mode. | EMBED

1
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/postgresml-embeddings.adoc
View File

@@ -49,6 +49,7 @@ The prefix `spring.ai.postgresml.embedding` is property prefix that configures t
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.postgresml.embedding.enabled | Enable PostgresML embedding client. | true
| spring.ai.postgresml.embedding.options.transformer | The Huggingface transformer model to use for the embedding. | distilbert-base-uncased
| spring.ai.postgresml.embedding.options.kwargs | Additional transformer specific options. | empty map
| spring.ai.postgresml.embedding.options.vectorType | PostgresML vector type to use for the embedding. Two options are supported: `PG_ARRAY` and `PG_VECTOR`. | PG_ARRAY

13
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/vertexai-embeddings.adoc
View File

@@ -49,7 +49,7 @@ dependencies {
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
=== Chat Properties
=== Embedding Properties
The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you connect to VertexAI.
@@ -61,13 +61,14 @@ The prefix `spring.ai.vertex.ai` is used as the property prefix that lets you co
| spring.ai.vertex.ai.api-key | The API Key | -
|====
The prefix `spring.ai.vertex.ai.chat` is the property prefix that lets you configure the chat client implementation for VertexAI Chat.
The prefix `spring.ai.vertex.ai.embedding` is the property prefix that lets you configure the embedding client implementation for VertexAI Chat.
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.vertex.ai.chat.model | This is the https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-embeddings[Vertex Embedding model] to use | chat-bison-001
| spring.ai.vertex.ai.embedding.enabled | Enable Vertex AI PaLM API Embedding client. | true
| spring.ai.vertex.ai.embedding.model | This is the https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text-embeddings[Vertex Embedding model] to use | embedding-gecko-001
|====
@@ -80,13 +81,13 @@ Add a `application.properties` file, under the `src/main/resources` directory, t
[source,application.properties]
----
spring.ai.vertex.ai.api-key=YOUR_API_KEY
spring.ai.vertex.ai.embedding.model=chat-bison-001
spring.ai.vertex.ai.embedding.model=embedding-gecko-001
----
TIP: replace the `api-key` with your VertexAI credentials.
This will create a `VertexAiEmbeddingClient` implementation that you can inject into your class.
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
Here is an example of a simple `@Controller` class that uses the embedding client for text generations.
[source,java]
----
@@ -149,7 +150,7 @@ EmbeddingResponse embeddingResponse = embeddingClient
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-vertex-ai/src/main/java/org/springframework/ai/vertex/api/VertexAiApi.java[VertexAiApi] provides is lightweight Java client for VertexAiApi Chat API.
Following class diagram illustrates the `VertexAiApi` chat interfaces and building blocks:
Following class diagram illustrates the `VertexAiApi` embedding interfaces and building blocks:
image::vertex-ai-chat-low-level-api.jpg[w=800,align="center"]