Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Introduce Hugging Face Starter

Browse Source 
* Added Spring Boot Starter for Spring AI Hugging Face
* Updated documentation with instructions using the starter dependency
* Fixed naming inconsistencies in the docs for Hugging Face

Fixes gh-838

Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
This commit is contained in:
Thomas Vitale
2024-06-07 17:50:51 +02:00
committed by Christian Tzolov
parent 44154c5c93
commit 2a592d4e84
13 changed files with 182 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -89,7 +89,7 @@ Spring AI supports many AI models. For an overview see here. Specific models c
* OpenAI
* Azure OpenAI
* Amazon Bedrock (Anthropic, Llama, Cohere, Titan, Jurassic2)
* HuggingFace
* Hugging Face
* Google VertexAI (PaLM2, Gemini)
* Mistral AI
* Stability AI

2
models/spring-ai-huggingface/README.md
View File

@@ -1,2 +1,2 @@
[Huggingface Chat Documentation](https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/api/chat/huggingface.html)
[Hugging Face Chat Documentation](https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/api/chat/huggingface.html)

1
pom.xml
View File

@@ -74,6 +74,7 @@
<module>spring-ai-spring-boot-starters/spring-ai-starter-anthropic</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-azure-openai</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-bedrock-ai</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-huggingface</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-minimax</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-mistral-ai</module>
<module>spring-ai-spring-boot-starters/spring-ai-starter-ollama</module>

6
spring-ai-bom/pom.xml
View File

@@ -260,6 +260,12 @@
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-huggingface-spring-boot-starter</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-milvus-store-spring-boot-starter</artifactId>

2
spring-ai-docs/src/main/antora/modules/ROOT/nav.adoc
View File

@@ -16,7 +16,7 @@
**** xref:api/chat/bedrock/bedrock-cohere.adoc[Cohere]
**** xref:api/chat/bedrock/bedrock-titan.adoc[Titan]
**** xref:api/chat/bedrock/bedrock-jurassic2.adoc[Jurassic2]
*** xref:api/chat/huggingface.adoc[HuggingFace]
*** xref:api/chat/huggingface.adoc[Hugging Face]
*** xref:api/chat/google-vertexai.adoc[Google VertexAI]
**** xref:api/chat/vertexai-palm2-chat.adoc[VertexAI PaLM2 ]
**** xref:api/chat/vertexai-gemini-chat.adoc[VertexAI Gemini]

162
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat/huggingface.adoc
View File

@@ -1,72 +1,134 @@
= HuggingFace Chat
= Hugging Face Chat
HuggingFace Inference Endpoints allow you to deploy and serve machine learning models in the cloud, making them accessible via an API.
== Getting Started
Further details on HuggingFace Inference Endpoints can be found link:https://huggingface.co/docs/inference-endpoints/index[here].
Hugging Face Inference Endpoints allow you to deploy and serve machine learning models in the cloud, making them accessible via an API.
== Prerequisites
Add the `spring-ai-huggingface` dependency:
[source,xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-huggingface</artifactId>
</dependency>
----
You should get your HuggingFace API key and set it as an environment variable
You will need to create an Inference Endpoint on Hugging Face and create an API token to access the endpoint.
Further details can be found link:https://huggingface.co/docs/inference-endpoints/index[here].
The Spring AI project defines a configuration property named `spring.ai.huggingface.chat.api-key` that you should set to the value of the API token obtained from Hugging Face.
There is also a configuration property named `spring.ai.huggingface.chat.url` that you should set to the inference endpoint URL obtained when provisioning your model in Hugging Face.
You can find this on the Inference Endpoint's UI link:https://ui.endpoints.huggingface.co/[here].
Exporting environment variables is one way to set these configuration properties:
[source,shell]
----
export HUGGINGFACE_API_KEY=your_api_key_here
export SPRING_AI_HUGGINGFACE_CHAT_API_KEY=<INSERT KEY HERE>
export SPRING_AI_HUGGINGFACE_CHAT_URL=<INSERT INFERENCE ENDPOINT URL HERE>
----
=== Add Repositories and BOM
Spring AI artifacts are published in Spring Milestone and Snapshot repositories.
Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build system.
== Auto-configuration
Spring AI provides Spring Boot auto-configuration for the Hugging Face Chat Client.
To enable it add the following dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-huggingface-spring-boot-starter</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-huggingface-spring-boot-starter'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
Note, there is not yet a Spring Boot Starter for this chat implementation.
=== Chat Properties
Obtain the endpoint URL of the Inference Endpoint.
You can find this on the Inference Endpoint's UI link:https://ui.endpoints.huggingface.co/[here].
The prefix `spring.ai.huggingface` is the property prefix that lets you configure the chat model implementation for Hugging Face.
== Making a call to the model
[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.huggingface.chat.api-key | API Key to authenticate with the Inference Endpoint. | -
| spring.ai.huggingface.chat.url | URL of the Inference Endpoint to connect to | -
| spring.ai.huggingface.chat.enabled | Enable Hugging Face chat model. | true
|====
== Sample Controller (Auto-configuration)
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-huggingface-spring-boot-starter` to your pom (or gradle) dependencies.
Add an `application.properties` file, under the `src/main/resources` directory, to enable and configure the Hugging Face chat model:
[source,application.properties]
----
spring.ai.huggingface.chat.api-key=YOUR_API_KEY
spring.ai.huggingface.chat.url=YOUR_INFERENCE_ENDPOINT_URL
----
TIP: replace the `api-key` and `url` with your Hugging Face values.
This will create a `HuggingfaceChatModel` implementation that you can inject into your class.
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
[source,java]
----
HuggingfaceChatModel chatModel = new HuggingfaceChatModel(apiKey, basePath);
Prompt prompt = new Prompt("Your text here...");
ChatResponse response = chatModel.call(prompt);
System.out.println(response.getGeneration().getText());
----
@RestController
public class ChatController {
== Example
private final HuggingfaceChatModel chatModel;
Using the example found link:https://www.promptingguide.ai/models/mistral-7b[here]
@Autowired
public ChatController(HuggingfaceChatModel chatModel) {
this.chatModel = chatModel;
}
[source,java]
----
String mistral7bInstruct = """
[INST] You are a helpful code assistant. Your task is to generate a valid JSON object based on the given information:
name: John
lastname: Smith
address: #1 Samuel St.
Just generate the JSON object without explanations:
[/INST]""";
Prompt prompt = new Prompt(mistral7bInstruct);
ChatResponse aiResponse = huggingfaceChatModel.call(prompt);
System.out.println(response.getGeneration().getText());
----
Will produce the output
[source,json]
----
{
"name": "John",
"lastname": "Smith",
"address": "#1 Samuel St."
@GetMapping("/ai/generate")
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", chatModel.call(message));
}
}
----
== Manual Configuration
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-huggingface/src/main/java/org/springframework/ai/huggingface/HuggingfaceChatModel.java[HuggingfaceChatModel] implements the `ChatModel` interface and uses the <<low-level-api>> to connect to the Hugging Face inference endpoints.
Add the `spring-ai-huggingface` dependency to your project's Maven `pom.xml` file:
[source, xml]
----
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-huggingface</artifactId>
</dependency>
----
or to your Gradle `build.gradle` build file.
[source,groovy]
----
dependencies {
implementation 'org.springframework.ai:spring-ai-huggingface'
}
----
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
Next, create a `HuggingfaceChatModel` and use it for text generations:
[source,java]
----
HuggingfaceChatModel chatModel = new HuggingfaceChatModel(apiKey, url);
ChatResponse response = chatModel.call(
new Prompt("Generate the names of 5 famous pirates."));
System.out.println(response.getGeneration().getResult().getOutput().getContent());
----

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chatmodel.adoc
View File

@@ -193,7 +193,7 @@ image::spring-ai-chat-completions-clients.jpg[align="center", width="800px"]
* xref:api/chat/openai-chat.adoc[OpenAI Chat Completion] (streaming & function-calling support)
* xref:api/chat/azure-openai-chat.adoc[Microsoft Azure Open AI Chat Completion] (streaming & function-calling support)
* xref:api/chat/ollama-chat.adoc[Ollama Chat Completion]
* xref:api/chat/huggingface.adoc[HuggingFace Chat Completion] (no streaming support)
* xref:api/chat/huggingface.adoc[Hugging Face Chat Completion] (no streaming support)
* xref:api/chat/vertexai-palm2-chat.adoc[Google Vertex AI PaLM2 Chat Completion] (no streaming support)
* xref:api/chat/vertexai-gemini-chat.adoc[Google Vertex AI Gemini Chat Completion] (streaming, multi-modality & function-calling support)
* xref:api/bedrock.adoc[Amazon Bedrock]

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/postgresml-embeddings.adoc
View File

@@ -50,7 +50,7 @@ The prefix `spring.ai.postgresml.embedding` is property prefix that configures t
|====
| Property | Description | Default
| spring.ai.postgresml.embedding.enabled | Enable PostgresML embedding model. | true
| spring.ai.postgresml.embedding.options.transformer | The Huggingface transformer model to use for the embedding. | distilbert-base-uncased
| spring.ai.postgresml.embedding.options.transformer | The Hugging Face transformer model to use for the embedding. | distilbert-base-uncased
| spring.ai.postgresml.embedding.options.kwargs | Additional transformer specific options. | empty map
| spring.ai.postgresml.embedding.options.vectorType | PostgresML vector type to use for the embedding. Two options are supported: `PG_ARRAY` and `PG_VECTOR`. | PG_ARRAY
| spring.ai.postgresml.embedding.options.metadataMode | Document metadata aggregation mode | EMBED

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/index.adoc
View File

@@ -14,7 +14,7 @@ Dropping down to access model specific features is also supported.
image::model-hierarchy.jpg[Model hierarchy, width=900, align="center"]
With support for AI Models from OpenAI, Microsoft, Amazon, Google, Amazon Bedrock, Huggingface and more.
With support for AI Models from OpenAI, Microsoft, Amazon, Google, Amazon Bedrock, Hugging Face and more.
image::spring-ai-chat-completions-clients.jpg[align="center", width="800px"]

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/getting-started.adoc
View File

@@ -144,7 +144,7 @@ Each of the following sections in the documentation shows which dependencies you
** xref:api/chat/openai-chat.adoc[OpenAI Chat Completion] (streaming and function-calling support)
** xref:api/chat/azure-openai-chat.adoc[Microsoft Azure Open AI Chat Completion] (streaming and function-calling support)
** xref:api/chat/ollama-chat.adoc[Ollama Chat Completion]
** xref:api/chat/huggingface.adoc[HuggingFace Chat Completion] (no streaming support)
** xref:api/chat/huggingface.adoc[Hugging Face Chat Completion] (no streaming support)
** xref:api/chat/vertexai-palm2-chat.adoc[Google Vertex AI PaLM2 Chat Completion] (no streaming support)
** xref:api/chat/vertexai-gemini-chat.adoc[Google Vertex AI Gemini Chat Completion] (streaming, multi-modality & function-calling support)
** xref:api/bedrock.adoc[Amazon Bedrock]

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/index.adoc
View File

@@ -11,7 +11,7 @@ These abstractions have multiple implementations, enabling easy component swappi
Spring AI provides the following features:
* Support for all major Model providers such as OpenAI, Microsoft, Amazon, Google, and Huggingface.
* Support for all major Model providers such as OpenAI, Microsoft, Amazon, Google, and Hugging Face.
* Supported Model types are Chat, Text to Image, Audio Transcription, Text to Speech, and more on the way.
* Portable API across AI providers for all models. Both synchronous and stream API options are supported. Dropping down to access model specific features is also supported.
* Mapping of AI Model output to POJOs.

14
spring-ai-spring-boot-autoconfigure/src/main/java/org/springframework/ai/autoconfigure/huggingface/HuggingfaceChatProperties.java
View File

@@ -17,17 +17,29 @@ package org.springframework.ai.autoconfigure.huggingface;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author Christian Tzolov
* @author Josh Long
* @author Mark Pollack
* @author Thomas Vitale
*/
@ConfigurationProperties(HuggingfaceChatProperties.CONFIG_PREFIX)
public class HuggingfaceChatProperties {
public static final String CONFIG_PREFIX = "spring.ai.huggingface.chat";
/**
* API Key to authenticate with the Inference Endpoint.
*/
private String apiKey;
/**
* URL of the Inference Endpoint.
*/
private String url;
/**
* Enable Huggingface chat model.
* Enable Hugging Face chat model.
*/
private boolean enabled = true;

42
spring-ai-spring-boot-starters/spring-ai-starter-huggingface/pom.xml Normal file
View File

@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai</artifactId>
<version>1.0.0-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<artifactId>spring-ai-huggingface-spring-boot-starter</artifactId>
<packaging>jar</packaging>
<name>Spring AI Starter - Hugging Face</name>
<description>Spring AI Hugging Face Starter</description>
<url>https://github.com/spring-projects/spring-ai</url>
<scm>
<url>https://github.com/spring-projects/spring-ai</url>
<connection>git://github.com/spring-projects/spring-ai.git</connection>
<developerConnection>git@github.com:spring-projects/spring-ai.git</developerConnection>
</scm>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-spring-boot-autoconfigure</artifactId>
<version>${project.parent.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-huggingface</artifactId>
<version>${project.parent.version}</version>
</dependency>
</dependencies>
</project>