Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add function-calling as a concept to the docs

Browse Source
This commit is contained in:
Christian Tzolov
2024-02-23 08:56:12 +01:00
parent c8dea142b5
commit b2d04a6e1e
3 changed files with 21 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/functions/openai-chat-functions.adoc
View File

@@ -40,7 +40,7 @@ You can also reference multiple function bean names in your prompt.
Let's create a chatbot that answer questions by calling our own function.
To support the response of the chatbot, we will register our own function that takes a location and returns the current weather in that location.
When the reponse to the prompt to the model needs to answer a question such as `"What’s the weather like in Boston?"` the AI model will invoke the client providing the location value as an argument to be passed to the function. This RPC-like data is passed as JSON.
When the response to the prompt to the model needs to answer a question such as `"What’s the weather like in Boston?"` the AI model will invoke the client providing the location value as an argument to be passed to the function. This RPC-like data is passed as JSON.
Our function can some SaaS based weather service API and returns the weather response back to the model to complete the conversation. In this example we will use a simple implementation named `MockWeatherService` that hard codes the temperature for various locations.

2
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/functions.adoc
View File

@@ -1,5 +1,5 @@
[[Function]]
= Function API
= Function Calling API
The integration of function support in AI models, such as ChatGPT, permits the model to request the execution of client-side functions, thereby accessing necessary information or performing tasks dynamically as required.

23
spring-ai-docs/src/main/antora/modules/ROOT/pages/concepts.adoc
View File

@@ -148,16 +148,18 @@ Note that the GPT 3.5/4.0 dataset extends only until September 2021.
Consequently, the model says that it does not know the answer to questions that require knowledge beyond that date.
An interesting bit of trivia is that this dataset is around 650GB.
Two techniques exist for customizing the AI model to incorporate your data:
Three techniques exist for customizing the AI model to incorporate your data:
* Fine Tuning: This traditional machine learning technique involves tailoring the model and changing its internal weighting.
* `Fine Tuning`: This traditional machine learning technique involves tailoring the model and changing its internal weighting.
However, it is a challenging process for machine learning experts and extremely resource-intensive for models like GPT due to their size. Additionally, some models might not offer this option.
* Prompt Stuffing: A more practical alternative involves embedding your data within the prompt provided to the model. Given a model's token limits, techniques are required to present relevant data within the model's context window.
* `Prompt Stuffing`: A more practical alternative involves embedding your data within the prompt provided to the model. Given a model's token limits, techniques are required to present relevant data within the model's context window.
This approach is colloquially referred to as "`stuffing the prompt.`"
The Spring AI library helps you implement solutions based on the "`stuffing the prompt`" technique otherwise known as Retrieval Augmented Generation (RAG).
* `Function Calling`: This technique allows registering custom, user functions that connect the large language models to the APIs of external systems.
Spring AI greatly simplifies code you need to write to support xref:api/functions.adoc[function calling].
[[concept-rag]]
== Retrieval Augmented Generation
@@ -188,6 +190,19 @@ The concepts map onto classes in Spring AI:
* `DocumentWriter`: Lets you persist the Documents into a database (most commonly in the AI stack, a vector database).
* `Embedding`: A representation of your data as a `List<Double>` that is used by the vector database to compute the "`similarity`" of a user's query to relevant documents.
== Function Calling
Large Language Models (LLMs) are frozen after training, leading to stale knowledge and they are unable to access or modify external data.
The `Function Calling` mechanism addresses these shortcomings.
It allows you register custom, user, functions that connect the large language models to the APIs of external systems.
These systems can provide LLMs with real-time data and perform data processing actions on their behalf.
Spring AI greatly simplifies code you need to write to support function invocation.
It brokers the function invocation conversation for you.
You can provide your function as a `@Bean` and then provide the bean name of the function in your prompt options to activate that function.
You can also define and reference multiple functions in a single prompt.
== Evaluating AI responses
Effectively evaluating the output of an AI system in response to user requests is very important to ensuring the accuracy and usefulness of the final application.