migrate remaining moonshot modules to community repo
This commit is contained in:
Mark Pollack
@@ -1,202 +1,5 @@
|
||||
= Function Calling
|
||||
|
||||
You can register custom Java functions with the `MoonshotChatModel` and have the Moonshot model intelligently choose to output a JSON object containing arguments to call one or many of the registered functions.
|
||||
This allows you to connect the LLM capabilities with external tools and APIs.
|
||||
The Moonshot models are trained to detect when a function should be called and to respond with JSON that adheres to the function signature.
|
||||
This functionality has been moved to the Spring AI Community repository.
|
||||
|
||||
The Moonshot API does not call the function directly; instead, the model generates JSON that you can use to call the function in your code and return the result back to the model to complete the conversation.
|
||||
|
||||
Spring AI provides flexible and user-friendly ways to register and call custom functions.
|
||||
In general, the custom functions need to provide a function `name`, `description`, and the function call `signature` (as JSON schema) to let the model know what arguments the function expects. The `description` helps the model to understand when to call the function.
|
||||
|
||||
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and responds with the result back to the model. Your function can in turn invoke other 3rd party services to provide the results.
|
||||
|
||||
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatModel`.
|
||||
|
||||
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
|
||||
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-model/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion Builder utility class to simplify the implementation and registration of Java callback functions.
|
||||
|
||||
// Additionally, the Auto-Configuration provides a way to auto-register any Function<I, O> beans definition as function calling candidates in the `ChatModel`.
|
||||
|
||||
|
||||
== How it works
|
||||
|
||||
Suppose we want the AI model to respond with information that it does not have, for example the current temperature at a given location.
|
||||
|
||||
We can provide the AI model with metadata about our own functions that it can use to retrieve that information as it processes your prompt.
|
||||
|
||||
For example, if during the processing of a prompt, the AI Model determines that it needs additional information about the temperature in a given location, it will start a server side generated request/response interaction. The AI Model invokes a client side function.
|
||||
The AI Model provides method invocation details as JSON and it is the responsibility of the client to execute that function and return the response.
|
||||
|
||||
The model-client interaction is illustrated in the <<spring-ai-function-calling-flow>> diagram.
|
||||
|
||||
Spring AI greatly simplifies code you need to write to support function invocation.
|
||||
It brokers the function invocation conversation for you.
|
||||
You can simply provide your function definition as a `@Bean` and then provide the bean name of the function in your prompt options.
|
||||
You can also reference multiple function bean names in your prompt.
|
||||
|
||||
== Quick Start
|
||||
|
||||
Let's create a chatbot that answer questions by calling our own function.
|
||||
To support the response of the chatbot, we will register our own function that takes a location and returns the current weather in that location.
|
||||
|
||||
When the response to the prompt to the model needs to answer a question such as `"What’s the weather like in Boston?"` the AI model will invoke the client providing the location value as an argument to be passed to the function. This RPC-like data is passed as JSON.
|
||||
|
||||
Our function calls some SaaS based weather service API and returns the weather response back to the model to complete the conversation. In this example we will use a simple implementation named `MockWeatherService` that hard codes the temperature for various locations.
|
||||
|
||||
The following `MockWeatherService.java` represents the weather service API:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
public class MockWeatherService implements Function<Request, Response> {
|
||||
|
||||
public enum Unit { C, F }
|
||||
public record Request(String location, Unit unit) {}
|
||||
public record Response(double temp, Unit unit) {}
|
||||
|
||||
public Response apply(Request request) {
|
||||
return new Response(30.0, Unit.C);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
=== Registering Functions as Beans
|
||||
|
||||
With the link:../minimax-chat.html#_auto_configuration[MoonshotChatModel Auto-Configuration] you have multiple ways to register custom functions as beans in the Spring context.
|
||||
|
||||
We start with describing the most POJO friendly options.
|
||||
|
||||
|
||||
==== Plain Java Functions
|
||||
|
||||
In this approach you define `@Beans` in your application context as you would any other Spring managed object.
|
||||
|
||||
Internally, Spring AI `ChatModel` will create an instance of a `FunctionCallback` instance that adds the logic for it being invoked via the AI model.
|
||||
The name of the `@Bean` is passed as a `ChatOption`.
|
||||
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Configuration
|
||||
static class Config {
|
||||
|
||||
@Bean
|
||||
@Description("Get the weather in location") // function description
|
||||
public Function<MockWeatherService.Request, MockWeatherService.Response> weatherFunction1() {
|
||||
return new MockWeatherService();
|
||||
}
|
||||
...
|
||||
}
|
||||
----
|
||||
|
||||
The `@Description` annotation is optional and provides a function description (2) that helps the model to understand when to call the function. It is an important property to set to help the AI model determine what client side function to invoke.
|
||||
|
||||
Another option to provide the description of the function is to the `@JacksonDescription` annotation on the `MockWeatherService.Request` to provide the function description:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
|
||||
@Configuration
|
||||
static class Config {
|
||||
|
||||
@Bean
|
||||
public Function<Request, Response> currentWeather3() { // (1) bean name as function name.
|
||||
return new MockWeatherService();
|
||||
}
|
||||
...
|
||||
}
|
||||
|
||||
@JsonClassDescription("Get the weather in location") // (2) function description
|
||||
public record Request(String location, Unit unit) {}
|
||||
----
|
||||
|
||||
It is a best practice to annotate the request object with information such that the generates JSON schema of that function is as descriptive as possible to help the AI model pick the correct function to invoke.
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/auto-configurations/models/spring-ai-autoconfigure-model-moonshot/src/test/java/org/springframework/ai/model/moonshot/autoconfigure/tool/FunctionCallbackWithPlainFunctionBeanIT.java[FunctionCallbackWithPlainFunctionBeanIT.java] demonstrates this approach.
|
||||
|
||||
|
||||
==== FunctionCallback Wrapper
|
||||
|
||||
Another way register a function is to create `FunctionCallback` instance like this:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@Configuration
|
||||
static class Config {
|
||||
|
||||
@Bean
|
||||
public FunctionCallback weatherFunctionInfo() {
|
||||
|
||||
return FunctionCallback.builder()
|
||||
.function("CurrentWeather", new MockWeatherService()) // (1) function name and instance
|
||||
.description("Get the weather in location") // (2) function description
|
||||
.inputType(MockWeatherService.Request.class) // (3) function signature
|
||||
.build();
|
||||
}
|
||||
...
|
||||
}
|
||||
----
|
||||
|
||||
It wraps the 3rd party, `MockWeatherService` function and registers it as a `CurrentWeather` function with the `MoonshotChatModel`.
|
||||
It also provides a description (2) and the function signature (3) to let the model know what arguments the function expects.
|
||||
|
||||
NOTE: By default, the response converter does a JSON serialization of the Response object.
|
||||
|
||||
NOTE: The `FunctionCallback` internally resolves the function call signature based on the `MockWeatherService.Request` class.
|
||||
|
||||
=== Specifying functions in Chat Options
|
||||
|
||||
To let the model know and call your `CurrentWeather` function you need to enable it in your prompt requests:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MoonshotChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
ChatResponse response = this.chatModel.call(new Prompt(List.of(this.userMessage),
|
||||
MoonshotChatOptions.builder().function("CurrentWeather").build())); // (1) Enable the function
|
||||
|
||||
logger.info("Response: {}", response);
|
||||
----
|
||||
|
||||
// NOTE: You can can have multiple functions registered in your `ChatModel` but only those enabled in the prompt request will be considered for the function calling.
|
||||
|
||||
Above user question will trigger 3 calls to `CurrentWeather` function (one for each city) and the final response will be something like this:
|
||||
|
||||
----
|
||||
Here is the current weather for the requested cities:
|
||||
- San Francisco, CA: 30.0°C
|
||||
- Tokyo, Japan: 10.0°C
|
||||
- Paris, France: 15.0°C
|
||||
----
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/auto-configurations/models/spring-ai-autoconfigure-model-moonshot/src/test/java/org/springframework/ai/model/moonshot/autoconfigure/tool/MoonshotFunctionCallbackIT.java[MoonshotFunctionCallbackIT.java] test demo this approach.
|
||||
|
||||
|
||||
=== Register/Call Functions with Prompt Options
|
||||
|
||||
In addition to the auto-configuration you can register callback functions, dynamically, with your Prompt requests:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MoonshotChatModel chatModel = ...
|
||||
|
||||
UserMessage userMessage = new UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?");
|
||||
|
||||
var promptOptions = MoonshotChatOptions.builder()
|
||||
.functionCallbacks(List.of(FunctionCallback.builder()
|
||||
.function("CurrentWeather", new MockWeatherService()) // (1) function name
|
||||
.description("Get the weather in location") // (2) function description
|
||||
.inputType(MockWeatherService.Request.class) // (3) function signature
|
||||
.build())) // function code
|
||||
.build();
|
||||
|
||||
ChatResponse response = this.chatModel.call(new Prompt(List.of(this.userMessage), this.promptOptions));
|
||||
----
|
||||
|
||||
NOTE: The in-prompt registered functions are enabled by default for the duration of this request.
|
||||
|
||||
This approach allows to dynamically chose different functions to be called based on the user input.
|
||||
|
||||
The https://github.com/spring-projects/spring-ai/blob/main/auto-configurations/models/spring-ai-autoconfigure-model-moonshot/src/test/java/org/springframework/ai/model/moonshot/autoconfigure/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `MoonshotChatModel` and use it in a prompt request.
|
||||
Please visit https://github.com/spring-ai-community/moonshot for the latest version.
|
||||
@@ -1,268 +1,5 @@
|
||||
= Moonshot AI Chat
|
||||
|
||||
Spring AI supports the various AI language models from Moonshot AI. You can interact with Moonshot AI language models and create a multilingual conversational assistant based on Moonshot models.
|
||||
This functionality has been moved to the Spring AI Community repository.
|
||||
|
||||
== Prerequisites
|
||||
|
||||
You will need to create an API with Moonshot to access Moonshot AI language models.
|
||||
Create an account at https://platform.moonshot.cn/console[Moonshot AI registration page] and generate the token on the https://platform.moonshot.cn/console/api-keys/[API Keys page].
|
||||
The Spring AI project defines a configuration property named `spring.ai.moonshot.api-key` that you should set to the value of the `API Key` obtained from https://platform.moonshot.cn/console/api-keys/[API Keys page].
|
||||
Exporting an environment variable is one way to set that configuration property:
|
||||
|
||||
[source,shell]
|
||||
----
|
||||
export SPRING_AI_MOONSHOT_API_KEY=<INSERT KEY HERE>
|
||||
----
|
||||
|
||||
=== Add Repositories and BOM
|
||||
|
||||
Spring AI artifacts are published in Maven Central and Spring Snapshot repositories.
|
||||
Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
|
||||
|
||||
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build system.
|
||||
|
||||
|
||||
|
||||
== Auto-configuration
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
There has been a significant change in the Spring AI auto-configuration, starter modules' artifact names.
|
||||
Please refer to the https://docs.spring.io/spring-ai/reference/upgrade-notes.html[upgrade notes] for more information.
|
||||
====
|
||||
|
||||
Spring AI provides Spring Boot auto-configuration for the Moonshot Chat Model.
|
||||
To enable it add the following dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source, xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-starter-model-moonshot</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-starter-model-moonshot'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
=== Chat Properties
|
||||
|
||||
==== Retry Properties
|
||||
|
||||
The prefix `spring.ai.retry` is used as the property prefix that lets you configure the retry mechanism for the Moonshot AI Chat model.
|
||||
|
||||
[cols="3,5,1", stripes=even]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.retry.max-attempts | Maximum number of retry attempts. | 10
|
||||
| spring.ai.retry.backoff.initial-interval | Initial sleep duration for the exponential backoff policy. | 2 sec.
|
||||
| spring.ai.retry.backoff.multiplier | Backoff interval multiplier. | 5
|
||||
| spring.ai.retry.backoff.max-interval | Maximum backoff duration. | 3 min.
|
||||
| spring.ai.retry.on-client-errors | If false, throw a NonTransientAiException, and do not attempt retry for `4xx` client error codes | false
|
||||
| spring.ai.retry.exclude-on-http-codes | List of HTTP status codes that should not trigger a retry (e.g. to throw NonTransientAiException). | empty
|
||||
| spring.ai.retry.on-http-codes | List of HTTP status codes that should trigger a retry (e.g. to throw TransientAiException). | empty
|
||||
|====
|
||||
|
||||
==== Connection Properties
|
||||
|
||||
The prefix `spring.ai.moonshot` is used as the property prefix that lets you connect to Moonshot.
|
||||
|
||||
[cols="3,5,1", stripes=even]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.moonshot.base-url | The URL to connect to | https://api.moonshot.cn
|
||||
| spring.ai.moonshot.api-key | The API Key | -
|
||||
|====
|
||||
|
||||
==== Configuration Properties
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Enabling and disabling of the chat auto-configurations are now configured via top level properties with the prefix `spring.ai.model.chat`.
|
||||
|
||||
To enable, spring.ai.model.chat=moonshot (It is enabled by default)
|
||||
|
||||
To disable, spring.ai.model.chat=none (or any value which doesn't match moonshot)
|
||||
|
||||
This change is done to allow configuration of multiple models.
|
||||
====
|
||||
|
||||
The prefix `spring.ai.moonshot.chat` is the property prefix that lets you configure the chat model implementation for Moonshot.
|
||||
|
||||
[cols="3,5,1", stripes=even]
|
||||
|====
|
||||
| Property | Description | Default
|
||||
|
||||
| spring.ai.moonshot.chat.enabled (Removed and no longer valid) | Enable Moonshot chat model. | true
|
||||
| spring.ai.model.chat | Enable Moonshot chat model. | moonshot
|
||||
| spring.ai.moonshot.chat.base-url | Optional overrides the spring.ai.moonshot.base-url to provide chat specific url | -
|
||||
| spring.ai.moonshot.chat.api-key | Optional overrides the spring.ai.moonshot.api-key to provide chat specific api-key | -
|
||||
| spring.ai.moonshot.chat.options.model | This is the Moonshot Chat model to use | `moonshot-v1-8k` (the `moonshot-v1-8k`, `moonshot-v1-32k`, and `moonshot-v1-128k` point to the latest model versions)
|
||||
| spring.ai.moonshot.chat.options.maxTokens | The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. | -
|
||||
| spring.ai.moonshot.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.7
|
||||
| spring.ai.moonshot.chat.options.topP | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. | 1.0
|
||||
| spring.ai.moonshot.chat.options.n | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Default value is 1 and cannot be greater than 5. Specifically, when the temperature is very small and close to 0, we can only return 1 result. If n is already set and>1 at this time, service will return an illegal input parameter (invalid_request_error) | 1
|
||||
| spring.ai.moonshot.chat.options.presencePenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | 0.0f
|
||||
| spring.ai.moonshot.chat.options.frequencyPenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | 0.0f
|
||||
| spring.ai.moonshot.chat.options.stop | Up to 5 sequences where the API will stop generating further tokens. Each string must not exceed 32 bytes | -
|
||||
|====
|
||||
|
||||
NOTE: You can override the common `spring.ai.moonshot.base-url` and `spring.ai.moonshot.api-key` for the `ChatModel` implementations.
|
||||
The `spring.ai.moonshot.chat.base-url` and `spring.ai.moonshot.chat.api-key` properties if set take precedence over the common properties.
|
||||
This is useful if you want to use different Moonshot accounts for different models and different model endpoints.
|
||||
|
||||
TIP: All properties prefixed with `spring.ai.moonshot.chat.options` can be overridden at runtime by adding a request specific <<chat-options>> to the `Prompt` call.
|
||||
|
||||
== Runtime Options [[chat-options]]
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/main/java/org/springframework/ai/moonshot/MoonshotChatOptions.java[MoonshotChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
|
||||
|
||||
On start-up, the default options can be configured with the `MoonshotChatModel(api, options)` constructor or the `spring.ai.moonshot.chat.options.*` properties.
|
||||
|
||||
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
|
||||
For example to override the default model and temperature for a specific request:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
ChatResponse response = chatModel.call(
|
||||
new Prompt(
|
||||
"Generate the names of 5 famous pirates.",
|
||||
MoonshotChatOptions.builder()
|
||||
.model(MoonshotApi.ChatModel.MOONSHOT_V1_8K.getValue())
|
||||
.temperature(0.5)
|
||||
.build()
|
||||
));
|
||||
----
|
||||
|
||||
TIP: In addition to the model specific link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/main/java/org/springframework/ai/moonshot/MoonshotChatOptions.java[MoonshotChatOptions] you can use a portable https://github.com/spring-projects/spring-ai/blob/main/spring-ai-model/src/main/java/org/springframework/ai/chat/ChatOptions.java[ChatOptions] instance, created with the https://github.com/spring-projects/spring-ai/blob/main/spring-ai-chat-client/src/main/java/org/springframework/ai/chat/ChatOptionsBuilder.java[ChatOptionsBuilder#builder()].
|
||||
|
||||
== Sample Controller (Auto-configuration)
|
||||
|
||||
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-starter-model-moonshot` to your pom (or gradle) dependencies.
|
||||
|
||||
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the Moonshot Chat model:
|
||||
|
||||
[source,application.properties]
|
||||
----
|
||||
spring.ai.moonshot.api-key=YOUR_API_KEY
|
||||
spring.ai.moonshot.chat.options.model=moonshot-v1-8k
|
||||
spring.ai.moonshot.chat.options.temperature=0.7
|
||||
----
|
||||
|
||||
TIP: replace the `api-key` with your Moonshot credentials.
|
||||
|
||||
This will create a `MoonshotChatModel` implementation that you can inject into your class.
|
||||
Here is an example of a simple `@Controller` class that uses the chat model for text generations.
|
||||
|
||||
[source,java]
|
||||
----
|
||||
@RestController
|
||||
public class ChatController {
|
||||
|
||||
private final MoonshotChatModel chatModel;
|
||||
|
||||
@Autowired
|
||||
public ChatController(MoonshotChatModel chatModel) {
|
||||
this.chatModel = chatModel;
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generate")
|
||||
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
return Map.of("generation", this.chatModel.call(message));
|
||||
}
|
||||
|
||||
@GetMapping("/ai/generateStream")
|
||||
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
|
||||
var prompt = new Prompt(new UserMessage(message));
|
||||
return this.chatModel.stream(prompt);
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
== Manual Configuration
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/main/java/org/springframework/ai/moonshot/MoonshotChatModel.java[MoonshotChatModel] implements the `ChatModel` and `StreamingChatModel` and uses the <<low-level-api>> to connect to the Moonshot service.
|
||||
|
||||
Add the `spring-ai-moonshot` dependency to your project's Maven `pom.xml` file:
|
||||
|
||||
[source, xml]
|
||||
----
|
||||
<dependency>
|
||||
<groupId>org.springframework.ai</groupId>
|
||||
<artifactId>spring-ai-moonshot</artifactId>
|
||||
</dependency>
|
||||
----
|
||||
|
||||
or to your Gradle `build.gradle` build file.
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
dependencies {
|
||||
implementation 'org.springframework.ai:spring-ai-moonshot'
|
||||
}
|
||||
----
|
||||
|
||||
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
|
||||
|
||||
Next, create a `MoonshotChatModel` and use it for text generations:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
var moonshotApi = new MoonshotApi(System.getenv("MOONSHOT_API_KEY"));
|
||||
|
||||
var chatModel = new MoonshotChatModel(this.moonshotApi, MoonshotChatOptions.builder()
|
||||
.model(MoonshotApi.ChatModel.MOONSHOT_V1_8K.getValue())
|
||||
.temperature(0.4)
|
||||
.maxTokens(200)
|
||||
.build());
|
||||
|
||||
ChatResponse response = this.chatModel.call(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
|
||||
// Or with streaming responses
|
||||
Flux<ChatResponse> streamResponse = this.chatModel.stream(
|
||||
new Prompt("Generate the names of 5 famous pirates."));
|
||||
----
|
||||
|
||||
The `MoonshotChatOptions` provides the configuration information for the chat requests.
|
||||
The `MoonshotChatOptions.Builder` is fluent options builder.
|
||||
|
||||
=== Low-level Moonshot Api Client [[low-level-api]]
|
||||
|
||||
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/main/java/org/springframework/ai/moonshot/api/MoonshotApi.java[MoonshotApi] provides is lightweight Java client for link:https://platform.moonshot.cn/docs/api-reference[Moonshot AI API].
|
||||
|
||||
Here is a simple snippet how to use the api programmatically:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
MoonshotApi moonshotApi =
|
||||
new MoonshotApi(System.getenv("MOONSHOT_API_KEY"));
|
||||
|
||||
ChatCompletionMessage chatCompletionMessage =
|
||||
new ChatCompletionMessage("Hello world", Role.USER);
|
||||
|
||||
// Sync request
|
||||
ResponseEntity<ChatCompletion> response = this.moonshotApi.chatCompletionEntity(
|
||||
new ChatCompletionRequest(List.of(this.chatCompletionMessage), MoonshotApi.ChatModel.MOONSHOT_V1_8K.getValue(), 0.7, false));
|
||||
|
||||
// Streaming request
|
||||
Flux<ChatCompletionChunk> streamResponse = this.moonshotApi.chatCompletionStream(
|
||||
new ChatCompletionRequest(List.of(this.chatCompletionMessage), MoonshotApi.ChatModel.MOONSHOT_V1_8K.getValue(), 0.7, true));
|
||||
----
|
||||
|
||||
Follow the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/main/java/org/springframework/ai/moonshot/api/MoonshotApi.java[MoonshotApi.java]'s JavaDoc for further information.
|
||||
|
||||
==== MoonshotApi Samples
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/test/java/org/springframework/ai/moonshot/api/MoonshotApiIT.java[MoonshotApiIT.java] test provides some general examples how to use the lightweight library.
|
||||
|
||||
* The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-moonshot/src/test/java/org/springframework/ai/moonshot/api/MoonshotApiToolFunctionCallIT.java[MoonshotApiToolFunctionCallIT.java] test shows how to use the low-level API to call tool functions.
|
||||
Please visit https://github.com/spring-ai-community/moonshot for the latest version.
|
||||
Reference in New Issue
Block a user