= Ollama Embeddings

With https://ollama.ai/[Ollama] you can run various Large Language Models (LLMs) locally and generate embeddings from them.
Spring AI supports the Ollama text embeddings with `OllamaEmbeddingModel`.

An embedding is a vector (list) of floating point numbers.
The distance between two vectors measures their relatedness.
Small distances suggest high relatedness and large distances suggest low relatedness.

== Prerequisites

You first need to run Ollama on your local machine.

Refer to the official Ollama project link:https://github.com/ollama/ollama[README] to get started running models on your local machine.

NOTE: installing `ollama run llama3` will download a 4.7GB model artifact.

=== Add Repositories and BOM

Spring AI artifacts are published in Spring Milestone and Snapshot repositories.   Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.

To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build system.


== Auto-configuration

Spring AI provides Spring Boot auto-configuration for the Azure Ollama Embedding Client.
To enable it add the following dependency to your Maven `pom.xml` file:

[source,xml]
----
<dependency>
   <groupId>org.springframework.ai</groupId>
   <artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
</dependency>
----

or to your Gradle `build.gradle` build file.

[source,groovy]
----
dependencies {
    implementation 'org.springframework.ai:spring-ai-ollama-spring-boot-starter'
}
----

TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.

The `spring.ai.ollama.embedding.options.*` properties are used to configure the default options used for all embedding requests.
(It is used as `OllamaEmbeddingModel#withDefaultOptions()` instance).

=== Embedding Properties

The prefix `spring.ai.ollama` is the property prefix to configure the connection to Ollama

[cols="3,6,2"]
|====
| Property | Description | Default

| spring.ai.ollama.base-url | Base URL where Ollama API server is running. | `http://localhost:11434`
|====

The prefix `spring.ai.ollama.embedding.options` is the property prefix that configures the `EmbeddingModel` implementation for Ollama.

[cols="3,5,1"]
|====
| Property | Description | Default

| spring.ai.ollama.embedding.enabled      | Enable Ollama embedding model. | true
| spring.ai.ollama.embedding.options.model  | The name of the https://github.com/ollama/ollama?tab=readme-ov-file#model-library[supported model] to use. | mistral
|====

The remaining `options` properties are based on the link:https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values[Ollama Valid Parameters and Values] and link:https://github.com/ollama/ollama/blob/main/api/types.go[Ollama Types]. The default values are based on: link:https://github.com/ollama/ollama/blob/b538dc3858014f94b099730a592751a5454cab0a/api/types.go#L364[Ollama type defaults].

[cols="3,5,1"]
|====
| Property | Description | Default
| spring.ai.ollama.embedding.options.numa              | Whether to use NUMA.                                           | false
| spring.ai.ollama.embedding.options.num-ctx           | Sets the size of the context window used to generate the next token. | 2048
| spring.ai.ollama.embedding.options.num-batch         | ???                                                             | -
| spring.ai.ollama.embedding.options.num-gqa           | The number of GQA groups in the transformer layer. Required for some models, for example, it is 8 for llama2:70b. | -
| spring.ai.ollama.embedding.options.num-gpu           | The number of layers to send to the GPU(s). On macOS it defaults to 1 to enable metal support, 0 to disable. | -
| spring.ai.ollama.embedding.options.main-gpu          | ???                                                             | -
| spring.ai.ollama.embedding.options.low-vram          | ???                                                             | -
| spring.ai.ollama.embedding.options.f16-kv            | ???                                                             | -
| spring.ai.ollama.embedding.options.logits-all        | ???                                                             | -
| spring.ai.ollama.embedding.options.vocab-only        | ???                                                             | -
| spring.ai.ollama.embedding.options.use-mmap          | ???                                                             | -
| spring.ai.ollama.embedding.options.use-mlock         | ???                                                             | -
| spring.ai.ollama.embedding.options.num-thread        | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | -
| spring.ai.ollama.embedding.options.num-keep          | ???                                                             | -
| spring.ai.ollama.embedding.options.seed              | Sets the random number seed to use for generation. Setting this to a specific number will make the model generate the same text for the same prompt.  | 0
| spring.ai.ollama.embedding.options.num-predict       | Maximum number of tokens to predict when generating text. (Default: 128, -1 = infinite generation, -2 = fill context) | 128
| spring.ai.ollama.embedding.options.top-k             | Reduces the probability of generating nonsense. A higher value (e.g., 100) will give more diverse answers, while a lower value (e.g., 10) will be more conservative.  | 40
| spring.ai.ollama.embedding.options.top-p             | Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text.  | 0.9
| spring.ai.ollama.embedding.options.tfs-z             | Tail-free sampling is used to reduce the impact of less probable tokens from the output. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. | 1
| spring.ai.ollama.embedding.options.typical-p         | ???                                                             | -
| spring.ai.ollama.embedding.options.repeat-last-n      | Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx) | 64
| spring.ai.ollama.embedding.options.temperature       | The temperature of the model. Increasing the temperature will make the model answer more creatively. | 0.8
| spring.ai.ollama.embedding.options.repeat-penalty    | Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. | 1.1
| spring.ai.ollama.embedding.options.presence-penalty  | ???                                                             | -
| spring.ai.ollama.embedding.options.frequency-penalty | ???                                                             | -
| spring.ai.ollama.embedding.options.mirostat          | Enable Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0) | 0
| spring.ai.ollama.embedding.options.mirostat-tau      | Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. | 5.0
| spring.ai.ollama.embedding.options.mirostat-eta      | Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. | 0.1
| spring.ai.ollama.embedding.options.penalize-newline  | ???                                                             | -
| spring.ai.ollama.embedding.options.stop              | Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return. Multiple stop patterns may be set by specifying multiple separate stop parameters in a modelfile. | -
|====

TIP: All properties prefixed with `spring.ai.ollama.embedding.options` can be overridden at runtime by adding a request specific <<embedding-options>> to the `EmbeddingRequest` call.

== Runtime Options [[embedding-options]]

The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-ollama/src/main/java/org/springframework/ai/ollama/api/OllamaOptions.java[OllamaOptions.java] provides the Ollama configurations, such as the model to use, the low level GPU and CPU tuning, etc.

The default options can be configured using the `spring.ai.ollama.embedding.options` properties as well.

At start-time use the `OllamaEmbeddingModel#withDefaultOptions()` to configure the  default options used for all embedding requests.
At run-time you can override the default options, using a `OllamaOptions` instance as part of your `EmbeddingRequest`.

For example to override the default model name for a specific request:

[source,java]
----
EmbeddingResponse embeddingResponse = embeddingModel.call(
    new EmbeddingRequest(List.of("Hello World", "World is big and salvation is near"),
        OllamaOptions.create()
            .withModel("Different-Embedding-Model-Deployment-Name"));
----

== Sample Controller

This will create a `EmbeddingModel` implementation that you can inject into your class.
Here is an example of a simple `@Controller` class that uses the `EmbeddingModel` implementation.

[source,java]
----
@RestController
public class EmbeddingController {

    private final EmbeddingModel embeddingModel;

    @Autowired
    public EmbeddingController(EmbeddingModel embeddingModel) {
        this.embeddingModel = embeddingModel;
    }

    @GetMapping("/ai/embedding")
    public Map embed(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        EmbeddingResponse embeddingResponse = this.embeddingModel.embedForResponse(List.of(message));
        return Map.of("embedding", embeddingResponse);
    }
}
----

== Manual Configuration

If you are not using Spring Boot, you can manually configure the `OllamaEmbeddingModel`.
For this add the spring-ai-ollama dependency to your project’s Maven pom.xml file:

[source,xml]
----
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-ollama</artifactId>
</dependency>
----

or to your Gradle `build.gradle` build file.

[source,groovy]
----
dependencies {
    implementation 'org.springframework.ai:spring-ai-ollama'
}
----

TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.

NOTE: The `spring-ai-ollama` dependency provides access also to the `OllamaChatModel`.
For more information about the `OllamaChatModel` refer to the link:../chat/ollama-chat.html[Ollama Chat Client] section.

Next, create an `OllamaEmbeddingModel` instance and use it to compute the similarity between two input texts:

[source,java]
----
var ollamaApi = new OllamaApi();

var embeddingModel = new OllamaEmbeddingModel(ollamaApi)
    .withDefaultOptions(OllamaOptions.create()
			.withModel(OllamaOptions.DEFAULT_MODEL)
            .toMap());

EmbeddingResponse embeddingResponse = embeddingModel
	.embedForResponse(List.of("Hello World", "World is big and salvation is near"));
----

The `OllamaOptions` provides the configuration information for all embedding requests.
