Spring/spring-ai
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs(onnx): Add a note when the onnxruntime_gpu dependency is missing.

Browse Source
This commit is contained in:
dongfeng
2024-08-08 19:02:46 +08:00
committed by Christian Tzolov
parent 66e4b8868f
commit aeba9c26ac
1 changed files with 14 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc
View File

@@ -77,7 +77,7 @@ The complete list of supported properties are:
| spring.ai.embedding.transformer.cache.directory | Directory path to cache remote resources, such as the ONNX models | ${java.io.tmpdir}/spring-ai-onnx-model
| spring.ai.embedding.transformer.onnx.modelUri | Existing, pre-trained ONNX model. | onnx/all-MiniLM-L6-v2/model.onnx
| spring.ai.embedding.transformer.onnx.modelOutputName | The ONNX model's output node name, which we'll use for embedding calculation. | last_hidden_state
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise. | -1
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise.(Requires additional onnxruntime_gpu dependency) | -1
| spring.ai.embedding.transformer.metadataMode | Specifies what parts of the Documents content and metadata will be used for computing the embeddings. | NONE
|===
@@ -114,6 +114,19 @@ The `model.onnx_data` is called link:https://onnx.ai/onnx/repo-docs/ExternalData
Currently the only workaround is to copy the large `model.onnx_data` in the folder you run your Boot applicaiton.
====
[NOTE]
====
If you get an error like `ai.onnxruntime.OrtException: Error code - ORT_EP_FAIL - message: Failed to find CUDA shared provider`,
that means that you are using the GPU parameters `spring.ai.embedding.transformer.onnx.gpuDeviceId` , but the onnxruntime_gpu dependency are missing.
----
<dependency>
<groupId>com.microsoft.onnxruntime</groupId>
<artifactId>onnxruntime_gpu</artifactId>
</dependency>
----
Please select the appropriate onnxruntime_gpu version based on the CUDA version(link:https://onnxruntime.ai/docs/get-started/with-java.html[ONNX Java Runtime]).
====
== Manual Configuration
If you are not using Spring Boot, you can manually configure the Onnx Transformers Embedding Model.