From aeba9c26acc0dedd1714d19c408cdfa374b7b9bd Mon Sep 17 00:00:00 2001 From: dongfeng Date: Thu, 8 Aug 2024 19:02:46 +0800 Subject: [PATCH] docs(onnx): Add a note when the onnxruntime_gpu dependency is missing. --- .../modules/ROOT/pages/api/embeddings/onnx.adoc | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc b/spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc index 1949f21b3..67af1a961 100644 --- a/spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc +++ b/spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc @@ -77,7 +77,7 @@ The complete list of supported properties are: | spring.ai.embedding.transformer.cache.directory | Directory path to cache remote resources, such as the ONNX models | ${java.io.tmpdir}/spring-ai-onnx-model | spring.ai.embedding.transformer.onnx.modelUri | Existing, pre-trained ONNX model. | onnx/all-MiniLM-L6-v2/model.onnx | spring.ai.embedding.transformer.onnx.modelOutputName | The ONNX model's output node name, which we'll use for embedding calculation. | last_hidden_state -| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise. | -1 +| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise.(Requires additional onnxruntime_gpu dependency) | -1 | spring.ai.embedding.transformer.metadataMode | Specifies what parts of the Documents content and metadata will be used for computing the embeddings. | NONE |=== @@ -114,6 +114,19 @@ The `model.onnx_data` is called link:https://onnx.ai/onnx/repo-docs/ExternalData Currently the only workaround is to copy the large `model.onnx_data` in the folder you run your Boot applicaiton. ==== +[NOTE] +==== +If you get an error like `ai.onnxruntime.OrtException: Error code - ORT_EP_FAIL - message: Failed to find CUDA shared provider`, +that means that you are using the GPU parameters `spring.ai.embedding.transformer.onnx.gpuDeviceId` , but the onnxruntime_gpu dependency are missing. +---- + + com.microsoft.onnxruntime + onnxruntime_gpu + +---- +Please select the appropriate onnxruntime_gpu version based on the CUDA version(link:https://onnxruntime.ai/docs/get-started/with-java.html[ONNX Java Runtime]). +==== + == Manual Configuration If you are not using Spring Boot, you can manually configure the Onnx Transformers Embedding Model.