Commit Graph

69 Commits

Author SHA1 Message Date
Mark Pollack
e1884d1d92 Next development version 2024-08-23 18:47:37 -04:00
Mark Pollack
43ad2bdb97 Release version 1.0.0-M2 2024-08-23 18:46:58 -04:00
Christian Tzolov
93fa2bf45a Add observability support to existing vector stores
Add observability support to:
 - Cassandra
 - Chroma
 - Elasticsearch
 - Milvus
 - Neo4j
 - OpenSearch
 - Qdrant
 - Redis
 - Typesense
 - Weaviate
 - Pinecone
 - Oracle
 - Gemifire
 - MongoDB
 - HanaDB

 Add autoconfiguration obsrvability for the above vector stores.
 Add integration tests for all vector stores.
2024-08-20 01:37:45 -04:00
Thomas Vitale
3fa102e78f Prompt content and completion as span events
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
2024-08-20 00:59:43 -04:00
Eddú Meléndez
0a07f65d6a Use RedisAutoConfiguration in RedisVectorStoreAutoConfiguration
Currently, `RedisVectorStoreAutoConfiguration` creates its own
configuration to connect with Redis. This commit reuse
`RedisAutoConfiguration` from spring boot project. It's limited
to Jedis.
2024-07-18 14:24:10 +02:00
Thomas Vitale
7209ee5c4a Fix leaked dependencies in autoconfigure module (#1023)
The spring-ai-spring-boot-autoconfigure module had a few dependencies
that were included in the final JAR instead of being marked as optional.
Also, the PostgreSQL dependency was duplicated.

Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
2024-07-10 15:38:39 -04:00
Christian Tzolov
c70c20b79d Add VertexAI Embedding Model support
- add new spring-ai-vertex-ai-embedding project.
 - add VertexAiTextEmbeddingModel and VertexAiMultimodalEmbeddingMode with related options configuration classes.
 - add ITs
 - add auto-configuraiton and boot starters.
 - register to BOM.
 - add documentation.
 - add multimodal embedding documentation
 - extend the Embedding metdata so that it can keep references to the source document's data, Id, mediatype

 Resolves #1013
 Related to #1009
2024-07-10 15:36:13 -04:00
GR
997e01ca6b Add support for QianFan AI models
- Add chat, embeddin and image models
- Tests
- Docs
2024-06-22 13:30:38 -04:00
Eddú Meléndez
d276d17c1f Add AWS OpenSearch AutoConfiguration
Currently, in order to use an OpenSearch instance provided by AWS,
 additional steps are needed. This commit introduces the required
 configuration.

 Add new starter and update docs
2024-06-21 22:57:06 +02:00
GR
d5b8123e60 Add support for Moonshot AI model
- Docs
- Tests
2024-06-21 15:23:54 -04:00
geetrawat
067a33dbe2 Improved GemFire support
- Adds spring boot auto-configuration support for GemFireVectorStore
- Adds integration test GemFireVectorStoreAutoConfigurationIT
- Includes gemfire-testcontainers in integration tests
- Adds unit test GemFireVectorStorePropertiesTests
- Refactors GemFireVectorStore.java extracting GemFireVectorStoreConfig.java
- Renames spring-ai-gemfire to spring-ai-gemfire-store
- Adds GemFireConnectionDetails
- Adds GemFireVectorStoreProperties with default values
- Remove gemfire-release-repo maven repository

Co-authored-by: Louis Jacome <louis.jacome@broadcom.com>
Co-authored-by: Jason Huyn <jason.huynh@broadcom.com>
2024-06-20 17:17:13 -04:00
Laura Trotta
aca3720f5b ElasticSearch: remove redundant dependency and add note to docs. 2024-06-17 20:56:57 +02:00
Laura Trotta
78f3797f8d Update Elasticsearch Store: Use KNN insettad of script_score
- knn instead of script_score, removed initialization
 - only using normalized similarities, adjusted unit test
 - making l2norm's distances consistent with others
 - update dependency version and docs
 - upate autoconfigure ITs
2024-06-17 15:17:48 +02:00
Christian Tzolov
958549ecce Rename all Embedding Client doc and variables occurrences into Embedding Model 2024-06-16 21:24:03 +02:00
Jemin Huh
46e47849ce Add OpenSearch vector store integration
- implement OpensSearchVectorStore
 - add opensearch auto-configuration and boot starter
 - add documentation for OpenSearch VectorStore
 - add bom dependecies
 - align with to new Spirng AI API
2024-06-16 12:43:09 +02:00
Eddú Meléndez
80a5f2f5b7 Remove property usage 2024-06-16 11:33:54 +02:00
Eddú Meléndez
879f992394 Use Testcontainers Vector DBs modules in autoconfiguartion module 2024-06-15 21:50:35 +02:00
PabloSanchi
3e2ed8b9bf Add Typesense vector store integration
- autoconfigure setup
 - add post bean initialization and create method
 - add embedding field
 - create collection add nested field options
 - add typesense tests
 - use embedding variable instead of word vec
 - check in runtime the number of documents in the collection
 - add typesense expression converter
 - add filter tests. add update document test and search with threshold test
 - distance threshold and add distance key into metadata
 - add typesesne boot starter
 - add typesense docs
 - add client properties in autoconfigure
 - add embedding dimension method
 - add typesense vector store autoconfiguration tests
 - add docs to nav.adoc and vectorsdb.adoc.
 - fix module name.
 - move the expression converter to the typesense project.
2024-06-15 19:55:05 +02:00
Christian Tzolov
baadf4cb53 Fix Oracle Vector Store dependencies
- force override the outdated SpringBoot Oracle version.
 - add auto-config tests.
2024-06-15 12:21:56 +02:00
LLEFEVRE
41cf693bc0 Add Support for Oracle 23ai as vector database
- add OracleVectorStore with metadata filter expression support.
 - add ITs using oracle-free-slim testcontainers.
 - add auto-configuration and boot starter.
 - add adoc documentation.
 - Adjust javadoc references.

 Resolves #703

 Co-authored-by: Eddú Meléndez Gonzales <eddu.melendez@gmail.com>
2024-06-14 06:46:01 +02:00
Mark Pollack
ac91302eed Next development version 2024-05-28 13:53:04 -04:00
Mark Pollack
0670575f3e Release version 1.0.0-M1 2024-05-28 13:49:11 -04:00
GR
99c3857788 Add Support for ZhiPu AI model
* See https://www.zhipuai.cn/
2024-05-20 20:20:29 -04:00
GR
6b674014ed Add support for the MiniMax Model
* See https://minimaxi.com/
2024-05-20 15:08:20 -04:00
Christian Tzolov
3475f17e98 Unify the vector store module and pom names
spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-cassandra -> spring-ai-cassandra-store
 spring-ai-pinecone -> spring-ai-pinecone-store
 spring-ai-redis -> spring-ai-redis-store
 spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-gemfire -> spring-ai-gemfire-store
 spring-ai-azure-vector-store-spring-boot-starter -> spring-ai-azure-store-spring-boot-starter
 spring-ai-redis-spring-boot-starter -> spring-ai-redis-store-spring-boot-starter
2024-05-17 17:05:09 +02:00
Christian Tzolov
3ee04ed8da fix code style and missing dependency renaming 2024-05-17 15:31:58 +02:00
Thomas Vitale
f91ccf0047 Ollama: Update APIs, Testcontainers, Documentation
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
2024-05-05 17:25:51 +03:00
mck
656d238285 Implement Apache Cassandra vector store
The CassandraVectorStore is for managing and querying vector data in an Apache Cassandra db.
  It offers functionalities like adding, deleting, and performing similarity searches on documents.

  The store utilizes CQL to index and search vector data. It allows for custom metadata fields in
  the documents to be stored alongside the vector and content data.

  This class requires a CassandraVectorStoreConfig configuration object for initialization, which
  includes settings like connection details, index name, field names, etc. It also requires an
  EmbeddingClient to convert documents into embeddings before storing them.

  A schema matching the configuration is automatically created if it doesn't exist. Missing columns
  and indexes in existing tables will also be automatically created. Disable this with the disallowSchemaCreation.

  This class is designed to work with brand new tables that it creates for you, or on top of existing
  Cassandra tables. The latter is appropriate when wanting to keep data in place, creating embeddings
  next to it, and performing vector similarity searches in-situ.

  Instances of this class are not dynamic against server-side schema changes. If you change the schema
  server-side you need a new CassandraVectorStore instance.

 - Add auto-configure with tests.
 - reformat code style
 - Change field terminology to column (as appropriate for cassandra and cql)
 - Add doc page with an advanced example.
 - Add the dependencies to Spring AI BOM
 – add to `AutoConfiguration.imports`

 - Add @since annotation
 - Fix javadoc issue
 - Streamline the adoc content and layout
2024-04-10 16:51:47 +02:00
Rahul
466b824840 Add SAP HanaDB vector store integration
- Implement a HanaCloudVectorStore and tests
 - Implement Autoconfiguraiton + properties
 - Add boot starter
 - Update BOM with vector store and boot dependencies.
 - Add antora docuementation
 - added junit for HanaCloudVectorStoreProperties.java and documentation
   to create a BTP trial account and provision an instance for SAP Hana Cloud db
 - updated license, formatting and javadoc
 - IT for HanaCloudVectorStoreAutoConfiguration
 - IT for HanaCloudVectorStoreAutoConfiguration

Additional
 - add @AutoConfiguration(after = { JpaRepositoriesAutoConfiguration.class })
 - update the handa docs structure.
2024-04-09 12:36:42 +02:00
Eddú Meléndez
50f549d960 Update testcontainers redis version to 2.2.0 2024-04-04 13:43:01 +02:00
Eddú Meléndez
8fa675b145 Update spring boot version to 3.2.4 2024-04-04 13:43:01 +02:00
Eddú Meléndez
b3f2516b7d Add Elasticsearch vector store auto-configuration
* Add spring-ai-elasticsearch-store-spring-boot-starter
* Register spring-ai-elasticsearch-store-spring-boot-starter in bom

Fixes #475
2024-03-21 13:18:41 +01:00
Pablo Sanchidrian
52d15f5dc1 Add Watsox.AI Chat Client integration
- feat: setup watsonx ai api
 - feat: add watsonx ai model options
 - feat: setup watsonx chat client
 - feat: add watsonx records/models
 - add watsonx-ai module to pom
 - add watsonx-ai module to bom
 - feat: add connection properties watsonx
 - feat: add  WatsonxAiAutoConfiguration with api client
 - feat: add starter watsonx.ai
 - feat: add generate method in watsonx ai api
 - feat: add watsonx ai api streaming generation method
 - feat: add watsonx message to prompt converter util
 - feat: implement call and stream mehtod
 - feat: add watsonx ai runtime hints
 - fix: filter null fields
 - feat: watsonx options tests
 - feat: add test dependencies
 - feat: add runtime hints tests
 - feat: add watsonx client tests
 - fix: apply linter
 - feat: add tests for message to prompt converter
 - feat: add signature
 - fix: change deprecated IamAuthenticator
 - fix: do not keep baseUrl in a class variable
 - fix: webClient request
 - feat: add default base url to autoconfigure
 - feat: add watsonx ai integration docs
 - fix: model options json
 - feat: add watsonx-ai spring boot starter
 - feat: enable watsonx api on watsonx chat client
 - fix: remove condition
 - feat: add watsonx autoconfigure import
 - feat: add watsonx module resource aot import
 - feat: add pom for watsonx ai module

 Additional pre-merge adjustments:

 - Rename all WatsonxAIXxx classes to WatsonxAiXxx.
 - Rename WatsonxChatClient to WatsonxAiChatClient.
 - Move WatsonxAiChatOptions out of the API.
 - Implement a Builder for WatsonxAiChatOptions (replace the inline withXxx code).
 - Add a WatsonxAiChatOptions field to WatsonxAiChatClient as default options.
   Later, it is also set by the auto-configuration properties.
 - Implement merging logic for default vs runtime options in WatsonxAiChatClient.
 - In Auto-config, add WatsonxAiChatProperties with enabled and options fields.
   Options are passed to the client.
 - Update the adoc to include the .chat.options properties.
 - Add the watsonxai doc to the nav.adoc.
 - Fix license headers and javadocs.
 - Move dependency versioning to the parent POM.
2024-03-20 15:18:48 +01:00
geetrawat
7634c6b780 Add Gemfire vector store
- Implement a GemFireVectorStore implementing the VectorStore interface.
 - Add unit and integration test.
 - Add antora documentation.
 - Add to BOM.
2024-03-19 07:41:17 +01:00
Christian Tzolov
ce2bb131e5 Implement Anthropic Claude3 Message API client support (direct)
This commit introduces support for the Anthropic Claude3 Message API
  (https://api.anthropic.com), enabling direct interaction with its services.
  This is not a Bedrock Anthropic Claude3 implemenation.

  Changes include:

  - Implementation of a low-level client, AnthropicApi, to interact with
    the message API endpoints specified in the Anthropic documentation
    (https://docs.anthropic.com/claude/reference/messages_post), including support for streaming.
  - Addition of AnthropicApi tests to ensure functionality and reliability.
  - Support for multimodal requests within AnthropicApi.
  - Adding the spring-ai-anthropic and boot starter into BOM and parent POM modules for streamlined usage.
  - Add Anthropic Auto-configuration and Boot Starter for seamless integration into existing projects.
  - Implementation of AnthropicChatClient with capabilities for synchronous and streaming communication,
    including support for multimodal messages.
  - Inclusion of both unit and integration tests to validate functionality across various scenarios.
  - Add Antora documentation with comprehensive guidance on using AnthropicApi and AnthropicChatClient.
  - Add of Ahead-of-Time (AOT) hints for AnthropicApi.
  - update anthropic diagram
2024-03-19 00:36:29 -04:00
Eddú Meléndez
8503078088 Add ServiceConnection support for
* ChromaDBContainer
 * MilvusContainer
 * QdrantContainer
 * RedisStackContainer
 * WeaviateContainer
 * OllamaContainer

Add docs
2024-03-18 23:29:55 -04:00
Eddú Meléndez
150b268415 Add MongoDB Atlas vector store auto-configuration
It also provides a starter.
2024-03-18 15:07:34 +01:00
Mark Pollack
4c617e16b4 Prepare next development iteration 2024-03-12 14:33:45 -04:00
Mark Pollack
490f3cd1cb Milestone Release 0.8.1 2024-03-12 14:28:28 -04:00
Christian Tzolov
076726c1ca Add Mistral AI Function Calling support
- Make MistralAiChatClient extend the AbstractFunctionCallSupport and implement the necessary abstract classes.
 - Extend the MistralAiApi to include the latest (undocumented) changes providing function calling support as well.
   The Mistral AI is almost identical to the OpenAI API except it doesn't support parallel function colling (e.g. missing tool_call_id).
 - Add MistralAiApi function calling tests (implement the Mistral tutorial).
 - Extend the misral chat options to include the new API features and function call abstractions.
 - Extend Mistral's chat auto-configration to accomodate the function callback support.
 - Add ITs for testing function calling.
 - Remove redundant code from MistralAiApi and OpenAiApi.
 - Simplify and improve the HTTP error handling in OpenAiApi, ImageAiApi and MistralAiApi.
2024-03-01 09:53:01 +01:00
Anush008
ea0b439dac Implement Qdrant vector store
- Implement QdrantVectorStore.
   Uses a custom parser for converting Spring AI metadata(Map<String, Object>) to Qdrant GRPC payload.
 - Implement Qdrant Expression Filter support.
   Uses a custom parser for converting Spring AI filters to Qdrant-compatible GRPC filters.
 - Add ITs using testcontainers.
 - Add antora docs adrant.adoc.
 - Add Qdrant vector store auto-configuraton and boot starter.

Additional (review) change:

 - Fix poms parent to 0.8.1-SNAPSHOT.
 - Rename ObjectFactory into QdrantObjectFactor.
 - Rename ValueFactory into QdrantValueFactory.
 - Move the org.springframework.ai.vectorstore package into org.springframework.ai.vectorstore.qdrant.
 - Add missing Autoconfigure definition.
 - Add missing license and JavaDocs.
 - Minor code style improvmentes.
 - Move the qdrant version to the main pom
 - Add QdrantVectorStoreAutoConfigurationIT
 - Remove guava dependency
 - Improve gdrant.adoc conent and structure.
 - Remove the grpc-protobuf dependency

Resolves #331
2024-02-28 19:31:54 +01:00
Christian Tzolov
65d42c9d4f Add full support or Vertex AI Gemini and Azure OpenAI function calling
- Extend the Spring AI Message with getMediaData() : List<MediaData>
   MediaData is a pair of MimeType and data of type Object.
   Message#getContent() return text only.

 - VertexAI Gemini Support
   - implement VertexAiGeiminChatClient for ChatClient and StreamingChat client and support for MediaData content.
     add IT tests for Chat, Streaming and Multimodality
   - add Auto-configuration + ITs
   - add Gemini Spring Boot starter.
   - add clients and boot starters to the Spring AI BOM.
   - add Anotra documentation for the Gemini chat client.
   - update gemini to latest 26.33.0 BOM.
   - add vertex ai gemini dependencies to the BOM.
   - add Vertex AI Gemini API Function Calling support
   - add Gemini API Function Calling Streaming support
   - add vertex ai gemini function calling documentation
   - factor out the Function Calling functionality into common abstraction used by OpenAI, Azure and Gemini.
   - group the VertexAI documentation under a common parent
   - add PortableFunctionCallingOption that implements FunctionCallingOptions and ChatOptions and provide builder for it.
   - remove some deprecated code.
   - allow authorization with GoogleCredentials form json file.
   - add AOT support for VertexAI Gemini.
   - move legacy Vertex AI into VertexAI PaLM2.
   - better handling for empty chat responses.
   - update the Gemini version to latest 26.33.0. This required lifting the protobuf-java to 3.25.2 as well.
   - fix a bug for handling System messages with Gemini.

- Implement Azure OpenAI Function Calling
  Uses the same the common abstractions used by OpenAI and Gemini: FunctionCallingOptions and AbstractFunctionCallSupport
2024-02-28 15:41:37 +01:00
ricken07
30c3530561 Add integration for Mistral AI Chat and Embedding models
- Implement MistrealAiApi as a REST client for the Mistral REST API.
 - Add MistralAiChatClient implementing ChatClient and StreamingChatClient.
   Add MistralAiChatOptions implementing ChatOptions.
 - Add MistralAiEmbeddingClient impelementing EmbeddingClient.
   Add MistralAiEmbeddingOptions implementing EmbeddingOptions.
 - Add Unit and IT tests for api and clients.
 - Add mistral  auto-configuration and boot strarter
 - update auto-config property for Mistral AI embedding client
 - update auto-config property for Mistral AI embedding client

 Additional, review changes:
 - Add missing license headers.
 - Fix the intialization of defaul options.
 - Code re-formatting to improve the readabitliy.
 - Many improvements
 - Add missing sine annotations
 - Remove stream field from MistralAiChatOptions. This handled internally.
 - Add MistralAiApi ChatModel and EmbeddingModel enums.
 - Add MistralChatClientIT for testeing chat, stream , parsers...
 - Add MistralAiApiIT tests
 - Refactor and streamline the MistralAiAutoConfiguration.
 - Add MistralAiAutoConfigurationIT
2024-02-28 13:29:35 +01:00
Mark Pollack
8532cb7bcc Bump to 0.8.1-SNAPSHOT 2024-02-23 11:11:14 -05:00
utkarsh
5718f9683d Added PostgresMlAutoConfiguration (#242)
Fixes issue #242
2024-01-25 16:52:42 -05:00
Mark Pollack
243cef976c Abstract API for AI model clients
* An abstract API for AI model clients
 * Providing portable client request options while still allowing vendor specific options when required.  Implemented only for StabilityAI/OpenAI ImageClient
 * Support for text->image for openai and stabilityai.

  Partial fix for #27 :  Text To Image and Fixes #266 and Fixes #261
2024-01-24 19:53:33 +01:00
oujingzhou
b625ab1c15 Add Spring Boot auto-configuration for Neo4j vector store
- resolve Neo4j auto-configuraion property expossing external API.
 - move neo4j auto-conf under the vectorstore parent package.
2023-12-24 09:11:52 +01:00
Christian Tzolov
472fadda61 Update the VecotorStore pom versions
- update boot to 3.2.1
  - update vector-stores: postgresql (42.7.1) , pgvecgor (0.1.4), pinecone (0.7.1)
    azure-search (11.6.1), weaviate (4.5.0)
  - clean the  auto-configuraito properties
  - resolve Milvus auto-conf property expossing external API.
2023-12-23 14:45:54 +01:00
Mark Pollack
82fe510b39 Moved Maven modules from top level directory and embedding-clients subdirectory to all be under a single models directory.
Rename artifact ID of

* `transformers-embedding` to `spring-ai-transformers`
2023-12-19 12:22:02 -05:00
Christian Tzolov
0e3192a8df Add AWS Bedrock AI support
- Add spring-ai-bedrock project with support for Cohere, Llama2, Ai21 Jurassic 2, Titan and Anthropic LLM models.
 - Add native API clients for CohereChat CohereEmbedding , Llama2Chat, JurassicChat, TitanChat and TitanEmbedding models, supporting both single shot and streaming completions (for the models that allows it)
 - Add ITs tests for the native API clients.
 - Implement Chat (AiClient) and ChatStreaming (AiStreamingClient) and EmbeddingClients (according to the models’ support for those) for Cohere, Llama2, and Anthropica. Titan and Jurassic2 are WIP.
 - Add ITs for the ChatClient, ChatStreamingClient and EmbeddingClient implementations.
 - Add Spring Boot Auto-configurations with flexible properties for the Llama2, Anthropic and Cohere modes + ITs
 - Add Spring Boot Starter configurations for all Bedrock models.
 - Add README documentations for all models.
 - Add BedrockAi APIs AOT hints
 - Add Ai21Jurassic2ChatBedrockApi, TitanEmbeddingBedrockApi, TitanChatBedrockApi
 - Add TitanChatBedrockApi

Resolves #66
2023-12-18 18:16:43 -05:00