The text-embedding-3-small has the same dimensions as previous text-embedding-ada-002.
The text-embedding-3-large has higher dimensionality not supported by some Vector Stores.
To make the Neo4j module more future-proof, this commit
replaces the old vector index creation syntax with the new style.
Also, the new pattern is in line with the standard Neo4j index creation
and supports the _IF NOT EXISTS_ clause to run idempotent.
This allows us to remove the preceding call to check if the index exists.
As a consequent, the module will require Neo4j to be at least on version 5.15.
At the moment, it is not possible to configure SpringAI
to use an existing index in the database.
This commit enables the user to provide the index name
for auto configuration or builder usage.
Moved up one package level the following classes
* org.springframework.ai.huggingface.client.HuggingfaceChatClient
* org.springframework.ai.openai.client.OpenAiChatClient and org.springframework.ai.openai.embedding.OpenAiEmbeddingClient
* org.springframework.ai.vertex.generation.VertexAiChatClient and org.springframework.ai.vertex.embedding.VertexAiEmbeddingClient
Fixes#211
- Add spring-ai-bedrock project with support for Cohere, Llama2, Ai21 Jurassic 2, Titan and Anthropic LLM models.
- Add native API clients for CohereChat CohereEmbedding , Llama2Chat, JurassicChat, TitanChat and TitanEmbedding models, supporting both single shot and streaming completions (for the models that allows it)
- Add ITs tests for the native API clients.
- Implement Chat (AiClient) and ChatStreaming (AiStreamingClient) and EmbeddingClients (according to the models’ support for those) for Cohere, Llama2, and Anthropica. Titan and Jurassic2 are WIP.
- Add ITs for the ChatClient, ChatStreamingClient and EmbeddingClient implementations.
- Add Spring Boot Auto-configurations with flexible properties for the Llama2, Anthropic and Cohere modes + ITs
- Add Spring Boot Starter configurations for all Bedrock models.
- Add README documentations for all models.
- Add BedrockAi APIs AOT hints
- Add Ai21Jurassic2ChatBedrockApi, TitanEmbeddingBedrockApi, TitanChatBedrockApi
- Add TitanChatBedrockApi
Resolves#66
- Added autoconfiguration for Redis vector store
- Added spring boot starter for Redis vector store
- Supports portable metadata filter expressions
Fixes#11
The old function got replaced and might get removed in future versions.
To ensure that the store is compatible with Neo4j 5.LATEST, this commit
changes the store function for embeddings to use the 5.13+ function.
The correct baseline database version is already mentioned in the README.
- Introducing a new OpenAiApi native client for OpenAI API and get rid of the theokanning library.
Amongst others the OpenAiApi allows:
- easy base-url configuration (e.g. TAS-AI)
- Flux response for streaming OpenAI results.
- Exposes the http headers containing important metadata
- Pure Spring ecosystem, making it easier for Graal VM
- Define a new AiStreamClient interface returning Flux<AiResponse>
- Refactor OpenAiClient and to use the new OpenAiApi and implement the AiStreamClient.
- Use spring-retry to improve the OpenAI EmbeddingClient stability on 503 error.
- Remove the OpenAI http header interceptor as the OpenAiApi returns ResponseEntity<T> that provides direct access to the headers.
- Refactor the metadata headers and usage extraction.
- Remove redundant and obsolete classes.
- Fix dependency issue with Pinecone, netty-codec-http2 and Spring Boot 3.2
- Add NOT expression type to the portable Filter.Expression model.
- Add NOT to the Antlr grammar and implement the related parser listener method to generate Filter NOT expressions.
- Add NOT support to the filter programming DSL.
- Implement FilterHelper.negation for logically transform any boolean expression with NOT statements into
semantically equivalent one with NOT applied to the leaf expressions.
- Add tests for paresers, converters and vectorsores ITs.
- Move the filter IN/NIN expansion logic to the FilterHelper
- Factor out the filter IN/NIN boolean expression expansion logic out of Weaviate up to the FilterHelper.
- add in/nin expantion FilterHelper tests
- Add Weaviate Vector Store implementation.
- Implement a converter of portable Filter.Expressions into native, Weaviate GraphQL Were expressions.
- Support for Weaviater schema auto-registration of filtarable metadata fields.
- Add auto-configration, spring properties and tests.
- WeaviateVectorStore ITs.
- Add README.md
Resolves#100
- Use Azure AI Search end point to implement the VectorStore interface.
- Add ITs and README.
- Create/update vector index on after properties set.
- Add boot auto-configuration and ITs.
- Add boot starter for the vector store.
Resolves: #82
- Implement ChromaApi client, based on Chroma REST API.
- Implement ChromaVectorStore, including support for filter expression conversion.
- Common VectorStoreUtil class to share to/from Float/Double list/array convertion as well as Json/Map convertions.
- Add ITs including for Basic Auth and Token autheticatios.
- Add ChromaApi security support for BasicAuth and Token.
- Fix an issue with Text filter expression parser, related to double-quoted identifiers.
- Add Chroma README.md.
- Add Chroma boot autoconfiguration
Resolves# #86
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
- Fix all affected code and tests.
- Bump the project version to 0.7.1.
- Add tests
- Add autoconfigurations for milvus, pinecone and pgvecor stores.
- Improve and unify the VectorStore ITs.
- Make use of TrasformersEmbeddingClient for auto-configurations ITs.
* Clean up README.md files in Milvus, PGvector, and Pinecone modules.
* Apply consistent treatment of 'model' when used as an AI concept, e.g. AI model or Embedding model.
* Apply consistent treatment of 'vector store' and 'vector database' references.
* Simplify sentence structures.
Closes#79
- Extend the VectorStore with similaritySearch using metadata filters using internal DSL and external DSL using Antlr
- Metdata support for Pinecone, Milvus, and pgvector vector stores
- PGVectorStore uses explict ::jsonpath casting for the pgvector filter expression to avoid injections
- Add unit tests for the filter converters, parser and DSL.
- Add ITs for the 3 vector stores
Resolves: #75
- Based on the official pinecone java library.
Later expects that indices are created externally via Ops.
- Map Document metadata to and from Pinecone's internal Struct.
Later converts the metadata into pinecone json format.
- Add integration tests and README.
