Moved up one package level the following classes
* org.springframework.ai.huggingface.client.HuggingfaceChatClient
* org.springframework.ai.openai.client.OpenAiChatClient and org.springframework.ai.openai.embedding.OpenAiEmbeddingClient
* org.springframework.ai.vertex.generation.VertexAiChatClient and org.springframework.ai.vertex.embedding.VertexAiEmbeddingClient
Fixes#211
- Add spring-ai-bedrock project with support for Cohere, Llama2, Ai21 Jurassic 2, Titan and Anthropic LLM models.
- Add native API clients for CohereChat CohereEmbedding , Llama2Chat, JurassicChat, TitanChat and TitanEmbedding models, supporting both single shot and streaming completions (for the models that allows it)
- Add ITs tests for the native API clients.
- Implement Chat (AiClient) and ChatStreaming (AiStreamingClient) and EmbeddingClients (according to the models’ support for those) for Cohere, Llama2, and Anthropica. Titan and Jurassic2 are WIP.
- Add ITs for the ChatClient, ChatStreamingClient and EmbeddingClient implementations.
- Add Spring Boot Auto-configurations with flexible properties for the Llama2, Anthropic and Cohere modes + ITs
- Add Spring Boot Starter configurations for all Bedrock models.
- Add README documentations for all models.
- Add BedrockAi APIs AOT hints
- Add Ai21Jurassic2ChatBedrockApi, TitanEmbeddingBedrockApi, TitanChatBedrockApi
- Add TitanChatBedrockApi
Resolves#66
- Added autoconfiguration for Redis vector store
- Added spring boot starter for Redis vector store
- Supports portable metadata filter expressions
Fixes#11
The old function got replaced and might get removed in future versions.
To ensure that the store is compatible with Neo4j 5.LATEST, this commit
changes the store function for embeddings to use the 5.13+ function.
The correct baseline database version is already mentioned in the README.
- Introducing a new OpenAiApi native client for OpenAI API and get rid of the theokanning library.
Amongst others the OpenAiApi allows:
- easy base-url configuration (e.g. TAS-AI)
- Flux response for streaming OpenAI results.
- Exposes the http headers containing important metadata
- Pure Spring ecosystem, making it easier for Graal VM
- Define a new AiStreamClient interface returning Flux<AiResponse>
- Refactor OpenAiClient and to use the new OpenAiApi and implement the AiStreamClient.
- Use spring-retry to improve the OpenAI EmbeddingClient stability on 503 error.
- Remove the OpenAI http header interceptor as the OpenAiApi returns ResponseEntity<T> that provides direct access to the headers.
- Refactor the metadata headers and usage extraction.
- Remove redundant and obsolete classes.
- Fix dependency issue with Pinecone, netty-codec-http2 and Spring Boot 3.2
- Add NOT expression type to the portable Filter.Expression model.
- Add NOT to the Antlr grammar and implement the related parser listener method to generate Filter NOT expressions.
- Add NOT support to the filter programming DSL.
- Implement FilterHelper.negation for logically transform any boolean expression with NOT statements into
semantically equivalent one with NOT applied to the leaf expressions.
- Add tests for paresers, converters and vectorsores ITs.
- Move the filter IN/NIN expansion logic to the FilterHelper
- Factor out the filter IN/NIN boolean expression expansion logic out of Weaviate up to the FilterHelper.
- add in/nin expantion FilterHelper tests
- Add Weaviate Vector Store implementation.
- Implement a converter of portable Filter.Expressions into native, Weaviate GraphQL Were expressions.
- Support for Weaviater schema auto-registration of filtarable metadata fields.
- Add auto-configration, spring properties and tests.
- WeaviateVectorStore ITs.
- Add README.md
Resolves#100
- Use Azure AI Search end point to implement the VectorStore interface.
- Add ITs and README.
- Create/update vector index on after properties set.
- Add boot auto-configuration and ITs.
- Add boot starter for the vector store.
Resolves: #82
- Implement ChromaApi client, based on Chroma REST API.
- Implement ChromaVectorStore, including support for filter expression conversion.
- Common VectorStoreUtil class to share to/from Float/Double list/array convertion as well as Json/Map convertions.
- Add ITs including for Basic Auth and Token autheticatios.
- Add ChromaApi security support for BasicAuth and Token.
- Fix an issue with Text filter expression parser, related to double-quoted identifiers.
- Add Chroma README.md.
- Add Chroma boot autoconfiguration
Resolves# #86
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
- Fix all affected code and tests.
- Bump the project version to 0.7.1.
- Add tests
- Add autoconfigurations for milvus, pinecone and pgvecor stores.
- Improve and unify the VectorStore ITs.
- Make use of TrasformersEmbeddingClient for auto-configurations ITs.
* Clean up README.md files in Milvus, PGvector, and Pinecone modules.
* Apply consistent treatment of 'model' when used as an AI concept, e.g. AI model or Embedding model.
* Apply consistent treatment of 'vector store' and 'vector database' references.
* Simplify sentence structures.
Closes#79
- Extend the VectorStore with similaritySearch using metadata filters using internal DSL and external DSL using Antlr
- Metdata support for Pinecone, Milvus, and pgvector vector stores
- PGVectorStore uses explict ::jsonpath casting for the pgvector filter expression to avoid injections
- Add unit tests for the filter converters, parser and DSL.
- Add ITs for the 3 vector stores
Resolves: #75
- Based on the official pinecone java library.
Later expects that indices are created externally via Ops.
- Map Document metadata to and from Pinecone's internal Struct.
Later converts the metadata into pinecone json format.
- Add integration tests and README.
As the LifeCycle#start() occures later in time than the InitializingBean#afterPropertySet()
it could cose some initialization issues with the vector clients.
- Add ContentFormatter and DefaultContentFormatter that can filter the metadata
and format the Document metadata and text according to predefined templates.
- Add content formatter tests
- Allow the TextSplitter to copy the document content-formatter to the children.
When the splitter breaks the parent Document into multiple chunks (e.g.
into a list of children Documents) copy the source content formatter to
the chunks by default. Use the copyContentFormatter flag to enable/disable copping.
- Add TextSplitter IT tests
- Add MetadataExtractors as DocumentTransformers.
- Bump spring-ai project version to 0.7.0-SNAPSHOT
- Configurable metadata-mode for EmbeddingClients
- Make the metadata mode configurable for the EmbeddingClient implementations.
- Use the EMBED mode by default.
Resolves#44
Leverage #28 to allow the vector stores to resolve the embedding dimensions dynamically.
The explicitly set dimensions (if set) precedence over other configurations.
If the embedding dimensions are not explicitly set, the embeddingClient is used
to determine them dynamically. If the client fails the it falls back to 1536.
- Ensures that the vector store similarity search by threshold tests
use dynamically computed threshold that is between the top 2 results
from the ordered search. This ensures that the threshold value is not
affected by changes in the embedding API results.
- Minor code style improvments.
This commit brings support for Neo4j graph database in general,
and uses the vector index functionality available since version 5.11.
Aligned with the existing PgVector store and its tests.
The module creates indexes, if needed, for the vector search and
the identifier of the document object.
Add neo4j to vectordb docs page
Co-authored-by: Michael Simons <michael@simons.ac>
Resolves#19
* Clear and stream line. Inject search distance response as metadata
* Fix similarity threshold by metric type
- Fix the similary search with threshold for L2 and IP types.
- Create config Builder Config (inspired by Neo4jVectorStore impelementation.
- add database-name, metric-type, index-type and index-param configurations.
* update vector store doc and readme
