- Introducing a new OpenAiApi native client for OpenAI API and get rid of the theokanning library.
Amongst others the OpenAiApi allows:
- easy base-url configuration (e.g. TAS-AI)
- Flux response for streaming OpenAI results.
- Exposes the http headers containing important metadata
- Pure Spring ecosystem, making it easier for Graal VM
- Define a new AiStreamClient interface returning Flux<AiResponse>
- Refactor OpenAiClient and to use the new OpenAiApi and implement the AiStreamClient.
- Use spring-retry to improve the OpenAI EmbeddingClient stability on 503 error.
- Remove the OpenAI http header interceptor as the OpenAiApi returns ResponseEntity<T> that provides direct access to the headers.
- Refactor the metadata headers and usage extraction.
- Remove redundant and obsolete classes.
- Fix dependency issue with Pinecone, netty-codec-http2 and Spring Boot 3.2
- Add NOT expression type to the portable Filter.Expression model.
- Add NOT to the Antlr grammar and implement the related parser listener method to generate Filter NOT expressions.
- Add NOT support to the filter programming DSL.
- Implement FilterHelper.negation for logically transform any boolean expression with NOT statements into
semantically equivalent one with NOT applied to the leaf expressions.
- Add tests for paresers, converters and vectorsores ITs.
- Move the filter IN/NIN expansion logic to the FilterHelper
- Factor out the filter IN/NIN boolean expression expansion logic out of Weaviate up to the FilterHelper.
- add in/nin expantion FilterHelper tests
- Use Azure AI Search end point to implement the VectorStore interface.
- Add ITs and README.
- Create/update vector index on after properties set.
- Add boot auto-configuration and ITs.
- Add boot starter for the vector store.
Resolves: #82
- Implement ChromaApi client, based on Chroma REST API.
- Implement ChromaVectorStore, including support for filter expression conversion.
- Common VectorStoreUtil class to share to/from Float/Double list/array convertion as well as Json/Map convertions.
- Add ITs including for Basic Auth and Token autheticatios.
- Add ChromaApi security support for BasicAuth and Token.
- Fix an issue with Text filter expression parser, related to double-quoted identifiers.
- Add Chroma README.md.
- Add Chroma boot autoconfiguration
Resolves# #86
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
- Fix all affected code and tests.
- Bump the project version to 0.7.1.
- Add tests
- Add autoconfigurations for milvus, pinecone and pgvecor stores.
- Improve and unify the VectorStore ITs.
- Make use of TrasformersEmbeddingClient for auto-configurations ITs.
* Clean up README.md files in Milvus, PGvector, and Pinecone modules.
* Apply consistent treatment of 'model' when used as an AI concept, e.g. AI model or Embedding model.
* Apply consistent treatment of 'vector store' and 'vector database' references.
* Simplify sentence structures.
Closes#79
- Extend the VectorStore with similaritySearch using metadata filters using internal DSL and external DSL using Antlr
- Metdata support for Pinecone, Milvus, and pgvector vector stores
- PGVectorStore uses explict ::jsonpath casting for the pgvector filter expression to avoid injections
- Add unit tests for the filter converters, parser and DSL.
- Add ITs for the 3 vector stores
Resolves: #75
- Based on the official pinecone java library.
Later expects that indices are created externally via Ops.
- Map Document metadata to and from Pinecone's internal Struct.
Later converts the metadata into pinecone json format.
- Add integration tests and README.
