Add NOT operator to VectorStore portable filter expressions

- Add NOT expression type to the portable Filter.Expression model.
 - Add NOT to the Antlr grammar and implement the related parser listener method to generate Filter NOT expressions.
 - Add NOT support to the filter programming DSL.
 - Implement FilterHelper.negation for logically transform any boolean expression with NOT statements into
   semantically equivalent one with NOT applied to the leaf expressions.
 - Add tests for paresers, converters and vectorsores ITs.
 - Move the filter IN/NIN expansion logic to the FilterHelper
 - Factor out the filter IN/NIN boolean expression expansion logic out of Weaviate up to the FilterHelper.
 - add in/nin expantion FilterHelper tests
This commit is contained in:
Christian Tzolov
2023-11-23 21:29:48 +01:00
committed by Mark Pollack
parent e3e5070256
commit ac9ae589f4
25 changed files with 731 additions and 153 deletions

View File

@@ -113,7 +113,7 @@ One approach involves presenting both the user's request and the AI model's resp
Furthermore, leveraging the information stored in the Vector Database as supplementary data can enhance the evaluation process, aiding in the determination of response relevance.
The Spring AI project currenlty provides some very basic examples of how you can evaluate the responses in the form of prompts to include in a JUnit test.
The Spring AI project currently provides some very basic examples of how you can evaluate the responses in the form of prompts to include in a JUnit test.

View File

@@ -87,8 +87,7 @@ country == 'UK' && year >= 2020 && isActive == true.
These are the available implementations of the `VectorStore` interface:
* `InMemoryVectorStore`
* `SimplePersistentVectorStore`
* `InMemoryVectorStore` and `SimplePersistentVectorStore`.
* Pinecone: https://www.pinecone.io/[PineCone] vector store.
* PgVector [`PgVectorStore`]: The https://github.com/pgvector/pgvector[PostgreSQL/PGVector] vector store.
* Milvus [`MilvusVectorStore`]: The https://milvus.io/[Milvus] vector store
@@ -117,7 +116,7 @@ The `VectorStore` implementation computes the embeddings and stores the JSON and
@Autowired
VectorStore vectorStore;
void load(String sourceFile) {}
void load(String sourceFile) {
JsonReader jsonReader = new JsonReader(new FileSystemResource(sourceFile),
"price", "name", "shortDescription", "description", "tags");
List<Document> documents = jsonReader.get();