Commit Graph

37 Commits

Author SHA1 Message Date
Christian Tzolov
ac9ae589f4 Add NOT operator to VectorStore portable filter expressions
- Add NOT expression type to the portable Filter.Expression model.
 - Add NOT to the Antlr grammar and implement the related parser listener method to generate Filter NOT expressions.
 - Add NOT support to the filter programming DSL.
 - Implement FilterHelper.negation for logically transform any boolean expression with NOT statements into
   semantically equivalent one with NOT applied to the leaf expressions.
 - Add tests for paresers, converters and vectorsores ITs.
 - Move the filter IN/NIN expansion logic to the FilterHelper
 - Factor out the filter IN/NIN boolean expression expansion logic out of Weaviate up to the FilterHelper.
 - add in/nin expantion FilterHelper tests
2023-11-28 18:17:56 -05:00
Harry9656
8e8be5da4a Fix link pointing to arxiv 2023-11-25 12:36:14 +01:00
Christian Tzolov
faee5fef6f Add support for Weaviate Vector Store
- Add Weaviate Vector Store implementation.
 - Implement a converter of portable Filter.Expressions into native, Weaviate GraphQL Were expressions.
 - Support for Weaviater schema auto-registration of filtarable metadata fields.
 - Add auto-configration, spring properties and tests.
 - WeaviateVectorStore ITs.
 - Add README.md

 Resolves #100
2023-11-21 18:16:34 -05:00
John Blum
eadcae7702 Define API for AI provider metadata collected during an AI request.
* Define GenerationMetadata property in AiResponse.
* Add OpenAI implementations of AiMetadata, RateLimit and Usage interfaces.
* Add REST Assured JsonPath dependency to spring-ai-openai module.
* Add OkHttp dependency to spring-ai-openai module.
* Add OkHttp Interceptor to parse OpenAI rate limit metadata from HTTP headers.
* Add OkHttp MockWebServer dependency to spring-ai-openai module, test scope
* Add Jakarta Servlet API dependency to spring-ai-openai module, test scope
* Add Spring Web MVC dependency to spring-ai-open-ai module., test scope
* Define OpenAI API response headers in an Enum.
* Add OpenAI test configuration using mock objects.
* Add integration test to assert successful extraction of OpenAI API response metadata.
* Include Spring Boot auto-configuration for (conditional) OpenAI metadata collection.
* Edit documentation and include information on AI metadata collected by Spring AI.
* Provide AI metadata implementation for Microsoft Azure OpenAI Service.
* Capture optional PromptMetadata in AiResponse.
* Define metadata for an AI generation choice.
* Capture AI choice metadata in Generation.
* Integrate ChoiceMetadata into AiResponse returned by OpenAI.

Fixes #98
2023-11-21 18:16:34 -05:00
KathrynBrusewitz
837c4080aa Make filter expression builder Op public 2023-11-15 19:35:49 +01:00
Christian Tzolov
952ff6c319 Simplify VectorStore interface
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
 - Fix all affected code and tests.
 - Bump the project version to 0.7.1.
 - Add tests
 - Add autoconfigurations for milvus, pinecone and pgvecor stores.
 - Improve and unify the VectorStore ITs.
 - Make use of TrasformersEmbeddingClient for auto-configurations ITs.
2023-11-14 15:39:01 -05:00
Mark Pollack
49656a036f Add link to research paper on effective prompts 2023-11-09 15:43:57 -05:00
Jay Bryant
04d9683f49 Editing pass for clarity, grammar, spelling, punctuation, and usage.
Also added several anchors and links.
2023-11-09 15:40:16 -05:00
John Blum
0a236993e3 Fix additional grammatical mistakes and revise wording in vectordbs.adoc.
* Clean up README.md files in Milvus, PGvector, and Pinecone modules.
* Apply consistent treatment of 'model' when used as an AI concept, e.g. AI model or Embedding model.
* Apply consistent treatment of 'vector store' and 'vector database' references.
* Simplify sentence structures.

Closes #79
2023-11-08 12:24:20 -08:00
Mark Pollack
d514f28cd4 update vector store docs 2023-11-07 17:05:14 -05:00
Mark Pollack
bc12d4364c add reference docs for AiClient 2023-11-04 13:58:27 -04:00
Mark Pollack
6d956a6f37 Add initial support for Hugging Face Inference Endpoints
- Add maven config to attach swagger generate code to vscode java classpath.
 - Add license headers.
 - Streamline the test.
2023-11-03 18:45:34 +01:00
John Blum
a97976e434 Clean up grammatical errors and misspellings in the reference documentation. (#77)
* Fixes misspelling in concepts.adco, 'Prompts' section.
* Uses plural form of AI Models in aiclient.adoc.
* Fixes several grammatical mistakes in vectordbs.adoc.
2023-11-03 09:46:39 -04:00
DOP
81e7aded94 add "List<Double>" right bracket
add "List<Double>" right bracket :
before:   List<Double   
after :     List<Double>
2023-10-30 16:55:02 +01:00
Mark Pollack
d824ff839d remove sections of API docs that are not yet relevant 2023-10-26 13:30:43 -04:00
Mark Pollack
ed89e5fd9a fix docs build 2023-10-06 08:25:33 +02:00
Mark Pollack
f37999f7c3 disable pdf/epub temporarily 2023-10-06 08:02:39 +02:00
Mark Pollack
f6db217c65 Docs update, WIP 2023-10-04 16:37:37 -04:00
Christian Tzolov
f4f1d6f821 Rename XXLoader to XXReader. Disable AcmeIT.acmeChain() test. 2023-10-04 11:25:01 +02:00
Christian Tzolov
76e44d68e4 Improve the Document API
- Add ContentFormatter and DefaultContentFormatter that can filter the metadata
   and format the Document metadata and text according to predefined templates.
 - Add content formatter tests
 - Allow the TextSplitter to copy the document  content-formatter to the children.
    When the splitter breaks the parent Document into multiple chunks (e.g.
    into a list of children Documents) copy the source content formatter to
    the chunks by default. Use the copyContentFormatter flag to enable/disable copping.
 - Add TextSplitter IT tests
 - Add MetadataExtractors as DocumentTransformers.
 - Bump spring-ai project version to 0.7.0-SNAPSHOT
 - Configurable metadata-mode for EmbeddingClients
   - Make the metadata mode configurable for the EmbeddingClient implementations.
   - Use the EMBED mode by default.

 Resolves #44
2023-10-03 14:39:17 -04:00
Gerrit Meier
a6fba7a3b6 GH-26 - Add support for Neo4j vector index.
This commit brings support for Neo4j graph database in general,
and uses the vector index functionality available since version 5.11.

Aligned with the existing PgVector store and its tests.

The module creates indexes, if needed, for the vector search and
the identifier of the document object.

Add neo4j to vectordb docs page

Co-authored-by: Michael Simons <michael@simons.ac>
2023-09-27 16:03:51 -04:00
Christian Tzolov
141f503341 Integrate Milvus as a Vestor Store
Resolves #19

* Clear and stream line. Inject search distance response as metadata
* Fix similarity threshold by metric type

 - Fix the similary search with threshold for L2 and IP types.
 - Create config Builder Config (inspired by Neo4jVectorStore impelementation.
   - add database-name, metric-type, index-type and index-param configurations.

* update vector store doc and readme
2023-09-27 12:24:52 -04:00
Rob Winch
768514e768 Add asciidoctor-mathjax 2023-09-18 16:25:21 -05:00
Mark Pollack
532ae01f41 bump antora ui version 2023-09-17 18:09:06 -04:00
Mark Pollack
9b0e600285 fix typo, rerun CI 2023-09-17 17:54:31 -04:00
Christian Tzolov
5a63cd840c Add PGVector VectorStore implementation
* Enable PGVector indexing support
* Add same document ID vector update support
* Add documentation for VectorStores
2023-09-17 17:41:37 -04:00
Simon Verhoeven
6b9a1dad5c Update documentation, remove unused imports
Signed-off-by: Simon Verhoeven <verhoeven.simon@gmail.com>
2023-08-26 06:50:07 +02:00
Mark Pollack
e3da80d86e update docs 2023-08-21 08:16:05 -07:00
Mark Pollack
8b0ad1a183 add getting started 2023-08-20 21:49:47 -07:00
Mark Pollack
a5c0ede2d5 add concepts docs 2023-08-20 21:05:18 -07:00
Janne Valkealahti
d7610b774e Fix resources dir 2023-08-16 17:25:03 +03:00
Janne Valkealahti
7e38e285b3 Add missing antora resources file 2023-08-16 17:19:05 +03:00
Janne Valkealahti
05a0520596 Initial antora docs structure 2023-08-16 07:42:19 +01:00
Mark Pollack
ca26c305ab change to 0.2.0-SNAPSHOT 2023-08-12 10:16:32 -04:00
Mark Pollack
e1b4bc2fff change to version 0.1.0-SNAPSHOT 2023-08-12 09:51:58 -04:00
Mark Pollack
e042a8e604 OpenAI implementation with autoconfig and starters 2023-08-05 11:13:42 -04:00
Mark Pollack
d5c1044aee iniital maven build system 2023-07-24 19:37:06 -04:00