Though the property names are different, for example 'spring.ai.openai.api-key' and 'spring.ai.openai.embedding-api-key,
by introducing an additional property namespace, 'spring.ai.openai.embedding.*', this enables the configuration of embeddings
to be properly encapsulated and related in Spring Boot fashion.
Closes#102
* Apply consistent treatment to the organization of the source code.
* Fix Logger statements missing message placeholder formatting.
* Simplify logic using Java 17 syntax where applicable.
Closes#101
- Use Azure AI Search end point to implement the VectorStore interface.
- Add ITs and README.
- Create/update vector index on after properties set.
- Add boot auto-configuration and ITs.
- Add boot starter for the vector store.
Resolves: #82
- Implement ChromaApi client, based on Chroma REST API.
- Implement ChromaVectorStore, including support for filter expression conversion.
- Common VectorStoreUtil class to share to/from Float/Double list/array convertion as well as Json/Map convertions.
- Add ITs including for Basic Auth and Token autheticatios.
- Add ChromaApi security support for BasicAuth and Token.
- Fix an issue with Text filter expression parser, related to double-quoted identifiers.
- Add Chroma README.md.
- Add Chroma boot autoconfiguration
Resolves# #86
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
- Fix all affected code and tests.
- Bump the project version to 0.7.1.
- Add tests
- Add autoconfigurations for milvus, pinecone and pgvecor stores.
- Improve and unify the VectorStore ITs.
- Make use of TrasformersEmbeddingClient for auto-configurations ITs.
* Clean up README.md files in Milvus, PGvector, and Pinecone modules.
* Apply consistent treatment of 'model' when used as an AI concept, e.g. AI model or Embedding model.
* Apply consistent treatment of 'vector store' and 'vector database' references.
* Simplify sentence structures.
Closes#79
- Extend the VectorStore with similaritySearch using metadata filters using internal DSL and external DSL using Antlr
- Metdata support for Pinecone, Milvus, and pgvector vector stores
- PGVectorStore uses explict ::jsonpath casting for the pgvector filter expression to avoid injections
- Add unit tests for the filter converters, parser and DSL.
- Add ITs for the 3 vector stores
Resolves: #75
* Fixes misspelling in concepts.adco, 'Prompts' section.
* Uses plural form of AI Models in aiclient.adoc.
* Fixes several grammatical mistakes in vectordbs.adoc.
- Based on the official pinecone java library.
Later expects that indices are created externally via Ops.
- Map Document metadata to and from Pinecone's internal Struct.
Later converts the metadata into pinecone json format.
- Add integration tests and README.
- EmbeddingClient implementation that computes, locally, sentence embeddings with SBERT transformers.
- Uses pre-trained transformer models, serialized into Open Neural Network Exchange (ONNX) format.
- Deep Java Library and the Microsoft ONNX Java Runtime are used to run
the ONNX models and compute the embeddings efficiently.
- Add default tokenizer.json and model.onnx for sentence-transformers/all-MiniLM-L6-v2.
- Add, configurable resource caching service to allow caching
remote (http/https) resources to the local FS.
- README.md provides information on how to serialize ONNX models.
- add Git LFS configuration for large onnx model files.
- Provides a rudimentary text extractions for multitude of document formats,
including PDF, Word Doc/Docx PowerPoint ppt/pptx and many more.
- Generates a single Document for the extracted text.
- No pre or post processing and cleansing for the text.
- Move the ExtractedTextFormatter from pdf reader to the core reader to enable reusability. Improve the tika reader
- Add a set of PDF readers fro per-page (PagePdfDocumentReader) and per-paragraph (ParagraphPdfDocumentReader) readers.
- Use a PDFLayoutTextStripper fork and PDFLayoutTextStripperByArea extension to preserve the structure of the extracted document.
- PdfDocumentReaderConfig and PageExtractedTextFormatter in standalone classes.
- Craeate a new document-readers top level model and the pdf-reader under.
As the LifeCycle#start() occures later in time than the InitializingBean#afterPropertySet()
it could cose some initialization issues with the vector clients.
