- Add postgremaddmbedding adoc page.
- Auto-configuration:
- add missing boot-starter.
- refactor autoconf class and properties to accomodate the PostgresMlEmbeddingOptions.
- PostgesMlEmbeddingClient
- Add the (default) options field and remove old fields.
- Implement default and request options merging.
- Add tests for options and merging.
- Remove redundant code.
- Code style fixes.
* An abstract API for AI model clients
* Providing portable client request options while still allowing vendor specific options when required. Implemented only for StabilityAI/OpenAI ImageClient
* Support for text->image for openai and stabilityai.
Partial fix for #27 : Text To Image and Fixes#266 and Fixes#261
- Implement a native client (OllamaApi) to leverage chat/streaming and embedding endpoints.
- Add a OllamaChatClient implementing the ChatClinet and StreamingChatClinet interfaces.
- Add a OllamaEmbedding clinent that impl. the EmbeddingClinet interface.
- Add AutoConfiguraitons with properties for the chat and the embedding clients.
- Add unit and ITs for the OllamaApi, OllamaChatClient, OllamaEmbeddingClient, and related auto-configuraitons.
- Remove the old ollama impl. classes and tests.
- minor fixes to the bedrok test methods names.
- Add spring-ai-bedrock project with support for Cohere, Llama2, Ai21 Jurassic 2, Titan and Anthropic LLM models.
- Add native API clients for CohereChat CohereEmbedding , Llama2Chat, JurassicChat, TitanChat and TitanEmbedding models, supporting both single shot and streaming completions (for the models that allows it)
- Add ITs tests for the native API clients.
- Implement Chat (AiClient) and ChatStreaming (AiStreamingClient) and EmbeddingClients (according to the models’ support for those) for Cohere, Llama2, and Anthropica. Titan and Jurassic2 are WIP.
- Add ITs for the ChatClient, ChatStreamingClient and EmbeddingClient implementations.
- Add Spring Boot Auto-configurations with flexible properties for the Llama2, Anthropic and Cohere modes + ITs
- Add Spring Boot Starter configurations for all Bedrock models.
- Add README documentations for all models.
- Add BedrockAi APIs AOT hints
- Add Ai21Jurassic2ChatBedrockApi, TitanEmbeddingBedrockApi, TitanChatBedrockApi
- Add TitanChatBedrockApi
Resolves#66
- Added autoconfiguration for Redis vector store
- Added spring boot starter for Redis vector store
- Supports portable metadata filter expressions
Fixes#11
- Introducing a new OpenAiApi native client for OpenAI API and get rid of the theokanning library.
Amongst others the OpenAiApi allows:
- easy base-url configuration (e.g. TAS-AI)
- Flux response for streaming OpenAI results.
- Exposes the http headers containing important metadata
- Pure Spring ecosystem, making it easier for Graal VM
- Define a new AiStreamClient interface returning Flux<AiResponse>
- Refactor OpenAiClient and to use the new OpenAiApi and implement the AiStreamClient.
- Use spring-retry to improve the OpenAI EmbeddingClient stability on 503 error.
- Remove the OpenAI http header interceptor as the OpenAiApi returns ResponseEntity<T> that provides direct access to the headers.
- Refactor the metadata headers and usage extraction.
- Remove redundant and obsolete classes.
- Fix dependency issue with Pinecone, netty-codec-http2 and Spring Boot 3.2
- Add Vertex AI Autoconfigurations for chat and embedding clients.
- Factor out the embeding client dimensions() computation into an abstract parent AbstractEmbeddingClient.
- Add ITs
- Vertex dos.
- Rename Generation#text to content and info to properties.
- Make Generation extend the AbstractMessage and default to ASSISTANT message type.
- Add vevertex-ai project with native api client for generation and embedding.
- Add unit and IT tests for the vertex-ai native client.
- Add AiClient and EmbeddingClient implementation for the the Vertex AI along with ITs.
- Add Weaviate Vector Store implementation.
- Implement a converter of portable Filter.Expressions into native, Weaviate GraphQL Were expressions.
- Support for Weaviater schema auto-registration of filtarable metadata fields.
- Add auto-configration, spring properties and tests.
- WeaviateVectorStore ITs.
- Add README.md
Resolves#100
* Define GenerationMetadata property in AiResponse.
* Add OpenAI implementations of AiMetadata, RateLimit and Usage interfaces.
* Add REST Assured JsonPath dependency to spring-ai-openai module.
* Add OkHttp dependency to spring-ai-openai module.
* Add OkHttp Interceptor to parse OpenAI rate limit metadata from HTTP headers.
* Add OkHttp MockWebServer dependency to spring-ai-openai module, test scope
* Add Jakarta Servlet API dependency to spring-ai-openai module, test scope
* Add Spring Web MVC dependency to spring-ai-open-ai module., test scope
* Define OpenAI API response headers in an Enum.
* Add OpenAI test configuration using mock objects.
* Add integration test to assert successful extraction of OpenAI API response metadata.
* Include Spring Boot auto-configuration for (conditional) OpenAI metadata collection.
* Edit documentation and include information on AI metadata collected by Spring AI.
* Provide AI metadata implementation for Microsoft Azure OpenAI Service.
* Capture optional PromptMetadata in AiResponse.
* Define metadata for an AI generation choice.
* Capture AI choice metadata in Generation.
* Integrate ChoiceMetadata into AiResponse returned by OpenAI.
Fixes#98
- Use Azure AI Search end point to implement the VectorStore interface.
- Add ITs and README.
- Create/update vector index on after properties set.
- Add boot auto-configuration and ITs.
- Add boot starter for the vector store.
Resolves: #82
- Implement ChromaApi client, based on Chroma REST API.
- Implement ChromaVectorStore, including support for filter expression conversion.
- Common VectorStoreUtil class to share to/from Float/Double list/array convertion as well as Json/Map convertions.
- Add ITs including for Basic Auth and Token autheticatios.
- Add ChromaApi security support for BasicAuth and Token.
- Fix an issue with Text filter expression parser, related to double-quoted identifiers.
- Add Chroma README.md.
- Add Chroma boot autoconfiguration
Resolves# #86
- Collapses all VectorStore similiaritySearch methdos into one with SearchRequest builder.
- Fix all affected code and tests.
- Bump the project version to 0.7.1.
- Add tests
- Add autoconfigurations for milvus, pinecone and pgvecor stores.
- Improve and unify the VectorStore ITs.
- Make use of TrasformersEmbeddingClient for auto-configurations ITs.
- Based on the official pinecone java library.
Later expects that indices are created externally via Ops.
- Map Document metadata to and from Pinecone's internal Struct.
Later converts the metadata into pinecone json format.
- Add integration tests and README.
- EmbeddingClient implementation that computes, locally, sentence embeddings with SBERT transformers.
- Uses pre-trained transformer models, serialized into Open Neural Network Exchange (ONNX) format.
- Deep Java Library and the Microsoft ONNX Java Runtime are used to run
the ONNX models and compute the embeddings efficiently.
- Add default tokenizer.json and model.onnx for sentence-transformers/all-MiniLM-L6-v2.
- Add, configurable resource caching service to allow caching
remote (http/https) resources to the local FS.
- README.md provides information on how to serialize ONNX models.
- add Git LFS configuration for large onnx model files.
- Provides a rudimentary text extractions for multitude of document formats,
including PDF, Word Doc/Docx PowerPoint ppt/pptx and many more.
- Generates a single Document for the extracted text.
- No pre or post processing and cleansing for the text.
- Move the ExtractedTextFormatter from pdf reader to the core reader to enable reusability. Improve the tika reader
- Add a set of PDF readers fro per-page (PagePdfDocumentReader) and per-paragraph (ParagraphPdfDocumentReader) readers.
- Use a PDFLayoutTextStripper fork and PDFLayoutTextStripperByArea extension to preserve the structure of the extracted document.
- PdfDocumentReaderConfig and PageExtractedTextFormatter in standalone classes.
- Craeate a new document-readers top level model and the pdf-reader under.
- Implement dimensions method
- Add MetadataMode support. Defaults to EMBED
- Drop the pgml extension between tests.
- Disable the PostgresMlEmbeddingClientIT by default.
Resolves#33
- Add ContentFormatter and DefaultContentFormatter that can filter the metadata
and format the Document metadata and text according to predefined templates.
- Add content formatter tests
- Allow the TextSplitter to copy the document content-formatter to the children.
When the splitter breaks the parent Document into multiple chunks (e.g.
into a list of children Documents) copy the source content formatter to
the chunks by default. Use the copyContentFormatter flag to enable/disable copping.
- Add TextSplitter IT tests
- Add MetadataExtractors as DocumentTransformers.
- Bump spring-ai project version to 0.7.0-SNAPSHOT
- Configurable metadata-mode for EmbeddingClients
- Make the metadata mode configurable for the EmbeddingClient implementations.
- Use the EMBED mode by default.
Resolves#44
This commit brings support for Neo4j graph database in general,
and uses the vector index functionality available since version 5.11.
Aligned with the existing PgVector store and its tests.
The module creates indexes, if needed, for the vector search and
the identifier of the document object.
Add neo4j to vectordb docs page
Co-authored-by: Michael Simons <michael@simons.ac>
Resolves#19
* Clear and stream line. Inject search distance response as metadata
* Fix similarity threshold by metric type
- Fix the similary search with threshold for L2 and IP types.
- Create config Builder Config (inspired by Neo4jVectorStore impelementation.
- add database-name, metric-type, index-type and index-param configurations.
* update vector store doc and readme
