Commit Graph

240 Commits

Author SHA1 Message Date
Eddú Meléndez
e4607a6c29 Support ChromaDB's CHROMA_SERVER_AUTHN_CREDENTIALS env var in Docker Compose and Testcontainers
Read `CHROMA_SERVER_AUTHN_CREDENTIALS` env var from compose file and
Testcontainers definition to configure the connection details for
ChromaDB.
2024-06-19 16:08:57 +02:00
Eddú Meléndez
6ce998f9d3 Add support for Typesense Service Connection
Docker Compose and Testcontainers Service Connection support for
Typesense.
2024-06-19 15:59:49 +02:00
Christian Tzolov
b7475ed060 Update Weaviate docker version. remove import wildcard definitions 2024-06-18 09:33:30 +02:00
Christian Tzolov
2c7dc3d6f1 Pinecone: Add configurable content and distance metadata fields (#882)
- Add `withContentFieldName` and `withDistanceMetadataFieldName` to Pinecone Config Builder
 - Add `spring.ai.vectorstore.pinecone.contentFieldName` and `spring.ai.vectorstore.pinecone.distanceMetadataFieldName`
   properties to the auto-config
 - Default content field name: "document_content", distance field: "distance"
 - Update tests and docs

 Resolves #882
2024-06-18 07:57:15 +02:00
Lee-ChungMu
dfabcc7f61 Use List.of instead of Arrays.asList In the SEARCH_OUTPUT_FIELDS field.
The Arrays.asList method can change elements. SEARCH_OUTPUT_FIELDS is an immutable field, so it has been improved to util List.of.
2024-06-17 15:51:26 +02:00
Laura Trotta
78f3797f8d Update Elasticsearch Store: Use KNN insettad of script_score
- knn instead of script_score, removed initialization
 - only using normalized similarities, adjusted unit test
 - making l2norm's distances consistent with others
 - update dependency version and docs
 - upate autoconfigure ITs
2024-06-17 15:17:48 +02:00
Christian Tzolov
958549ecce Rename all Embedding Client doc and variables occurrences into Embedding Model 2024-06-16 21:24:03 +02:00
Eddú Meléndez
704dec20d9 Chroma Vector Store: replace RestTemplate by RestClient
- replace RestTemplate by  RestClient but default to SimpleClientHttpRequestFactory as Chroma seems to have issues with HTTP2.
 - update the documentation.
 - update the ghcr.io/chroma-core/chroma version to 0.5.0. Fix the withBasicAuthCredentials.

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
2024-06-16 21:07:03 +02:00
Jemin Huh
46e47849ce Add OpenSearch vector store integration
- implement OpensSearchVectorStore
 - add opensearch auto-configuration and boot starter
 - add documentation for OpenSearch VectorStore
 - add bom dependecies
 - align with to new Spirng AI API
2024-06-16 12:43:09 +02:00
Christian Tzolov
476164b6e3 Move the vector expression converters from core to their projects 2024-06-15 20:24:29 +02:00
PabloSanchi
3e2ed8b9bf Add Typesense vector store integration
- autoconfigure setup
 - add post bean initialization and create method
 - add embedding field
 - create collection add nested field options
 - add typesense tests
 - use embedding variable instead of word vec
 - check in runtime the number of documents in the collection
 - add typesense expression converter
 - add filter tests. add update document test and search with threshold test
 - distance threshold and add distance key into metadata
 - add typesesne boot starter
 - add typesense docs
 - add client properties in autoconfigure
 - add embedding dimension method
 - add typesense vector store autoconfiguration tests
 - add docs to nav.adoc and vectorsdb.adoc.
 - fix module name.
 - move the expression converter to the typesense project.
2024-06-15 19:55:05 +02:00
Christian Tzolov
1d479e3aab Minor PgVector ITs improvements 2024-06-15 17:49:21 +02:00
Christian Tzolov
baadf4cb53 Fix Oracle Vector Store dependencies
- force override the outdated SpringBoot Oracle version.
 - add auto-config tests.
2024-06-15 12:21:56 +02:00
Christian Tzolov
a5d92ec30f Adjust Oracle Vector store class names 2024-06-14 06:55:54 +02:00
LLEFEVRE
41cf693bc0 Add Support for Oracle 23ai as vector database
- add OracleVectorStore with metadata filter expression support.
 - add ITs using oracle-free-slim testcontainers.
 - add auto-configuration and boot starter.
 - add adoc documentation.
 - Adjust javadoc references.

 Resolves #703

 Co-authored-by: Eddú Meléndez Gonzales <eddu.melendez@gmail.com>
2024-06-14 06:46:01 +02:00
yinpeng
20c6a1adcf fix: typo 2024-06-03 15:00:19 +02:00
Mark Pollack
ac91302eed Next development version 2024-05-28 13:53:04 -04:00
Mark Pollack
0670575f3e Release version 1.0.0-M1 2024-05-28 13:49:11 -04:00
Josh Long
2d43e40024 Add property to initialize schema for vector stores
* Default is fale
* Update docs
2024-05-26 00:31:19 -04:00
skewgod
593e5fd747 docs(qdrant): added additional info to setup api key for qdrant (#736) 2024-05-24 11:36:26 -04:00
Mark Pollack
4a2ad60059 Package refactoring
* Move ChatClient and related classes into the chat.client package
* Move ChatModel and related class into the chat.model package
* Smaller refactorings to remove DSM cycles
* Update README.md
2024-05-23 17:52:27 -04:00
Josh Long
fbfc87e814 Refactoring of ChatClient to add fluent API and introduce Model as dependent object
* Rename the ModelClient class hierarchy into Model:
  - Rename ModelClient into Model. Update all code and doc references.
  - Rename ChatClient to ChatModel. Update all ChatClient suffixes and chatClient fields and variables in code and doc.
  - Rename EmbeddingClient into EmbeddingModel. Update the XxxEmbeddingClient class and variable suffixes and embeddingClient variables and fields in code and docs.
  - Rename ImageClient into ImageModel.
  - Rename SpeechClient into SpeechModel.
  - Rename TranscriptionClient into TranscriptionModel.
  - Update all javadocs and antora pages. Update the related diagrams.

* Create fluent API in ChatClient interface that now includes streaming support
* Add OpenAI FunctionCallbackWrapper2IT auto-config tests.
* Add ChatClientTest mockito testing.
* Add ChatModel#getDefaultOptions(), and remove @FunctionalInterface

* ChatModel enums extend the new ModelDescription interface.
* Implement fromOptions copy method in every ChatOptions implementation.
* Extend ChatClient to use the model default options if not provided explicitly.

* Update readme to provide guidance on how to adapt to breaking changes.

Co-authored-by: Christian Tzolov <ctzolov@vmware.com>
Co-authored-by: Mark Pollack <mpollack@vmware.com>
2024-05-22 16:07:12 -04:00
Dimitrios Begnis
bce45c2d2f Use <> for inequality operator in Neo4j filter expression implementation 2024-05-21 17:17:32 -04:00
Christian Tzolov
3475f17e98 Unify the vector store module and pom names
spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-cassandra -> spring-ai-cassandra-store
 spring-ai-pinecone -> spring-ai-pinecone-store
 spring-ai-redis -> spring-ai-redis-store
 spring-ai-qdrant -> spring-ai-qdrant-store
 spring-ai-gemfire -> spring-ai-gemfire-store
 spring-ai-azure-vector-store-spring-boot-starter -> spring-ai-azure-store-spring-boot-starter
 spring-ai-redis-spring-boot-starter -> spring-ai-redis-store-spring-boot-starter
2024-05-17 17:05:09 +02:00
Christian Tzolov
1985824fa9 clean code imports 2024-05-17 15:32:46 +02:00
Christian Tzolov
3ee04ed8da fix code style and missing dependency renaming 2024-05-17 15:31:58 +02:00
Josh Long
04d854cc49 make project names consistent 2024-05-17 15:31:58 +02:00
Josh Long
c5b7dd9c46 i ran the spring javaformat maven plugin to clean up any source code incompatabilities. 2024-05-17 15:31:57 +02:00
Josh Long
8c758617f3 moving to ApplicationListener<ApplicationReadyEvent> to avoid slowing down the application on startup. 2024-05-17 15:31:53 +02:00
Eddú Meléndez
d610dd6f1d Add default collection name to Qdrant
Add a default collection name similar to other vector store
implementations. Currently, when using starters, qdrant requires
a collection name. Otherwise, it fails.
2024-05-15 12:02:05 +02:00
Christian Tzolov
b0799babf4 Update Qdrant client version from 1.7.1 to 1.9.1
- Also update the qdrant/qdrant docker image version from v1.7.4 to v1.9.2
2024-05-15 06:19:46 +02:00
Eddú Meléndez
cdb80e83e9 Fix ChromaDB basic authentication
Currently, `username` is added as a password in `BasicAuthenticationInterceptor`.
This commit fixes the issue and also make sure the integration test
setup is correct.
2024-05-11 14:51:28 +02:00
Eddú Meléndez
5beef21a4e Expose QdrantClient and WeaviateClient as beans
Currently, `QdrantClient` and `WeaviateClient` are not exposed as
  beans. Having access to those would benefit to perform operations
  with an already configured client.

  - Deprecate  QdrantVectorStoreConfig.
  - Update Qdrant manual config adoc.
  - Improve Qdrant adoc.
  - Update Weaviate docs.
2024-05-10 15:15:14 +03:00
Christian Tzolov
6154fa663f Disable MondoDB IT due to #698 2024-05-09 13:27:57 +03:00
Mark Pollack
dfb8bf6a44 Add a new abstraction to simplify implementation of common ChatBot use cases
* Add ChatBot and basic DefaultChatBot
* Add streaming ChatBot support.
* Add Evaluator interface and RelevancyEvaluator implementation
* Add Content data type abstraction for Document and Message
* Renaming and package refactoring
* update .gitignore to allow node package name
* Add List<Media> to node and move ai.transformer package to ai.prompt.transformer
* Add Short/Long term memory support.
* Add mixing transformers support

Docs TBD
2024-05-08 15:14:50 -04:00
Christian Tzolov
670bf8e259 Adhere to Spring Framework code sytle
- Wildcard imports such as import java.util.* or import static org.assertj.core.api.Assertions.* are forbidden, even in test code.
   https://github.com/spring-projects/spring-framework/wiki/Code-Style#import-statements
2024-05-04 20:32:37 +03:00
mck
8a5f9dfb22 Add option to CassandraVectorStore to return embeddings in documents from similarity searches 2024-05-04 16:15:13 +02:00
mck
ac1a0d67e9 Avoid calling CREATE KEYSPACE|TABLE IF NOT EXISTS … unnecessarily
This also is a fix for AstraDB, which throws an exception if you try to `CREATE KEYSPACE IF NOT EXISTS …`
And use more compatible `USING 'StorageAttachedIndex'` index creation syntax.
2024-05-04 15:36:24 +02:00
wmz7year
f90c51a729 Improve ElasticsearchVectorStore,
- add index configuration and add support for ES response error handling.
 - rename dims to dimension propety.
 - add property javadocs
 - improve the elasticsearch javadoc.
2024-04-30 09:58:21 +03:00
Vaclav Bartacek
b9ba62507d Fix Neo4jVectorStore max allowed dimension size to 4096
- The Neo4j "vector-2.0" index provider allows up to 4096 dimensions.
  - Remove upper dimension boundry check
2024-04-28 20:30:06 +03:00
Christian Tzolov
9026263fd0 Fix javadoc after daf131be2c 2024-04-28 20:28:15 +03:00
mck
daf131be2c Fix column creation, when adding additional normal and embedding colums), and make index name unique (for when there are multiple vector indexes in the same keyspace)
And change stream to for-loop when converting List<Double> to Float[] for performance
2024-04-28 18:34:15 +03:00
Sagar Bhat
773b7bd3ca PgVectorStore: creating an index only if missing
- Resolve an issue where a new index keeps getting created during application start up.
 - Solution is is to create an index only if an index on the embedding column does not exist.
 - Add missing index name.
2024-04-26 19:03:45 +02:00
Ikko Eltociear Ashimine
db6f7cda3e Update AzureVectorStore.java
minor fix
2024-04-26 15:55:35 +02:00
omar.mahamid
6178b3dcc4 Fix CassandraVectorStoreConfig logger name
aligne logger name with the className
2024-04-20 07:40:01 +02:00
mck
0eaf7d05c9 Cassandra Vector Store initial impl follow up
- add concurrency to store.add(..) (bc embeddingClient is slow)
- CassandraVectorStoreAutoConfiguration uses CassandraAutoConfiguration
- driver profiles for production stability+performance,
- small cleanups and naming fixes,
- main doc tidy-up
- astradb compatibility (protocol V4)
– don't create embeddings again for documents that already have them
  similar to https://github.com/spring-projects/spring-ai/pull/413
2024-04-19 18:13:11 +02:00
Eddú Meléndez
eea3801fbd Remove @Testcontainers from HanaDB Tests
HanaDB tests don't rely on Testcontainers.
2024-04-10 17:31:28 +02:00
mck
656d238285 Implement Apache Cassandra vector store
The CassandraVectorStore is for managing and querying vector data in an Apache Cassandra db.
  It offers functionalities like adding, deleting, and performing similarity searches on documents.

  The store utilizes CQL to index and search vector data. It allows for custom metadata fields in
  the documents to be stored alongside the vector and content data.

  This class requires a CassandraVectorStoreConfig configuration object for initialization, which
  includes settings like connection details, index name, field names, etc. It also requires an
  EmbeddingClient to convert documents into embeddings before storing them.

  A schema matching the configuration is automatically created if it doesn't exist. Missing columns
  and indexes in existing tables will also be automatically created. Disable this with the disallowSchemaCreation.

  This class is designed to work with brand new tables that it creates for you, or on top of existing
  Cassandra tables. The latter is appropriate when wanting to keep data in place, creating embeddings
  next to it, and performing vector similarity searches in-situ.

  Instances of this class are not dynamic against server-side schema changes. If you change the schema
  server-side you need a new CassandraVectorStore instance.

 - Add auto-configure with tests.
 - reformat code style
 - Change field terminology to column (as appropriate for cassandra and cql)
 - Add doc page with an advanced example.
 - Add the dependencies to Spring AI BOM
 – add to `AutoConfiguration.imports`

 - Add @since annotation
 - Fix javadoc issue
 - Streamline the adoc content and layout
2024-04-10 16:51:47 +02:00
Christian Tzolov
0b04c8928a Update vector-stores README links 2024-04-09 20:55:39 +02:00
Rahul
466b824840 Add SAP HanaDB vector store integration
- Implement a HanaCloudVectorStore and tests
 - Implement Autoconfiguraiton + properties
 - Add boot starter
 - Update BOM with vector store and boot dependencies.
 - Add antora docuementation
 - added junit for HanaCloudVectorStoreProperties.java and documentation
   to create a BTP trial account and provision an instance for SAP Hana Cloud db
 - updated license, formatting and javadoc
 - IT for HanaCloudVectorStoreAutoConfiguration
 - IT for HanaCloudVectorStoreAutoConfiguration

Additional
 - add @AutoConfiguration(after = { JpaRepositoriesAutoConfiguration.class })
 - update the handa docs structure.
2024-04-09 12:36:42 +02:00