- Implement dimensions method
- Add MetadataMode support. Defaults to EMBED
- Drop the pgml extension between tests.
- Disable the PostgresMlEmbeddingClientIT by default.
Resolves#33
- Add ContentFormatter and DefaultContentFormatter that can filter the metadata
and format the Document metadata and text according to predefined templates.
- Add content formatter tests
- Allow the TextSplitter to copy the document content-formatter to the children.
When the splitter breaks the parent Document into multiple chunks (e.g.
into a list of children Documents) copy the source content formatter to
the chunks by default. Use the copyContentFormatter flag to enable/disable copping.
- Add TextSplitter IT tests
- Add MetadataExtractors as DocumentTransformers.
- Bump spring-ai project version to 0.7.0-SNAPSHOT
- Configurable metadata-mode for EmbeddingClients
- Make the metadata mode configurable for the EmbeddingClient implementations.
- Use the EMBED mode by default.
Resolves#44
Leverage #28 to allow the vector stores to resolve the embedding dimensions dynamically.
The explicitly set dimensions (if set) precedence over other configurations.
If the embedding dimensions are not explicitly set, the embeddingClient is used
to determine them dynamically. If the client fails the it falls back to 1536.
- Ensures that the vector store similarity search by threshold tests
use dynamically computed threshold that is between the top 2 results
from the ordered search. This ensures that the threshold value is not
affected by changes in the embedding API results.
- Minor code style improvments.
This commit brings support for Neo4j graph database in general,
and uses the vector index functionality available since version 5.11.
Aligned with the existing PgVector store and its tests.
The module creates indexes, if needed, for the vector search and
the identifier of the document object.
Add neo4j to vectordb docs page
Co-authored-by: Michael Simons <michael@simons.ac>
- Add dimension method to the EmbeddingClient interface.
Default dimension implementation uses the embed method to produce results and counts the result dimensions.
- Add EmbeddingUtil#dimensions utilities that look up the model dimensions from a pre-defined (static) file.
If the requested model is unknown, fallback to the default behaviour.
- Override the dimensions method in the OpenAiEmbeddingClient and AzureOpenAiEmbeddingClient to implement local caching.
- Add unit and IT tests.
Resolves#28
Resolves#19
* Clear and stream line. Inject search distance response as metadata
* Fix similarity threshold by metric type
- Fix the similary search with threshold for L2 and IP types.
- Create config Builder Config (inspired by Neo4jVectorStore impelementation.
- add database-name, metric-type, index-type and index-param configurations.
* update vector store doc and readme
- PgVectorStore implements the SmartLifecycle to initialize & clean the DB.
- Fix a bogus threshlold similarity search and add test for it.
- Add an internal embeddinDistance method.
