Commit Graph

165 Commits

Author SHA1 Message Date
Christian Tzolov
e5410d7e8d undo last commit 2024-11-21 09:13:45 +01:00
Christian Tzolov
68f8113098 Prevent skipping integration tests by default 2024-11-21 08:01:05 +01:00
Mark Pollack
67a8896422 Next development version 2024-11-20 18:03:30 -05:00
Mark Pollack
33c05c399c Release version 1.0.0-M4 2024-11-20 18:02:47 -05:00
Mark Pollack
66886346fa Group together deps that are also managed by boot in pom.xml See issue #1212 2024-11-19 17:05:09 -05:00
Mark Pollack
a4462420d8 Add support for optional keyless authentication.
Implemented a new configuration property 'useKeylessAuth' to toggle
between API key and Azure default credential authentication.

Added necessary dependencies and updated tests to reflect the new changes.

co-authored-by: mattgotteiner <mattgotteiner@users.noreply.github.com>
2024-11-19 16:15:07 -05:00
Gareth Evans
a7b5652790 Upgrade azure-openai library to beta.12
- Correct merge utils after lib update
2024-11-19 15:11:23 -05:00
Thomas Vitale
263fe2fba7 Modular RAG - Query Analysis
Query Analysis
* Introduce Query Analysis Module
* Define QueryTransformer API and TranslationQueryTransformer implementation
* Define QueryExpander API and MultiQueryExpander implementation
* Support QueryTransformer in RetrievalAugmentationAdvisor (support for QueryExpander will be in the next PR together with the needed DocumentFuser API).

Improvements
* Refine Retrieval and Augmentation Modules for increased robustness
* Expand test coverage for both modules
* Define clone() method for ChatClient.Builder

Tests
* Introduce “spring-ai-integration-tests” for full-fledged integration tests
* Add integration tests for RAG modules
* Add integration tests for RAG advisor

Query Analysis
* Introduce Query Analysis Module
* Define QueryTransformer API and TranslationQueryTransformer implementation
* Define QueryExpander API and MultiQueryExpander implementation
* Support QueryTransformer in RetrievalAugmentationAdvisor (support for QueryExpander will be in the next PR together with the needed DocumentFuser API).

Improvements
* Refine Retrieval and Augmentation Modules for increased robustness
* Expand test coverage for both modules
* Define clone() method for ChatClient.Builder

Tests
* Introduce “spring-ai-integration-tests” for full-fledged integration tests
* Add integration tests for RAG modules
* Add integration tests for RAG advisor

Relates to #gh-1603

Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
2024-11-13 15:58:36 -05:00
Mark Pollack
1d540a6dd1 CI Actions imporvements 2024-11-12 15:24:23 -05:00
Mark Pollack
a98af02f62 Minimize time to run main CI build action
- Add maven properties for all vector stores such as
skip.vectorstore.azure-cosmos-db to control IT test execution
- Chroma and PGVector IT tests are enabled by default
- Docker Compose and Testcontainers module ITs are skipped by default
- Add parallel job to run docker-compose and testcontainers ITs
2024-11-11 16:19:08 -05:00
Christian Tzolov
5f6b892eda Add function calling support to invoke methods with dynamic arguments and return values
This change enables more flexible integration between Spring AI and LLM function
calling capabilities while maintaining type safety and ease of use.

- Add new MethodFunctionCallback class to support method invocation via reflection
 - Supports both static and non-static method calls
 - Handles multiple parameter types including primitives, objects, collections
 - Supports empty parameters and empty response
 - Auto-generates JSON schema from method parameters
 - Special handling for ToolContext parameters
 - Builder pattern for easy configuration

- Add comprehensive unit tests for MethodFunctionCallback
- Add integration tests for MethodFunctionCallback with both Anthropic and OpenAI clients
- Add jackson-module-jsonSchema dependency
- Modify FunctionCallback to check for empty tool context

Testing coverage includes:
- Static method invocation scenarios
- Non-static method calls with various parameter types
- Void return type methods
- Complex parameter types (enums, records, lists)
- Tool context handling
- Error cases and validation

Add MethodFunctionCallback reference docs
2024-11-10 22:58:40 -05:00
Anders Swanson
fb65ed01bd Add OCI GenAI Cohere Chat integration
Adds Oracle Cloud Infrastructure (OCI) Generative AI's Cohere chat model support
to expand Spring AI's cloud provider capabilities. This allows developers to use
OCI's managed Cohere models through both dedicated and on-demand serving modes.

The integration provides auto-configuration for simple setup while allowing full
customization of model parameters through OCICohereChatOptions. Teams can now
use OCI's Cohere models alongside other providers in Spring AI applications.

This change complements the existing OCI embedding support, offering a complete
set of GenAI capabilities for Oracle Cloud users.

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
2024-11-06 17:43:48 -05:00
Aleks Seovic
f6a648e85c Add Oracle Coherence vector store implementation
Integrates Oracle Coherence 24.09+ as a vector store backend for Spring AI. The
implementation provides:

- Support for vector similarity search with configurable distance metrics (Cosine,
  Inner Product, L2)
- Multiple indexing options including HNSW and Binary Quantization indexes
- Filter expression evaluation for metadata-based filtering
- Vector normalization capabilities
- Comprehensive test coverage including integration tests

The implementation includes:
- CoherenceVectorStore main implementation
- CoherenceFilterExpressionConverter for Spring AI to Coherence filter conversion
- Auto-configuration support via spring-boot-starter
- Documentation and usage examples

Requires Oracle Coherence 24.09 or later.
2024-11-06 16:07:20 -05:00
Ilayaperumal Gopinathan
d44a6aff7c GH-924 Remove PaLM API support
https://github.com/spring-projects/spring-ai/pull/1664

  - As a follow up to decommission the PaLM API, the PaLM API support is removed.
  - Remove the model and vector store classes
  - Remove the documentation entries
  - Reference: https://ai.google.dev/palm_docs/deprecation

Resolves #924
2024-11-04 15:22:50 -05:00
Christian Tzolov
0d2d4b7385 Add Bedrock Converse API chat model support
Introduces support for Amazon Bedrock Converse API through a new BedrockProxyChatModel
implementation. This enables integration with Bedrock's conversation models with features
including:
- Support for sync/async chat completions
- Stream response handling
- Tool/function calling capabilities
- System message support
- Image input support
- Observation and metrics integration
- Configurable model parameters and AWS credentials

Adds core support classes:
- BedrockUsage: Implements Usage interface for token tracking
- ConverseApiUtils: Utility class for handling Bedrock API responses including:
  - Tool use event aggregation and processing
  - Chat response transformation from stream outputs
  - Model options conversion
  - Support for metadata aggregation
- URLValidator: Utility for URL validation and normalization with support for:
  - Basic and strict URL validation
  - URL normalization
  - Multimodal input handling
- Enhanced FunctionCallingOptionsBuilder with merge capabilities for both ChatOptions
  and FunctionCallingOptions
- Added BEDROCK_CONVERSE to AiProvider enum for metrics tracking
- Extended AWS credentials support with session token capability
- Added configurable session token property to BedrockAwsConnectionProperties

Adds new auto-configuration support:
- BedrockConverseProxyChatAutoConfiguration for automatic setup of the Bedrock Converse chat model
- BedrockConverseProxyChatProperties for configuration including:
  - Model selection (defaults to Claude 3 Sonnet)
  - Timeout settings (defaults to 5 minutes)
  - Temperature and token control
  - Top-K and Top-P sampling parameters
- Integration with existing BedrockAwsConnectionConfiguration for AWS credentials

Updates to testing infrastructure:
- Adds comprehensive test suite for Bedrock Converse properties and auto-configuration
- Integration tests for chat completion and streaming scenarios
- Property validation tests for configuration options
- Temporarily disabled other Bedrock tests due to AWS quota limitations
- Added ObjectMapper configuration for proper JSON handling

Added new spring-ai-bedrock-converse-spring-boot-starter module

Updates module configuration in parent POM and BOM to include new bedrock-converse
modules and starters. Adds necessary auto-configuration imports for seamless integration
with Spring Boot applications.

Unrelated changes:
- Disabled several Bedrock model tests (Jurassic2, Llama, Titan) due to AWS quota limitations
- Disabled PaLM2 tests due to API decommissioning by Google

Resolves #809, #802

Add docs and fix configs

- Move timeout configuration from chat properties to connection properties
- Add comprehensive documentation for Bedrock Converse API usage and configuration
- Update tests to reflect configuration changes

Co-authored-by: maxjiang153 <maxjiang153@users.noreply.github.com>

Standardize AWS credential handling in integration tests

- Improve how we manage AWS credentials across our integration test
suite and ensures consistent test configuration. We're replacing individual
environment variable checks with @RequiresAwsCredentials
annotation and standardizing the use of BedrockTestUtils for context creation
in tests

We also align all AWS regions to US_EAST_1 for consistency and add missing
dependency versioning for Oracle Free.

These changes make our AWS tests more easier to maintain.

Key changes:
- Replace @EnabledIfEnvironmentVariable with @RequiresAwsCredentials
- Standardize context creation via BedrockTestUtils
- Set AWS region to US_EAST_1
- Add Oracle Free dependency version in pom.xml
2024-11-03 17:10:24 -05:00
leevh0908
3fc03233c0 Milvus vector store: Pass databaseName in DeleteParam for non-default databases
When using a non-default database, MilvusVectorStore's doDelete method wasn't
passing the specified databaseName in DeleteParam, causing searches to fall back
to the default database where collections couldn't be found. Added explicit
databaseName parameter to fix this issue.

- Add databaseName to DeleteParam builder
- Upgrade milvus-sdk from 2.3.4 to 2.3.5
2024-10-30 20:37:55 -04:00
jito
1e980d13b2 Azure cosmos-db vector store maven cleanup (#1631)
- Define the version for Azure Cosmos DB
- Add dependencies for the azure-cosmos-db-store module and boot starter
- Include the artifacts in spring-ai-bom

Signed-off-by: jitokim <pigberger70@gmail.com>
2024-10-30 17:07:13 -04:00
jitokim
6d2b22fae3 Add spring-ai-starter-azure-cosmos-db-store submodule to pom.xml
Signed-off-by: jitokim <pigberger70@gmail.com>
2024-10-29 17:31:32 -04:00
Sébastien Deleuze
8c0c2bae08 Introduce Kotlin tests
This commit configures Maven to support compiling Kotlin
main and test classes, and adds a JacksonUtilsKotlinTests
class that ensures that serialization and deserialization
of Kotlin data classes works as expected.

The Maven configuration follows recommendations from
https://kotlinlang.org/docs/maven.html.
2024-10-29 17:26:44 -04:00
Mark Pollack
3f85f792c2 fix cassandra tests 2024-10-24 20:31:19 -04:00
Soby Chacko
8e758dbd00 Introduce checkstyle plugin
- Based on https://github.com/spring-io/spring-javaformat
- In this iteration, checkstyles are only enabled for spring-ai-core
2024-10-24 16:43:59 -04:00
Theo van Kraay
7b06fcf98b Add Azure CosmosDB vector store support
- Implement core vector store module for CosmosDB integration
- Add Spring Boot auto-configuration capabilities
- Integrate batch processing strategy for optimized operations
- Include comprehensive tests for core and auto-config modules
- Add reference docs for the CosmosDB vector store support
2024-10-22 15:50:16 -04:00
Christian Tzolov
1252216f3a Revert antora-component-version-maven-plugin 2024-10-20 13:39:05 +02:00
Mark Pollack
d0c53392aa Update spring-javaformat to version 0.0.43
- Ran ./mvnw spring-javaformat:apply that made some new formatting changes
2024-10-16 15:50:04 -04:00
Christian Tzolov
2e0a51fac5 Streamline dependencies and improve API consistency
- Remove unnecessary spring-web dependencies
 - Update third-party library versions
 - Refactor API classes to use consistent header handling
 - Remove ApiUtils class and inline its functionality
 - Adjust RestClient and WebClient builder usage in autoconfiguration
 - Replace direct RestClient.Builder injections with ObjectProvider<RestClient.Builder>
   and WebClient.Builder injections with ObjectProvider<WebClient.Builder>
 - Update ChromaVectorStoreAutoConfiguration to use ObjectProvider
 - Rename MongoDbAtlasLocalContainerConnectionDetailsFactoryTest to IT
 - Switch spring-ai-chroma-store dependency from spring-web to spring-webflux
 - Simplify ChromaApi constructor by using method reference for default headers
 - Adjust import order

 Resolves #1066
 Resolves #524
2024-10-16 15:46:03 -04:00
Mark Pollack
4c83fe8302 Guard against NPE in ZhiPu embedding model
- Update retry test to pass - needs investigation
2024-10-08 23:37:00 +02:00
Mark Pollack
4a892b5269 Release version 1.0.0-M3 2024-10-08 23:18:50 +02:00
Mark Pollack
300561af53 Update to use Boot 3.3.4 2024-10-08 12:54:40 +02:00
Anders Swanson
ccf190c77c Add OCI GenAI embedding model support
This commit introduces support for Oracle Cloud Infrastructure (OCI)
GenAI embedding models in Spring AI. It includes:

* New OCIEmbeddingModel class for interacting with OCI GenAI API
* Auto-configuration for easy setup and integration
* Properties for configuring OCI connection and embedding options
* Documentation updates explaining usage and configuration
* Integration tests to verify functionality

Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
2024-09-26 17:04:05 -04:00
Christian Tzolov
8bddd5f531 Update DJL to 0.30.0 2024-09-18 10:39:23 +02:00
Christian Tzolov
df17f08f5b Udate onnx-runtime to 1.19.2 and djl to 0.29.0 2024-09-11 18:07:12 +02:00
Mark Pollack
3c9ae4e1d6 fix javadoc build in CI 2024-09-09 16:51:40 -04:00
Christian Tzolov
5b6a60db9a Disable javadock lint 2024-09-07 12:26:48 +02:00
Christian Tzolov
da5ad96d57 fix javadoc erros 2024-09-05 21:58:09 +02:00
Mark Pollack
e1884d1d92 Next development version 2024-08-23 18:47:37 -04:00
Mark Pollack
43ad2bdb97 Release version 1.0.0-M2 2024-08-23 18:46:58 -04:00
Mark Pollack
2f46f04983 disable javadoc linting 2024-08-23 12:55:47 -04:00
Mark Pollack
449790f73a Update to Boot 3.3.3
Add to docs that spring ai supports 3.2.x and 3.3.x
2024-08-23 12:37:01 -04:00
Piotr Olaszewski
56e678c487 Add Markdown document reader with enhanced features
This commit introduces a new Markdown document reader with several
key features and improvements:

* Add support for text with various formatting elements
* Implement handling for horizontal rules and hard line breaks
* Add functionality for inline and block code sections
* Incorporate blockquote handling
* Support ordered and unordered lists
* Introduce additional metadata capabilities
* Include JavaDocs

Update ETL documentation to reflect these new features and usage.

Fixes #105
2024-08-22 13:35:44 -04:00
Eddú Meléndez
80007d4d6c Update testcontainers version to 1.20.1
Weaviate module contains the wait strategy.
2024-08-07 09:25:58 +02:00
David
3e920faede Bump org.springframework.cloud:spring-cloud-function-context from 4.1.1 to 4.1.3
Updated version resolves CVE-2024-22271
2024-07-30 13:05:37 +02:00
Piotr Olaszewski
0e97f9c579 Introduce AzureOpenAI transcription support
- Breaking changes: Classes from the org.springframework.ai.openai.metadata.audio.transcription package have been moved to the org.springframework.ai.audio.transcription package.
 - The AzureOpenAiAudioTranscriptionModel has been added to the auto-configuration.
 - The spring.ai.azure.openai.audio.transcription prefix was introduced for properties.
 - Introduces options properties which cover all of them (see: AzureOpenAiAudioTranscriptionOptions).
 - fix missing MutableResponseMetadata
 - add docs
 - adjust code to updated ResponseMetadata design
 - add test to AzureOpenAiAutoConfiguration
 - add missing AzureOpenAiAudioTranscriptionModel tests
2024-07-22 14:08:11 +02:00
David
a5682f752d Update jtokkit from 1.0.0 to 1.1.0
Adds o200k_base encoding used by gpt4o and gpt4o-mini
2024-07-20 15:24:37 +02:00
mck
4aacab020b Implement CassandraChatMemory
- provides a flexible schema, can be combined with a vector store, and supports time-to-live rows.
 - fix initialize-schema docs and so that it actually works.
 - move CommonVectorStoreProperties to .vectorstore. package
 - add CassandraAutoConfiguration to the AutoConfiguration.imports
2024-07-17 11:42:09 +02:00
Christian Tzolov
c70c20b79d Add VertexAI Embedding Model support
- add new spring-ai-vertex-ai-embedding project.
 - add VertexAiTextEmbeddingModel and VertexAiMultimodalEmbeddingMode with related options configuration classes.
 - add ITs
 - add auto-configuraiton and boot starters.
 - register to BOM.
 - add documentation.
 - add multimodal embedding documentation
 - extend the Embedding metdata so that it can keep references to the source document's data, Id, mediatype

 Resolves #1013
 Related to #1009
2024-07-10 15:36:13 -04:00
Christian Tzolov
61a56db98b Update Azure OpenAI to 1.0.0-beta.10 2024-07-09 13:08:42 +02:00
Christian Tzolov
dd22342c0c Improve ONNX Tansformer Embedding Model
- Fix model output name handling.
 - Improve documentation.
2024-07-09 11:21:53 +02:00
Nicholas Zhan
5417d19d9a Fix prompt template validation if a variable name contains hyphen 2024-07-02 11:13:54 -04:00
GR
997e01ca6b Add support for QianFan AI models
- Add chat, embeddin and image models
- Tests
- Docs
2024-06-22 13:30:38 -04:00
Christian Tzolov
f9f6247f7e AWS/Bedrock housekeeping updates
- upadate bedrockruntime version to 2.26.7 and align it with aws sdk dependecy.
 - minor ITs configuration, consistency and statbility fixes
   Use the EnvironmentVariableCredentialsProvider instead of the Profile credential provider.
 - update the Bedrock getting started documentation.
 - update the Genemi Prerequisites docs.
2024-06-22 12:22:10 +02:00