This commit adds comprehensive documentation for the BatchingStrategy
in vector stores and enhances the TokenCountBatchingStrategy class.
Key changes:
- Explain batching necessity due to embedding model thresholds
- Describe BatchingStrategy interface and its purpose
- Detail TokenCountBatchingStrategy default implementation
- Provide guidance on using and customizing batching strategies
- Note pre-configured vector stores with default strategy
- Add new constructor for custom TokenCountEstimator in
TokenCountBatchingStrategy
- Implement null checks with Spring's Assert utility
- Update docs with new customization options and code examples
4c8a6ee8b0