- Explain the need for batching due to embedding model thresholds
- Describe the BatchingStrategy interface and its purpose
- Detail the default TokenCountBatchingStrategy implementation
- Provide information on using and customizing batching strategies
- Mention that several vector stores are pre-configured with the default strategy
Comment From: markpollack
Added new constructor for custom TokenCountEstimator in TokenCountBatchingStrategy and null checks in ctors
updated doc with example usage of additional constructors
merged in 4c8a6ee8b0deec797901f1164cb63dc5340f0b84