• When embedding documents, allow batching the documents using some criteria.
  • BatchingStrategy interface with a TokenCountBatchingStrategy implementation that uses the openai max input token size of 8191 as the default.
  • Add a default method in EmbeddingModel to embed document using this new batching strategy.
  • Change MilvusVectorStore to make use of this new batching API.
  • Adding unit tests for TokenCountBatchingStrategy.
  • Adding openai integration test to call the embed API that uses batching.

Resolves https://github.com/spring-projects/spring-ai/issues/1214

Comment From: markpollack

merged in 949f1ed4e80345fe45acb0ba75de32f09989f111