Today I was using Spring AI and ran into a problem with org.springframework.ai.vectorstore.MilvusVectorStore. In method add(List documents), when going through each document separately, a request embeddingModel.embed(document); is sent for each of them. I have 4,000 documents in my case, so the method takes a very long time (50 minutes).

To speed up the process, I extended from MilvusVectorStore and already after executing the loop over the documents I used the List> embed(List texts) method from org.springframework.ai.embedding.EmbeddingModel with contentArray, which significantly accelerated execution (now performance lasts 1 minute).

Comment From: sobychacko

@solenyk Based on the ideas in this PR, we decided to make some API changes that are overarching to how we embed documents and then use those embeddings in other vector stores. At the core of those changes is a new BatchingStrategy API. Take a look at this PR: https://github.com/spring-projects/spring-ai/pull/1215

These changes were included in the recently released M2. Currently, the batching call is only applied to the Milvus vector store. We are going to evaluate other vector stores for similar usage.

Thank you for the PR; it certainly geared us toward providing it as a general pattern that other vector stores can use. I am closing the PR now. More PR contributions are welcomed!