Spring-ai The document segmentation vectorization failed.

The reason is that there is a problem with the batching logic in the TokenCountBatchingStrategy:

public List> batch(List documents) { List> batches = new ArrayList(); int currentSize = 0; List currentBatch = new ArrayList();

int tokenCount; for(Iterator var5 = documents.iterator(); var5.hasNext(); currentSize += tokenCount) { Document document = (Document)var5.next(); tokenCount = this.tokenCountEstimator.estimate(document.getFormattedContent(this.contentFormater, this.metadataMode)); if (tokenCount > this.maxInputTokenCount) { throw new IllegalArgumentException("Tokens in a single document exceeds the maximum number of allowed input tokens"); }

if (currentSize + tokenCount > this.maxInputTokenCount) { batches.add(currentBatch); currentBatch.clear(); currentSize = 0; }

currentBatch.add(document); }

if (! currentBatch.isEmpty()) { batches.add(currentBatch); }

return batches; }

The following statement should be placed within the loop: if (! currentBatch.isEmpty()) { batches.add(currentBatch); }