Spring-ai Remove use of Supplier/Consumer in org.springframework.ai.document and replace with Function

The use of Supplier/Consumer was found to be limiting in a few scenarios. In the case of loading documents from a Resource, the use of a Supplier made the class stateful, meaning a new class instance was needed per Resource. Instead using a Functional interface so that the Resource is the input makes it easier to process many files.

So move from

interface DocumentReader extends Supplier<List<Document>>

interface DocumentReader<I> extends Function<I, List<Document>>

and have an implementation that takes a Spring Resource as input:

public class ResourceNewDocumentReader implements NewDocumentReader<Resource> {
    @Override
    public List<Document> apply(Resource resource) {
        ...
    }
}

The case of consumer is a bit less clear, perhaps the current DocumentWriter can still exist, but we could have our current implementations that write to the vector store implement DocumentTransformer so that if we wanted to chain together writin to multiple vector stores, the output from one could be changed to another. Maybe DocumentProcessor would be better then instead of DocumentTransformer

Using Void in a function instead of consumer is also an option, though it looks a little odd now that I write it out.

public interface DocumentWriter extends Consumer<List<Document>>

public interface DocumentCreator<O> extends Function<List<Document>, O> {

}

and

public class DocumentWriter implements DocumentCreator<Void>{
    @Override
    public Void apply(List<Document> documents) {
        return null;
    }
}

Comment From: markpollack

Perhaps we do not need to remove them, but for sure we need to support a ResourceDocumentReader like above to make it easy to load in many resources in a loop in as simple a way as possible.

Comment From: markpollack

Changed mind, will implement https://github.com/spring-projects/spring-ai/issues/221 and keep Supplier and Consumer in the ETL design. Note that spring cloud function adapter supplier and consumer to function.

Comment From: markpollack