I created a simple Spring Boot demo project for testing remote partitioning with spring-integration-kafka, I found that partition steps are executed sequentially:
2025-03-05T13:10:26.039+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition9] executed in 10s504ms
2025-03-05T13:10:36.561+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition4] executed in 10s504ms
2025-03-05T13:10:47.004+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition5] executed in 10s424ms
2025-03-05T13:10:57.413+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition6] executed in 10s392ms
2025-03-05T13:11:07.874+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition0] executed in 10s444ms
2025-03-05T13:11:18.320+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition1] executed in 10s427ms
2025-03-05T13:11:28.771+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition7] executed in 10s432ms
2025-03-05T13:11:39.225+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition2] executed in 10s438ms
2025-03-05T13:11:49.668+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition3] executed in 10s427ms
2025-03-05T13:12:00.120+08:00 INFO 34621 --- [ scheduling-1] o.s.batch.core.step.AbstractStep : Step: [workerStep:partition8] executed in 10s436ms
I'd like they are executed concurrently to reduce total execute time, It doesn't work even I configured spring.task.scheduling.pool.size > 1, because taskScheduler in IntegrationAutoConfiguration.IntegrationTaskSchedulerConfiguration is not used by default poller.
I'm beginner of Spring Integration, not sure it's overlooked or intentional.
Here is the demo project: batch-demo.zip