Spring Body token not expected error when trying to upload a large multipart file

I observe not an entirely deterministic issue while uploading big multipart files. It does not occur for small files. This error appears in the server logs:

java.lang.IllegalStateException: Body token not expected
at org.springframework.http.codec.multipart.PartGenerator$WritingFileState.body(PartGenerator.java:744) ~[spring-web-5.3.23.jar:5.3.23]

I created a minimal reproducible example to illustrate this issue.

This is the endpoint (full source code available here):

@RestController
class FilesController {

    @PostMapping(value = "/files")
    Mono<Integer> uploadFiles(@RequestBody Flux<Part> parts) {
        return parts
                .filter(FilePart.class::isInstance)
                .map(FilePart.class::cast)
                .flatMap(part -> DataBufferUtils.join(part.content())
                        .map(buffer -> {
                            byte[] data = new byte[buffer.readableByteCount()];
                            buffer.read(data);
                            DataBufferUtils.release(buffer);
                            return data;
                        })
                )
                .map(bytes -> bytes.length)
                .reduce(0, Integer::sum);
    }
}

And these are the tests (full source code available here):

@SpringBootTest
class BigMultipartFileUploadTest {

    private static final int NUMBER_OF_RESOURCES = 30;
    private WebTestClient webTestClient;

    @BeforeEach
    void before(ApplicationContext applicationContext) {
        webTestClient = WebTestClient.bindToApplicationContext(applicationContext).build();
    }

    /**
     * This test passes all the time
     */
    @Test
    void shouldUploadSmallFile() {
        // given
        int resourceSize = 5_000;
        var body = buildMultiValueMap(resourceSize);

        // expect
        sendPostRequest(body).isEqualTo(resourceSize * NUMBER_OF_RESOURCES);
    }

    /**
     * This test fails most of the time
     */
    @Test
    void shouldUploadBigFile() {
        // given
        int resourceSize = 5_000_000;
        var body = buildMultiValueMap(resourceSize);

        // expect
        sendPostRequest(body).isEqualTo(resourceSize * NUMBER_OF_RESOURCES);
    }

    private WebTestClient.BodySpec<Integer, ?> sendPostRequest(MultiValueMap<String, HttpEntity<?>> body) {
        return webTestClient.post()
                .uri("/files")
                .contentType(MediaType.MULTIPART_FORM_DATA)
                .body(BodyInserters.fromMultipartData(body))
                .exchange()
                .expectBody(Integer.class);
    }

    private MultiValueMap<String, HttpEntity<?>> buildMultiValueMap(int resourceSize) {
        MultipartBodyBuilder builder = new MultipartBodyBuilder();
        for (int i = 0; i < NUMBER_OF_RESOURCES; i++) {
            builder.part("item-" + i, newByteArrayResource(i, resourceSize));
        }
        return builder.build();
    }

    private ByteArrayResource newByteArrayResource(int fileIndex, int resourceSize) {
        return new ByteArrayResource(RandomUtils.nextBytes(resourceSize)) {
            @Override
            public String getFilename() {
                return "filename-" + fileIndex;
            }
        };
    }
}

As you can see, the only difference between these two tests is the resourceSize. The first test always works, and the second one doesn't work most of the time.

Interestingly, this problem does not occur in the Spring Boot version 2.3.12.RELEASE, so it looks like a regression.

Comment From: karolkrasnowski

I don't want this to sound rude, but when could I expect someone to take care of this? If this cannot be resolved in the foreseeable future, we will have to schedule a downgrade, so I'd like to know what we should prepare for.

Comment From: karolkrasnowski

@poutsma thanks a lot for fixing this!