Spring Better documentation of HTTP streaming responses with Webflux & Reactor Netty

I've tried Stackoverflow for this but didn't receive an answer. I open this issue for two reasons - to ask whether what I try to do is possible (and how?), and to raise the fact that there is a lack of documentation for Webflux's HTTP streaming. I think the answer to this basic question should be added to the documentation of HTTP streaming with Webflux (a use case that fits Webflux very well).

Currently, Webflux doesn't include an example for HTTP streaming, and on the internet that's pretty easy to find simple streaming use cases of returning String/ POJOs. It's hard to find examples for returning bytes. My use case is that I want to create a Webflux server that has a route that receives some parameters, sends a request to another server that streams a .zip file, and streams its response to the user. The third-party streaming API works properly, but I'm not sure how can I return its response to the user with Webflux and Reactor Netty.

I've tried:

@GetMapping(value = "/zip/stream")
public WebClient.ResponseSpec getZipStream(@RequestParam String myParam) {
    Mono<ZipRequestBody> requestBody = getRequestBody(myParam);
    return WebClient.create(appConfig.getRemoteApiUrl())
            .post().uri("/download")
            .body(BodyInserters.fromProducer(requestBody, ZipRemoteEntry.class))
            .accept(MediaType.APPLICATION_OCTET_STREAM)
            .retrieve();
}

Which (with justification) results:

org.springframework.core.codec.CodecException: Type definition error: [simple type, class org.springframework.web.reactive.function.client.DefaultWebClient$DefaultResponseSpec]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.springframework.web.reactive.function.client.DefaultWebClient$DefaultResponseSpec and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)

I've also tried that with DataBuffer (that has a built-in codec):

@GetMapping(value = "/zip/stream")
public Flux<DataBuffer> getZipStream(@RequestParam String myParam) {
    Mono<ZipRequestBody> requestBody = getRequestBody(myParam);
    return WebClient.create(appConfig.getRemoteApiUrl())
            .post().uri("/download")
            .body(BodyInserters.fromProducer(requestBody, ZipRemoteEntry.class))
            .accept(MediaType.APPLICATION_OCTET_STREAM)
            .retrieve()
            .bodyToFlux(DataBuffer.class);
}

That resulted (same when using ByteBuffer or ByteBufFlux):

2022-06-21 21:17:46.000 ERROR 32492 --- [ctor-http-nio-2] a.w.r.e.AbstractErrorWebExceptionHandler : [dc0db90c-1]  500 Server Error for HTTP GET "/zip/stream?myParam=--"

org.springframework.web.reactive.function.client.WebClientResponseException$BadRequest: 400 Bad Request from POST https://my-test-server/download
    at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:196) ~[spring-webflux-5.3.18.jar:5.3.18]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 

That also makes sense, since I didn't really use Netty properly (I haven't closed the buffer since I'm not sure how to do this with Reactor Netty & Webflux).

I'd like to know what is the correct way of streaming data like zip. I know I probably need to stream DataBuffer or ByteBuffer kind of object, but I didn't manage to find how to do that with Reactor Netty and Webflux.

Comment From: bclozel

Let's continue this discussion on your StackOverflow question.