Creating an issue here as suggested in https://github.com/reactor/reactor-core/issues/2300#issuecomment-669253301

Motivation

When working with binary data (bytes), it is sometimes necessary to send the data in chunks of exact size, with the possible exception for the last one, of course. Note that I'm not talking about TCP packets, but data sent by the client to the server without a POJO abstraction. This turns out to be surprisingly difficult to achieve in a non-blocking manner using the WebClient. The closest I've come is the following:

HttpClient.create()
        .httpResponseDecoder { it.maxChunkSize(chunkSize) }

However, this doesn't guarantee that all chunks are exactly of size chunkSize, it only guarantees that the chunk size is less than or equal to chunkSize.

Desired solution

  1. Provide a BodyExtractor that creates a Flux<DataBuffer>, where each DataBuffer contains exactly the number of bytes specified. It'll be the responsibility of the BodyExtractor to resize the packets it receives from Netty.
  2. Provide an encoder on the TcpClient that can send chunks of exact size.

Considered alternatives

Short of writing my own Netty handlers (which isn't difficult, except for the pesky reference counting), the only available client is I could find that could do this is Ktor Http Client. Ktor is developed by JetBrains mostly for the server side, and to say their documentation is poor is an understatement. They are also not as battle-tested as WebClient, so I'd like to be able to use WebClient if possible.

Additional context

One of the use cases for this feature is streaming audio files. Uncompressed format like WAV comes with a header (44 bytes for WAV), that the recipient may not want. Using exact-size chunks would let us drop the header and only transmit the data. Netty provides the LengthFieldBasedFrameDecoder for this exact purpose.

Comment From: kaladhar-mummadi

@asarkar The provided it.maxChunkSize(size) solution is being deprecated in netty and reactor-netty in upcoming versions. netty: https://github.com/netty/netty/issues/8430 reactor-netty: https://github.com/reactor/reactor-netty/pull/2249

Without this we wrote a bunch of custom code to keep track of incoming DataBuffers and split them based on some size, it would be useful to have this utility. This can be a simple util that splits already read data.

Comment From: asarkar

@kaladhar-mummadi I agree. If you read my post again, it.maxChunkSize was never a solution for this use case.

Comment From: rstoyanchev

@asarkar thanks for details and context. Why not make use of the LengthFieldBasedFrameDecoder but inserting into the Netty channel pipeline?

Comment From: asarkar

Unfortunately, after 3 years, I no longer remember what was done.

Comment From: snicoll

I am reviewing this and wonder what the next step should be. @rstoyanchev is this still actionable?

Comment From: spring-projects-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-projects-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.