After the webclient is used to create the persistent connection, when the server returns data, the returned data is not in JSON structure occasionally. However, the return data structure defined by the server is in JSON format. In addition, clients in other languages can normally read the JSON response.

related code as this:

webClient.get()
    .uri("/test/bugs")
    .accept(MediaType.APPLICATION_JSON)
    .exchangeToFlux(clientResponse -> {
        System.out.println("++++Response is " + clientResponse.statusCode());
        return clientResponse.body(BodyExtractors.toDataBuffers());
    })
    .subscribe(dataBuffer -> {
        byte[] bytes = new byte[dataBuffer.readableByteCount()];
        dataBuffer.read(bytes);
        DataBufferUtils.release(dataBuffer);
        String str = new String(bytes, StandardCharsets.UTF_8);
        System.out.println("++++Str is " + str);
        # some method convert str to object at here， occur Exception due to str is not a valid JSON.

    });

The problem symptom is as follows: normal repsonse like : [object1,object2, object3] exception response like: first print : [object1, obje second print : ct2, object3]

Motivation

Occasionally responds like this： first print : [object1, obje second print : ct2, object3]

Desired solution

I want to respond every time, like this : [object1,object2, object3],while server return data , but not block unitl last data package.

Considered alternatives

  1. Concatenate str A and B in subscribe to a valid JSON string C and parse the string C.
  2. I'm not sure if the webclient persistent connection is the mechanism that does not guarantee the integrity of the data read. In this case, I think clients in other languages may be more friendly, such as go-resty.
  3. I'm not sure if it's my problem with using webclient. Can anyone give me some advice?

Additional context

reactor-netty has same issue https://github.com/reactor/reactor-netty/issues/2870#issuecomment-1671208710

Comment From: rstoyanchev

This is expected, when you read low level chunks of data, you get them as they come. To aggregate, either decode to a single DataBuffer or String:

DataBuffer bodyBuffer = webClient.get()
        .uri("/test/bugs")
        .accept(MediaType.APPLICATION_JSON)
        .retrieve()
        .bodyToMono(DataBuffer.class) // or String.class
        .block();

or to higher level Objects:

List<Foo> fooList = webClient.get()
        .uri("/test/bugs")
        .accept(MediaType.APPLICATION_JSON)
        .retrieve()
        .bodyToFlux(Foo.class)
        .collectList()
        .block();