Spring Hundreds of reactor-http-nio-XXX threads being created when WebClient requests are made in parallel without pre-warming.

Ive discovered some strange behaviour where it appears WebClient + reactor-netty is creating up to hundreds of reactor-http-nio threads when we make parallel calls using WebClient, without performing any initial warmup. I've managed to reproduce this relatively easily with the following code where I submit tasks to an Executor service that uses the WebClient instance to call a remote service in parallel.

import org.junit.jupiter.api.Test;
import org.springframework.http.client.reactive.ReactorClientHttpConnector;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.netty.http.client.HttpClient;

import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Executors;

public class ParallelTest {

    @Test
    public void testBreakIt3() throws Exception {
        final WebClient webClient = WebClient.builder()
                .baseUrl("<the service>") // swap me
                .build();

        final ExecutorCompletionService service = new ExecutorCompletionService(Executors.newFixedThreadPool(5));

        final int iterations = 300;
        for (int i = 0; i < iterations; i++) {
            service.submit(() -> {
                webClient.get()
                        .uri("<the url>").   // swap me
                        .exchange()
                        .block();
                return null;
            });
        }

        int success = 0;
        int errors = 0;

        for (int i = 0; i < iterations; i++) {
            try {
                service.take().get();
                ++success;
            } catch (Exception ex) {
                ++errors;
            }
        }

        System.out.println("Errors: " + errors);
        System.out.println("Success:" + success);
    }
}

When DEBUG logging is enabled, we see many high range numbered reactor netty threads, e.g reactor-http-nio-45 reactor-http-nio-18

Attaching a profiler (Ive used Java Flight Recorder along with Intellij to inspect Java events), we notice many reactor-http-nio threads being started, well beyond the number we would expect based on our CPU.

If I change the above code to make a WebClient call before we enter the loop that makes many parallel calls, then we end up with a correct number of reactor threads (in this case, reactor-http-nio-1 through to reactor-http-nio-12).

If we use warmup() method, the problem goes away.

        HttpClient httpClient = HttpClient.create();
        httpClient.warmup().block();

        final WebClient webClient = WebClient.builder()
                .baseUrl("<the url>")
                .clientConnector(new ReactorClientHttpConnector(httpClient))
                .build();

Tested using: - spring-webflux 5.3.4 - reactor-netty 1.0.3

Comment From: violetagg

@gordz There should be exactly 12 (for your use case) running threads regardless of the number that they have in the name. Please check these: https://github.com/reactor/reactor-netty/pull/1426 https://github.com/reactor/reactor-netty/pull/1453

Comment From: spring-projects-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-projects-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.