Spring org.apache.tomcat.util.net.NioEndpoint : Failed to register socket with selector from poller

I'm using WebClient to make requests to a downstream service. With Webflux running it on tomcat.

  @Bean
  public WebClient webClient(
      @Value("${http.client.connection-timeout-millis}") final int connectionTimeoutMillis,
      @Value("${http.client.socket-timeout-millis}") final int socketTimeoutMillis,
      @Value("${http.client.wire-tap-enabled}") final boolean wireTapEnabled,
      final ObjectMapper objectMapper) {

    Consumer<Connection> doOnConnectedConsumer = connection ->
        connection
            .addHandler(new ReadTimeoutHandler(socketTimeoutMillis, MILLISECONDS))
            .addHandler(new WriteTimeoutHandler(connectionTimeoutMillis, MILLISECONDS));

    TcpClient tcpClient = TcpClient.newConnection()
        .wiretap(wireTapEnabled)
        .option(CONNECT_TIMEOUT_MILLIS, connectionTimeoutMillis)
        .doOnConnected(doOnConnectedConsumer);

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient).compress(true)))
        .exchangeStrategies(customExchangeStrategies(objectMapper))
        .build();
  } 

// ..........

    MultiValueMap<String, String> params = getParams(t1, t2);

    return webClient.get()
        .uri(HttpUtils.buildUrl(serviceUrl, params, name))
        .exchange()
        .flatMap(this::handleClientResponse)
        .onErrorMap(Exception.class, ex -> handleUnexpectedEx(ex, name, params));
  }

Everything works fine, but im running gatling performance tests, and sending multiple requests to my service, that is sending requests to my downstream services.

when i run the performance tests, most of the requests return with 200 and success. but most of times one of the requests throw the log entry:

2019-07-24 17:35:23.230 ERROR [-,,,] 28076 --- [25-ClientPoller] org.apache.tomcat.util.net.NioEndpoint   : Failed to register socket with selector from poller
java.nio.channels.ClosedChannelException: null
    at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:206)
    at org.apache.tomcat.util.net.NioEndpoint$PollerEvent.run(NioEndpoint.java:506)
    at org.apache.tomcat.util.net.NioEndpoint$Poller.events(NioEndpoint.java:631)
    at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:729)
    at java.base/java.lang.Thread.run(Thread.java:834)

and it basically close the connection with the gatling client without return any response, no errors, nothing. it just close the connection between client and server.

i'm trying to get the logs with wiretap enable, but its kinda tricky to reproduce this error, because sometimes happen, sometimes don't

any clue what can be causing this problem? i think it might be related with the way Tomcat handles the pooled connections. but again i have no clue.

Comment From: rstoyanchev

@xDouglasx hard to tell but the stack trace is quite close to the socket, so I would guess it's an issue for Tomcat, especially if you can pass on to them an isolated reproducible sample.

Comment From: xDouglasx

Hi, Thanks for replying, i will redirect this issue to tomcat, but as i said, its hard to create a sample and reproduce this behavior, because i have no clue why it is happening, and it happens sometimes, sometimes works fine, it makes even harder to identify the problem.