Spring Cloud Netflix DefaultEurekaClientHttpRequestFactorySupplier HttpClient is not closed

Describe the bug spring cloud 2021.0.0 Eureka Client has a bug, the bug is that every time the heartbeat and request Eureka Server will create a new HttpClient, but this new HttpClient will not be manually closed, only when the FullGC will be cleaned up, so it leads to the problem of too much TCP CLOSE_WAIT.

Sample The solution is to provide a shared HttpClient, rather than re-creating a new one for each request.

Comment From: OlgaMaciaszek

Hello, @Ive4 thanks for submitting this. I'm not sure what you mean. In the CloudEurekaClient that handles publishing the HeartbeatEvent, this method is called that should reuse an existing EurekaHttpClient. Could you please point me to the code you have in mind or provide a minimal, complete, verifiable example that reproduces the issue along with the steps to reproduce?

Comment From: Ive4

Sorry, I didn't make it clear. Eureka publishing HeartbeatEvent will invoke RedirectingEurekaHttpClient.execute. EurekaClient will shutdown when an exception occurs on an http request, but not shutdown HttpClient

Comment From: OlgaMaciaszek

Hello, @Ive4 if I understand correctly, the issue concerns the RedirectingEurekaHttpClient itself rather than its Spring integration provided by Spring Cloud? If so, please create an issue in the Eureka repo instead.

Comment From: spring-cloud-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-cloud-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Comment From: BenEfrati

Hi @OlgaMaciaszek , @Ive4 is right

EurekaClient will shutdown when an exception occurs on an http request, but not shutdown HttpClient

This bug is related to https://github.com/spring-cloud/spring-cloud-netflix/blob/27ac3379f6b391ccd605d212512f15c439825ecb/spring-cloud-netflix-eureka-client/src/main/java/org/springframework/cloud/netflix/eureka/http/RestTemplateEurekaHttpClient.java#L203

In case of exception here: https://github.com/Netflix/eureka/blob/ed0da19ca1c049c87e3dbf75b6015c1861d5c2d0/eureka-client/src/main/java/com/netflix/discovery/shared/transport/decorator/RedirectingEurekaHttpClient.java#L96 new HttpClient will be created without closing the existing one - this causes CLOSE_WAIT connections

I'll try to explain the issue, let me know if you still want complete mvce. This supplier creates new CloseableHttpClient for every call to https://github.com/spring-cloud/spring-cloud-netflix/blob/27ac3379f6b391ccd605d212512f15c439825ecb/spring-cloud-netflix-eureka-client/src/main/java/org/springframework/cloud/netflix/eureka/http/RestTemplateTransportClientFactory.java#L103

public class DefaultEurekaClientHttpRequestFactorySupplier implements EurekaClientHttpRequestFactorySupplier {

    @Override
    public ClientHttpRequestFactory get(SSLContext sslContext, @Nullable HostnameVerifier hostnameVerifier) {
        HttpClientBuilder httpClientBuilder = HttpClients.custom();
        if (sslContext != null) {
            httpClientBuilder = httpClientBuilder.setSSLContext(sslContext);
        }
        if (hostnameVerifier != null) {
            httpClientBuilder = httpClientBuilder.setSSLHostnameVerifier(hostnameVerifier);
        }
        CloseableHttpClient httpClient = httpClientBuilder.build();
        HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
        requestFactory.setHttpClient(httpClient);
        return requestFactory;
    }

}

so in case of shutdown, currentEurekaClient shutdown don't closes connections: https://github.com/spring-cloud/spring-cloud-netflix/blob/27ac3379f6b391ccd605d212512f15c439825ecb/spring-cloud-netflix-eureka-client/src/main/java/org/springframework/cloud/netflix/eureka/http/RestTemplateEurekaHttpClient.java#L203

as @Ive4 said, only Full GC closes the opened connections.

possible solution will be trying to close the HttpClient: shutdown could be

public void shutdown() {
        Optional.of(unwrapRequestFactoryIfNecessary(restTemplate.getRequestFactory()))
                .filter(HttpComponentsClientHttpRequestFactory.class::isInstance)
                .map(HttpComponentsClientHttpRequestFactory.class::cast)
                .ifPresent(requestFactory-> {
                    try {
                        requestFactory.destroy();
                    } catch (Exception e) {
                    }
                });
    }

unwrapRequestFactoryIfNecessary https://github.com/spring-projects/spring-boot/blob/47516b50c39bd6ea924a1f6720ce6d4a71088651/spring-boot-project/spring-boot/src/main/java/org/springframework/boot/web/client/RestTemplateBuilder.java#L746

spring-projects/spring-boot#31075