Spring Cloud Netflix LoadBalanced RestTemplate does not work well When apache httpclient or OkHttp is enabled

Describe the bug When enabled apache httpclient，the configuration in HttpClientRibbonConfiguration is well, but I did not find where RibbonLoadBalancingHttpClient is used in RestTemplate。The RestTemplate is annotated with @LoadBalanced。

RestTemplate finally uses SimpleClientHttpRequestFactory with HttpURLConnection to make HTTP requests。I don't know if this is a bug。

Sample ① pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.3.RELEASE</version>
    </parent>
    <artifactId>demo-consumer</artifactId>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Hoxton.SR8</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
        </dependency>
    </dependencies>
</project>

② The basic application.yml:

ribbon:
  ReadTimeout: 1000
  ConnectTimeout: 1000
  OkToRetryOnAllOperations: false
  MaxAutoRetries: 1
  MaxAutoRetriesNextServer: 1
  httpclient:
    enabled: true
  restclient:
    enabled: false
  okhttp:
    enabled: false

③ Then use the load-balanced RestTemplate to make http requests

@Bean
@LoadBalanced
public RestTemplate restTemplate() {
    return new RestTemplate();
}

ResponseEntity<String> result = restTemplate.getForEntity("http://demo-producer/v1/uuid", String.class);

④ You can see that the bean object of RibbonLoadBalancingHttpClient has been created.

⑤ The request will finally enter InterceptingClientHttpRequest，can see that it uses SimpleClientHttpRequestFactory

Comment From: spencergibb

Please add spring-cloud-starter-netflix-eureka-ribbon as a dependency.

Comment From: bojiangzhou

Please add spring-cloud-starter-netflix-eureka-ribbon as a dependency.

Hello, from mvnrepository and spring-cloud-netflix, the dependency of spring-cloud-starter-netflix-eureka-ribbon is not found, do you mean spring-cloud-starter-netflix-eureka-client or spring-cloud-starter-netflix-ribbon ? I have added these two dependencies.

Comment From: spencergibb

Sorry, typo it's that later. Did that help?

Comment From: bojiangzhou

Sorry, typo it's that later. Did that help?

No, still the same problem

Comment From: bojiangzhou

I think that when httpclient or okhttp is enabled, the requestFactory of RestTemplate needs to be modified，and then use RibbonLoadBalancingHttpClient or OkHttpLoadBalancingClient.

E.g. httpclient:

public class ApacheClientHttpRequest extends RibbonHttpRequest {

    private final URI uri;

    private final HttpMethod httpMethod;

    private final String serviceId;

    private final RibbonLoadBalancingHttpClient client;

    private final IClientConfig config;

    private final boolean retryable;

    public ApacheClientHttpRequest(URI uri,
                                   HttpMethod httpMethod,
                                   String serviceId,
                                   RibbonLoadBalancingHttpClient client,
                                   IClientConfig config,
                                   boolean retryable) {
        super(uri, null, null, config);
        this.uri = uri;
        this.httpMethod = httpMethod;
        this.serviceId = serviceId;
        this.client = client;
        this.config = config;
        this.retryable = retryable;
        if (retryable && !(client instanceof RetryableRibbonLoadBalancingHttpClient)) {
            throw new IllegalArgumentException("Retryable client must be RetryableRibbonLoadBalancingHttpClient");
        }
    }

    @Override
    protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
        try {
            RibbonApacheHttpRequest request = new RibbonApacheHttpRequest(buildCommandContext(headers));

            HttpResponse response;
            if (retryable) {
                response = client.execute(request, config);
            } else {
                response = client.executeWithLoadBalancer(request, config);
            }

            return new RibbonHttpResponse(response);
        } catch (Exception e) {
            throw new IOException(e);
        }
    }

    protected RibbonCommandContext buildCommandContext(HttpHeaders headers) throws IOException {
        ByteArrayInputStream requestEntity = null;
        ByteArrayOutputStream bufferedOutput = (ByteArrayOutputStream) this.getBodyInternal(headers);
        long contentLength = 0L;
        if (bufferedOutput != null) {
            byte[] bytes = bufferedOutput.toByteArray();
            contentLength = bytes.length;
            requestEntity = new ByteArrayInputStream(bytes);
            bufferedOutput.close();
        }

        return new RibbonCommandContext(serviceId, httpMethod.name(), uri.toString(), retryable,
                headers, new LinkedMultiValueMap<>(), requestEntity, new ArrayList<>(), contentLength);
    }
}

public class ApacheClientHttpRequestFactory extends HttpComponentsClientHttpRequestFactory {

    private final SpringClientFactory clientFactory;
    private final boolean retryable;

    public ApacheClientHttpRequestFactory(SpringClientFactory clientFactory, boolean retryable) {
        this.clientFactory = clientFactory;
        this.retryable = retryable;
    }

    @Override
    @NonNull
    public ClientHttpRequest createRequest(URI originalUri, HttpMethod httpMethod) throws IOException {
        String serviceId = originalUri.getHost();
        if (serviceId == null) {
            throw new IOException(
                    "Invalid hostname in the URI [" + originalUri.toASCIIString() + "]");
        }
        IClientConfig clientConfig = this.clientFactory.getClientConfig(serviceId);
        RibbonLoadBalancingHttpClient httpClient = this.clientFactory.getClient(serviceId, RibbonLoadBalancingHttpClient.class);

        return new ApacheClientHttpRequest(originalUri, httpMethod, serviceId, httpClient, clientConfig, retryable);
    }
}

@Configuration
@ConditionalOnClass(RestTemplate.class)
@AutoConfigureAfter(name = "org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration")
@AutoConfigureBefore({LoadBalancerAutoConfiguration.class, AsyncLoadBalancerAutoConfiguration.class})
public class ApacheClientHttpRequestFactoryConfiguration {

    @Configuration(proxyBeanMethods = false)
    @ConditionalOnClass(HttpClient.class)
    @ConditionalOnProperty(name = "ribbon.httpclient.enabled", matchIfMissing = true)
    static class ClientHttpRequestFactoryConfiguration {

        @Autowired
        private SpringClientFactory springClientFactory;

        @Bean
        @ConditionalOnMissingBean
        public RestTemplateCustomizer restTemplateCustomizer(
                final ApacheClientHttpRequestFactory apacheClientHttpRequestFactory) {
            return restTemplate -> {
                // set requestFactory
                restTemplate.setRequestFactory(apacheClientHttpRequestFactory);

                // Remove the Content-Length request header, otherwise an error will be reported. See org.apache.http.protocol.RequestContent#process
                ClientHttpRequestInterceptor removeHeaderLenInterceptor = (request, bytes, execution) -> {
                    request.getHeaders().remove(HTTP.CONTENT_LEN);
                    return execution.execute(request, bytes);
                };

                List<ClientHttpRequestInterceptor> interceptors = new ArrayList<>(restTemplate.getInterceptors());
                interceptors.add(removeHeaderLenInterceptor);
                restTemplate.setInterceptors(interceptors);
            };
        }

        @Bean
        @ConditionalOnMissingClass("org.springframework.retry.support.RetryTemplate")
        public ApacheClientHttpRequestFactory apacheClientHttpRequestFactory() {
            return new ApacheClientHttpRequestFactory(springClientFactory, false);
        }

        @Bean
        @ConditionalOnClass(name = "org.springframework.retry.support.RetryTemplate")
        public ApacheClientHttpRequestFactory retryableApacheClientHttpRequestFactory() {
            return new ApacheClientHttpRequestFactory(springClientFactory, true);
        }
    }
}

Comment From: spencergibb

I think that when httpclient or okhttp is enabled, the requestFactory of RestTemplate needs to be modified，and then use RibbonLoadBalancingHttpClient or OkHttpLoadBalancingClient.

Yes, we do that when you create an @LoadBalanced RestTemplate.

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

Comment From: bojiangzhou

The sample file: demo-consumer.zip

You can run demo-comsumer and request http://localhost:8020/v1/test in your browser. The request will enter the execute of InterceptingClientHttpRequest, and you will find that the type of requestFactory is SimpleClientHttpRequestFactory.

Comment From: OlgaMaciaszek

Hi, @bojiangzhou. That, in fact, does not work, since the LoadBalancerInterceptor looks for instances of LoadBalancerClient and RibbonLoadBalancingHttpClient does not implement that interfaces. However, according to the docs, the HttpClient-based requests have only been added for Zuul and not @LoadBalanced RestTemplate objects. Thus, it would be considered an enhancement and not a bug and as this module has entered maintenance mode, the Spring Cloud team will no longer be adding new features to the module. We will fix blocker bugs and security issues, and we will also consider and review small pull requests from the community.