BufferingClientHttpRequestFactory
creates requests which are not repeatable
A RestTemplate
obtained form the following snippet will lose the default request configuration applied to the HttpComponentsClientHttpRequestFactory
, in this case the setRedirectsEnabled(true)
is not honored.
- Affected Version: spring-boot 3.2.0-M3
- Works with stable: spring-boot 3.1.4
- Affected http client library is Apache HttpComponents 5 (
HttpComponentsClientHttpRequestFactory
).
I prepared a reproducible example here. Just run com.example.httpclient5demo.Httpclient5DemoApplicationTests
RestTemplate restTemplate = restTemplateBuilder
.requestFactory(() -> {
RequestConfig.Builder requestConfig = RequestConfig.custom()
.setRedirectsEnabled(true);
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create()
.setDefaultRequestConfig(requestConfig.build());
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClientBuilder.build());
//return requestFactory;
return new BufferingClientHttpRequestFactory(requestFactory);
})
.rootUri("https://httpbin.org")
.build();
String response = restTemplate.getForObject("/redirect-to?url=https://httpbin.org/base64/SFRUUEJJTiBpcyBhd2Vzb21l", String.class);
assertThat(response).isEqualTo("HTTPBIN is awesome");
Comment From: wilkinsona
Thanks for the report and sample project. This doesn't appear to be a Spring Boot problem. The following tests illustrate the behavior without using Spring Boot:
package com.example.httpclient5demo;
import static org.assertj.core.api.Assertions.assertThat;
import org.apache.hc.client5.http.config.RequestConfig;
import org.apache.hc.client5.http.impl.classic.HttpClientBuilder;
import org.junit.jupiter.api.Test;
import org.springframework.http.client.BufferingClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
class Httpclient5DemoApplicationTests {
@Test
void redirectsCanBeEnabled() {
RestTemplate restTemplate = new RestTemplate();
RequestConfig.Builder requestConfig = RequestConfig.custom()
.setRedirectsEnabled(true);
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create()
.setDefaultRequestConfig(requestConfig.build());
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClientBuilder.build());
restTemplate.setRequestFactory(requestFactory);
String response = restTemplate.getForObject("https://httpbin.org/redirect-to?url=https://httpbin.org/base64/SFRUUEJJTiBpcyBhd2Vzb21l", String.class);
assertThat(response).isEqualTo("HTTPBIN is awesome");
}
@Test
void redirectsCanBeEnabledWhenBuffering() {
RestTemplate restTemplate = new RestTemplate();
RequestConfig.Builder requestConfig = RequestConfig.custom()
.setRedirectsEnabled(true);
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create()
.setDefaultRequestConfig(requestConfig.build());
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClientBuilder.build());
restTemplate.setRequestFactory(new BufferingClientHttpRequestFactory(requestFactory));
String response = restTemplate.getForObject("https://httpbin.org/redirect-to?url=https://httpbin.org/base64/SFRUUEJJTiBpcyBhd2Vzb21l", String.class);
assertThat(response).isEqualTo("HTTPBIN is awesome");
}
}
redirectsCanBeEnabled
passes but redirectsCanBeEnabledWhenBuffering
fails. We'll transfer this to the Framework team so that they can take a look.
Comment From: cachescrubber
Thank you @wilkinsona for analyzing the issue. I updated the example with your test cases.
Comment From: cachescrubber
I spotted another issue. Basic Auth (401) and Proxy Auth (407) Challenges are not working when buffering is enabled.
@Test
void basicAuthCanBeEnabledWhenBuffering() {
RestTemplate restTemplate = new RestTemplate();
BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope("httpbin.org", 443),
new UsernamePasswordCredentials("demo", "secret".toCharArray()));
HttpClientBuilder httpClientBuilder = HttpClientBuilder.create()
.setDefaultCredentialsProvider(credentialsProvider);
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClientBuilder.build());
restTemplate.setRequestFactory(new BufferingClientHttpRequestFactory(requestFactory));
ResponseEntity<String> forEntity = restTemplate.getForEntity("https://httpbin.org//basic-auth/demo/secret", String.class);
assertThat(forEntity.getStatusCode().is2xxSuccessful()).isTrue();
}
The example has been updated with a test case.
Comment From: cachescrubber
The issue is not a configuration issue, as I assumed initialy. I'll change the issue title accordingly.
It is caused by changes introduced in #30557 . BufferingClientHttpRequestWrapper
is creating an empty body, which is passed to HttpComponents as org.springframework.http.client.HttpComponentsClientHttpRequest.BodyEntity
which in turn implements org.apache.hc.core5.http.HttpEntity#isRepeatable
to return false
unconditionally.
HttpEntity#isRepeatable
returning false
disables all kind of functionality in the library where requests are repeated - redirects (3xx), authentication challenges (401,407) but also internal retries.
A quick fix would be to not pass in empty HttpComponentsClientHttpRequest.BodyEntity
for requests where no body is expected (GET, HEAD, ...).
But I think the changes in #30557 are problematic in this regard. As it is implemented right now, even requests using the BufferingClientHttpRequestFactory
are converted into a streaming body, even though they are in-memory anyway.
Comment From: cachescrubber
Thanks @poutsma , my HttpComponents based test are good again. From reading the code I think at least OkHttp3ClientHttpRequest.BodyRequestBody#isOneShot
need to be updated as well. Not sure for the other implementations.
Comment From: poutsma
Thanks for spotting that, it should now be fixed.
Comment From: poutsma
@cachescrubber I made some further related changes in 6dd93d4, which should enable repeatable writes for many types of request bodies, meaning that you typically don't need to wrap your request factory in a BufferingClientHttpRequestFactory
any more. Please try a 6.1.0 Spring Framework snapshot and see if it works for you.
Comment From: cachescrubber
Thanks for the heads-up. I use BufferingClientHttpRequestFactory
to implement request/response logging in a ClientHttpRequestInterceptor
. IIRC we needed the buffering in order to use response#getBody
multiple times.
Comment From: karlovskiy
Hi, I found this issue today during my investigation of redirects which stopped working after spring update.
I found that in this line we have
streamingOutputMessage.setBody(outputStream -> StreamUtils.copy(body, outputStream));
but if body was set like this with lambda than repeatable will
be false.
Is it intended behavior ?
I thought that if we have byte array in InterceptingClientHttpRequest
maybe body there also should be repeatable ?
PS: Currently, I resolved my redirect issue with wrapping HttpComponentsClientHttpRequestFactory
with BufferingClientHttpRequestFactory
.
Comment From: poutsma
@karlovskiy well spotted, this should be fixed in 6.1.4.