Issue with Feign client GET request call with query PARAM having special characters are failing with below exception
Spring version : 2.3.4.RELEASE Spring Cloud version : Hoxton.SR8 Java version : open jdk 11 Environment : Kubernates cluster environment with microservices. Cluster setup is with IPV6. All the communiation between microservices via IPV6.
Debugging details:
Kubernates setup with two nodes where local eureka server is configured as discovery server.
Two microservices should be configured with IPV6. Configurations provided below. application.yml server: address: fe80::1126:88ba:243:2db0 port: 8003
eureka: client: serviceUrl: defaultZone: ${EUREKA_URI:http://[fe80::1126:88ba:243:2db0]:8761/eureka} instance: ip-address: fe80::1126:88ba:243:2db0 preferIpAddress: true
JVM arguments to make IPV6 use by application. -Djava.net.preferIPv4Stack=false -Djava.net.preferIPv6Addresses=true
Test case schenario :
Request received from Frontend to one of the microservice say "Service A". Microservice Service A required information from Microservice Service B. To get the data from Microservice Service B, we are makign a Feign Client call from Microservice Service A. Request URL from Service A to Service B is: http://serviceB/api/v1/checkEmailUnique?email=sampleemail@gmail.com
When we make a call, java trace as
Step1 :
BlockingLoadBalancerClient.reconstructURI(BlockingLoadBalancerClient.java:75)
@Override
public URI reconstructURI(ServiceInstance serviceInstance, URI original) {
return LoadBalancerUriTools.reconstructURI(serviceInstance, original);
}
Here serviceInstance has information about the IP address (IPV6 : 240b:c0e0:202:5e2b:b424:2:0:4b4), port (8001) and service name (serviceB). original : http://serviceB/api/v1/checkEmailUnique?email=sampleemail%40gmail.com
Step2 :
/**
* Modifies the URI in order to redirect the request to a service instance of choice.
* @param serviceInstance the {@link ServiceInstance} to redirect the request to.
* @param original the {@link URI} from the original request
* @return the modified {@link URI}
*/
public static URI reconstructURI(ServiceInstance serviceInstance, URI original) {
if (serviceInstance == null) {
throw new IllegalArgumentException("Service Instance cannot be null.");
}
return doReconstructURI(serviceInstance, original);
}
Step3 :
private static URI doReconstructURI(ServiceInstance serviceInstance, URI original) {
String host = serviceInstance.getHost();
String scheme = Optional.ofNullable(serviceInstance.getScheme())
.orElse(computeScheme(original, serviceInstance));
int port = computePort(serviceInstance.getPort(), scheme);
if (Objects.equals(host, original.getHost()) && port == original.getPort()
&& Objects.equals(scheme, original.getScheme())) {
return original;
}
boolean encoded = containsEncodedParts(original);
return UriComponentsBuilder.fromUri(original).scheme(scheme).host(host).port(port)
.build(encoded).toUri();
}
In above code, encoded value is requred as true because url is encloded. boolean encoded = containsEncodedParts(original);
Step 4 :
public UriComponents build(boolean encoded) {
return buildInternal(encoded ?
EncodingHint.FULLY_ENCODED :
this.encodeTemplate ? EncodingHint.ENCODE_TEMPLATE : EncodingHint.NONE);
}
Step 5: EncodingHint.FULLY_ENCODED send
private UriComponents buildInternal(EncodingHint hint) {
UriComponents result;
if (this.ssp != null) {
result = new OpaqueUriComponents(this.scheme, this.ssp, this.fragment);
}
else {
HierarchicalUriComponents uric = new HierarchicalUriComponents(this.scheme, this.fragment,
this.userInfo, this.host, this.port, this.pathBuilder.build(), this.queryParams,
hint == EncodingHint.FULLY_ENCODED);
result = hint == EncodingHint.ENCODE_TEMPLATE ? uric.encodeTemplate(this.charset) : uric;
}
if (!this.uriVariables.isEmpty()) {
result = result.expand(name -> this.uriVariables.getOrDefault(name, UriTemplateVariables.SKIP_VALUE));
}
return result;
}
Step 6: Because encoding value is true its calls a method verify()... below...
HierarchicalUriComponents uric = new HierarchicalUriComponents(this.scheme, this.fragment,
this.userInfo, this.host, this.port, this.pathBuilder.build(), this.queryParams,
hint == EncodingHint.FULLY_ENCODED);
/**
* Package-private constructor. All arguments are optional, and can be {@code null}.
* @param scheme the scheme
* @param userInfo the user info
* @param host the host
* @param port the port
* @param path the path
* @param query the query parameters
* @param fragment the fragment
* @param encoded whether the components are already encoded
*/
HierarchicalUriComponents(@Nullable String scheme, @Nullable String fragment, @Nullable String userInfo,
@Nullable String host, @Nullable String port, @Nullable PathComponent path,
@Nullable MultiValueMap<String, String> query, boolean encoded) {
super(scheme, fragment);
this.userInfo = userInfo;
this.host = host;
this.port = port;
this.path = path != null ? path : NULL_PATH_COMPONENT;
this.queryParams = query != null ? CollectionUtils.unmodifiableMultiValueMap(query) : EMPTY_QUERY_PARAMS;
this.encodeState = encoded ? EncodeState.FULLY_ENCODED : EncodeState.RAW;
// Check for illegal characters..
if (encoded) {
verify();
}
}
Step 7: In verify method. verifyUriComponent(this.host, getHostType());
/**
* Check if any of the URI components contain any illegal characters.
* @throws IllegalArgumentException if any component has illegal characters
*/
private void verify() {
verifyUriComponent(getScheme(), Type.SCHEME);
verifyUriComponent(this.userInfo, Type.USER_INFO);
verifyUriComponent(this.host, getHostType());
this.path.verify();
this.queryParams.forEach((key, values) -> {
verifyUriComponent(key, Type.QUERY_PARAM);
for (String value : values) {
verifyUriComponent(value, Type.QUERY_PARAM);
}
});
verifyUriComponent(getFragment(), Type.FRAGMENT);
}
Step 8: Here host IP is : 240b:c0e0:202:5e2b:b424:2:0:4b4 and this doesn't have "[" hence its treated as IPV4 address.
private Type getHostType() {
return (this.host != null && this.host.startsWith("[") ? Type.HOST_IPV6 : Type.HOST_IPV4);
}
Step 9: verifyUriComponent method throws below exception....
verifyUriComponent(this.host, getHostType());
private static void verifyUriComponent(@Nullable String source, Type type) {
if (source == null) {
return;
}
int length = source.length();
for (int i = 0; i < length; i++) {
char ch = source.charAt(i);
if (ch == '%') {
if ((i + 2) < length) {
char hex1 = source.charAt(i + 1);
char hex2 = source.charAt(i + 2);
int u = Character.digit(hex1, 16);
int l = Character.digit(hex2, 16);
if (u == -1 || l == -1) {
throw new IllegalArgumentException("Invalid encoded sequence \"" +
source.substring(i) + "\"");
}
i += 2;
}
else {
throw new IllegalArgumentException("Invalid encoded sequence \"" +
source.substring(i) + "\"");
}
}
else if (!type.isAllowed(ch)) {
throw new IllegalArgumentException("Invalid character '" + ch + "' for " +
type.name() + " in \"" + source + "\"");
}
}
}
Exception stack trace:
2020-12-18 08:50:53.488 DEBUG [,bf7c6d79a8c83a34,bf543439eefefbf5,true] 10 --- [r-management-24] c.f.TraceFeignBlockingLoadBalancerClient : Before send
2020-12-18 08:50:53.489 DEBUG [,bf7c6d79a8c83a34,bf543439eefefbf5,true] 10 --- [r-management-24] c.f.TraceFeignBlockingLoadBalancerClient : Exception thrown
java.lang.IllegalArgumentException: Invalid character ':' for HOST_IPV4 in "240b:c0e0:202:5e2b:b424:2:0:4b4"
at org.springframework.web.util.HierarchicalUriComponents.verifyUriComponent(HierarchicalUriComponents.java:416)
at org.springframework.web.util.HierarchicalUriComponents.verify(HierarchicalUriComponents.java:379)
at org.springframework.web.util.HierarchicalUriComponents.<init>(HierarchicalUriComponents.java:145)
at org.springframework.web.util.UriComponentsBuilder.buildInternal(UriComponentsBuilder.java:417)
at org.springframework.web.util.UriComponentsBuilder.build(UriComponentsBuilder.java:406)
at org.springframework.cloud.client.loadbalancer.LoadBalancerUriTools.doReconstructURI(LoadBalancerUriTools.java:111)
at org.springframework.cloud.client.loadbalancer.LoadBalancerUriTools.reconstructURI(LoadBalancerUriTools.java:95)
at org.springframework.cloud.loadbalancer.blocking.client.BlockingLoadBalancerClient.reconstructURI(BlockingLoadBalancerClient.java:75)
at org.springframework.cloud.openfeign.loadbalancer.FeignBlockingLoadBalancerClient.execute(FeignBlockingLoadBalancerClient.java:73)
at org.springframework.cloud.sleuth.instrument.web.client.feign.TraceFeignBlockingLoadBalancerClient.execute(TraceFeignBlockingLoadBalancerClient.java:76)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:119)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:89)
at feign.hystrix.HystrixInvocationHandler$1.run(HystrixInvocationHandler.java:109)
at com.netflix.hystrix.HystrixCommand$2.call(HystrixCommand.java:302)
at com.netflix.hystrix.HystrixCommand$2.call(HystrixCommand.java:298)
at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46)
at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51)
at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
at rx.Observable.unsafeSubscribe(Observable.java:10327)
at rx.internal.operators.OperatorSubscribeOn$SubscribeOnSubscriber.call(OperatorSubscribeOn.java:100)
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction$1.call(HystrixContexSchedulerAction.java:56)
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction$1.call(HystrixContexSchedulerAction.java:47)
at org.springframework.cloud.sleuth.instrument.async.TraceCallable.call(TraceCallable.java:71)
at com.netflix.hystrix.strategy.concurrency.HystrixContexSchedulerAction.call(HystrixContexSchedulerAction.java:69)
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
2020-12-18 08:50:53.490 DEBUG [,bf7c6d79a8c83a34,bf7c6d79a8c83a34,true] 10 --- [nio-8003-exec-2] .m.m.a.ExceptionHandlerExceptionResolver : Using @ExceptionHandler com.mavenir.dep.accountmanagement.exception.CustomResponseEntityExceptionHandler#handleHystrixRuntimeException(HystrixRuntimeException, WebRequest)
2020-12-18 08:50:53.491 ERROR [,bf7c6d79a8c83a34,bf7c6d79a8c83a34,true] 10 --- [nio-8003-exec-2] a.e.CustomResponseEntityExceptionHandler : Hystrix Runtime Exception caused by :
Comment From: rstoyanchev
I've edited your comment to improve the formatting. You might want to check out this Mastering Markdown guide for future reference. Especially when posting a very long description, increase your chances of getting help by using eligible formatting.
Comment From: rstoyanchev
According to the URI spec the syntax for IPV6 requires surrounding "[" and "]" and therefore HierarchicalUriComponents is correct to reject it.
This could be an issue of application misconfiguration or it could be a bug in https://github.com/spring-cloud/spring-cloud-openfeign, I'm not sure which one, but there is nothing I can see that we can solve at the Spring Framework level.
Comment From: vasureddys
Thank you and please close this ticket. I will follow with Spring cloud openfeign.