SpringBoot "invalid scheme [lb]" when upgrading from boot 3.1.5 to 3.1.6 (also present in 3.1.7)

The code below works fine with Spring Cloud 2022.0.4 and Boot 3.1.5, but fails when upgrading to boot 3.1.6 or 3.1.7

return loadBalancedWebClientBuilder.build().get()
        .uri("lb://backend-service/test") //
        .exchangeToFlux( response -> response.bodyToFlux(String.class));

The error is :

org.springframework.web.reactive.function.client.WebClientRequestException: Invalid scheme [lb]
    at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136) ~[spring-webflux-6.0.14.jar:6.0.14]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ Request to GET lb://backend-service/test [DefaultWebClient]
    *__checkpoint ⇢ Handler com.example.demo.TestController#test() [DispatcherHandler]
    *__checkpoint ⇢ HTTP GET "/test" [ExceptionHandlingWebHandler]

I've stepped into ReactorLoadBalancerExchangeFilterFunction and it seems to be working fine, next.exchange(newRequest) is called with a correct newRequest (lb://backend-service has been properly transformed), but the next filter function has the old request. I'd gladly help further, but I don't see what's going on here.

I have a project that reproduces the problem here https://github.com/snussbaumer/repro-invalid-scheme-lb (change spring-boot-parent version to 3.1.5 and the test will work properly)

There is no workaround that I know of, and I can't upgrade.

Context of the upgrade : I wanted to address CVE-2023-34053

Comment From: snussbaumer

note that this impacts also spring cloud gateway : routing towards "lb://xxx" fails with the same invalid scheme [lb] error

Comment From: bclozel

@snussbaumer can you maybe first report this to Spring Cloud Loadbalancer? They should first investigate and decide whether it's a bug in Spring Cloud or in Spring Framework.

I've used your project to debug the filter function and found that it's passing lb://localhost:9004/test to the filter chain, so the scheme has not been transformed. Thanks!

Comment From: snussbaumer

thanks @bclozel, hum you're right, I must have been tired, I saw that "backend-service was transformed to localhost:9004 and didn't see that the "lb://" didn't change to http:// ...

I'll check this a bit more and submit it to Spring Cloud Loadbalancer if necessary.

thanks !

Comment From: vvalencia-cl

thanks @bclozel, hum you're right, I must have been tired, I saw that "backend-service was transformed to localhost:9004 and didn't see that the "lb://" didn't change to http:// ...

I'll check this a bit more and submit it to Spring Cloud Loadbalancer if necessary.

thanks !

Hi @snussbaumer do you have any advance with this?

I have the same problem, And also checked that the spring-cloud-loadbalancer dependency has the same version (4.0.4) between SB 3.1.5 and 3.1.7

But, this dependency changes: reactor-netty, from 1.1.12 to 1.1.14 And, only in the last version, the added a validation for only http(s) and ws(s) schemes: https://github.com/reactor/reactor-netty/commit/b1dd46b9a424ca27f7f770be6561faa84d812e5b#diff-0c6d2cbc84c0863f21506bab81ff507e76aef7bef80207e3bdc311bf0cd349edR137

Caused by: java.lang.IllegalArgumentException: Invalid scheme [lb]
    at reactor.netty.http.client.UriEndpointFactory.validateScheme(UriEndpointFactory.java:139)
    at reactor.netty.http.client.UriEndpointFactory.createUriEndpoint(UriEndpointFactory.java:81)
    at reactor.netty.http.client.HttpClientConnect$HttpClientHandler.<init>(HttpClientConnect.java:512)
    at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.subscribe(HttpClientConnect.java:208)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
    at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:82)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:98)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:44)
    at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
    at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
    at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)
    at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.complete(MonoIgnoreThen.java:292)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onNext(MonoIgnoreThen.java:187)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:236)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:203)
    at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onComplete(MonoIgnoreElements.java:89)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onComplete(FluxDematerialize.java:121)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:91)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onNext(FluxDematerialize.java:44)
    at reactor.core.publisher.FluxIterable$IterableSubscription.fastPath(FluxIterable.java:402)
    at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:291)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.request(FluxDematerialize.java:127)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.request(FluxPeek.java:138)
    at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onSubscribe(MonoIgnoreElements.java:72)
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onSubscribe(FluxPeek.java:171)
    at reactor.core.publisher.FluxDematerialize$DematerializeSubscriber.onSubscribe(FluxDematerialize.java:77)
    at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:201)
    at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:83)
    at reactor.core.publisher.Mono.subscribe(Mono.java:4495)
    at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:263)
    at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:51)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)
    at reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
    at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
    at reactor.core.publisher.DrainUtils.postCompleteDrain(DrainUtils.java:132)
    at reactor.core.publisher.DrainUtils.postComplete(DrainUtils.java:187)
    at reactor.core.publisher.FluxMaterialize$MaterializeSubscriber.onComplete(FluxMaterialize.java:141)
    at reactor.core.publisher.FluxLimitRequest$FluxLimitRequestSubscriber.onNext(FluxLimitRequest.java:104)
    at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
    at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
    at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.onNext(FluxTimeout.java:180)
    at reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
    at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)

Comment From: vvalencia-cl

As a workaround, you can change the Reactor BOM version to this:

<reactor-bom.version>2022.0.12</reactor-bom.version>

Comment From: snussbaumer

My problem came from SimpleReactiveDiscoveryClient, that I only use in integration tests. SimpleReactiveDiscoveryClient uses DefaultServiceInstance to store the differente instances, dans DefaultServiceInstance does not implement "getScheme" from the ServiceInstance interface => hence it returns always null. When trying to construct the real url, we get to LoadBalancerUriTools.doReconstructURI, and here we see :

String scheme = Optional.ofNullable(serviceInstance.getScheme())
                .orElse(computeScheme(original, serviceInstance));

serviceInstance.getScheme() is null, computeScheme will keep the "lb" scheme from "original".

This is not a problem with the Eureka DiscoveryClient for example, because it properly returns a scheme in the ServiceInstance. For the few cases that used a SimpleReactiveDiscoveryClient a did something similar (and simpler for me), that returned a proper scheme.

Comment From: snussbaumer

I'm not sure a lot of people use SimpleReactiveDiscoveryClient ... and not sure even this is really a bug. Maybe it is for spring cloud load balancer ? the fix would be quite simple in DefaultServiceInstance though.

Comment From: vvalencia-cl

I'm not sure a lot of people use SimpleReactiveDiscoveryClient ... and not sure even this is really a bug. Maybe it is for spring cloud load balancer ? the fix would be quite simple in DefaultServiceInstance though.

My problem is also in Integration Tests, using SimpleReactiveDiscoveryClient. I've never tried to run the app, because the tests weren't passing

Comment From: snussbaumer

that makes two of us, maybe it's time someone open a bug for spring cloud : https://github.com/spring-cloud/spring-cloud-commons/issues/1320