WebClient Builder is not configured with ObservationRegistry and ObservationConvention for http.client.requests, by default Tried using ObservationWebClientCustomizer.customize() but has varying outcomes based on how the WebClient builder is supplied.

Way 1: 

    @Bean
    public WebClient webClientFromBuilder(WebClient.Builder webClientFromBuilder) {
                return webClientFromBuilder
                .baseUrl("http://localhost:6543")
                .build();
         }

Outcome:

  • No client metrics generated if the builder object is not customized using customize() method as shown in the next way of creating the WebClient bean
  • Http Client metrics do not get generated in the actuator/prometheus endpoint,
  • Which means that the WebClient builder is not configured the way RestTemplate builder is and hence is missing the ObservationRegistry object and comes back with a NoOpObservationObject type. Also, has the ObservationConvention as null.

Way 2:

    @Bean
    public WebClient webClientFromBuilder(WebClient.Builder webClientFromBuilder) {
        webClientCustomizer.customize(webClientFromBuilder);
                return webClientFromBuilder
                .baseUrl("http://localhost:6543")
                .build();       
    }

Outcome: - Client metrics are generated and the URI values are also populated based on the invocation time URI formation method. - Both the ObservationRegistry and the ObservationConvention are configured successfully using the ObservationWebClientCustomizer.customize()

Way 3: 

    @Bean
    public WebClient webClient() {
        return WebClient.create();      
    }

Outcome: - Does not produce any client metrics. - No Observation Registry or Convention configured in the builder by default. - Not sure if this is a recommended way of using WebClient, but since we have this usage too as part of our consumer applications in which our library would be used, we had to cater to this too.

Invocation:

          private void commandLineArgsBuilder(WebClient webClientObject) {
        Random highCardinalityValues = new Random(); // Simulates potentially large number of values
        String highCardinalityUserId = String.valueOf(highCardinalityValues.nextLong(100_000));
        JsonNode jsonNodeResponse = null;
        Object response = webClientObject
                .get()
                .uri("http://localhost:7654/user/{userId}", String.class, highCardinalityUserId)
                .retrieve()
                .bodyToMono(String.class)
                .block();
        log.info("Got response [{}]", response); // ... so will this line
    }

Is this the expected behavior? As per documentation here: https://docs.spring.io/spring-framework/reference/integration/observability.html It suggests that the WebClient would behave indifferently from RestTemplate with respect to the http client requests' instrumentation. But in reality it is otherwise.

Furthermore, the URI tag value was being populated as "none" in the above WebClient usages, and I happened to learn from this issue that, the URI has to be formed in a particular way for it to be populated as is, and not as "none" in the metric tag. It would be great if an explicit mention of this behavior can be made in any of the documentation or a pointer to one such if it already exists would be appreciated too. As this discovery took around 3 days, tbh and could save time to many in my shoes.

Thank you, in advance.

Comment From: philwebb

Spring Boot should apply the ObservationWebClientCustomizer automatically to the WebClient.Builder bean so I believe that "way 1" is something we'd expect to work. Could you please provide a small sample application that shows the problem so that we can debug what's going on.

Comment From: abhaygitty

Spring Boot should apply the ObservationWebClientCustomizer automatically to the WebClient.Builder bean so I believe that "way 1" is something we'd expect to work. Could you please provide a small sample application that shows the problem so that we can debug what's going on.

Sure thing. Here is the link to a project that I have created by forking one of existing sample projects provided by @marcingrzejszczak https://github.com/abhaygitty/observability-boot-blog-post Replication steps: - Start the Server application. - Start the Client application in debug mode with a breakpoint on line 40 of ClientApplication.java. - You should see that the WebClient builder's ObservationRegistry is a NoopObservation type object, and ObservationConvention is null. Hence we wouldn't see any http_client_request metrics being populated. - Now, uncomment line 39 for the ObservationWebClientCustomizer's customize() method to take effect and restart the client application. You should now be able to see that the WebClient builder's ObservationRegistry is no more a NoopObservation type object and is a SimpleObservationRegistry type, and ObservationConvention is not null and is initialized to a value of http.client.requests. And, the http_client metrics are generated now opposed to the former behavior.

Thank you.

Comment From: philwebb

Thanks for the sample, but I'm afraid it doesn't compile for me. I get:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project client: Compilation failure: Compilation failure: 
[ERROR] /Volumes/Data/projects/spring-boot/samples/observability-boot-blog-post/client/src/main/java/com/example/client/ClientApplication.java:[23,56] package org.springframework.web.reactive.function.client does not exist
[ERROR] /Volumes/Data/projects/spring-boot/samples/observability-boot-blog-post/client/src/main/java/com/example/client/ClientApplication.java:[38,56] package WebClient does not exist
[ERROR] /Volumes/Data/projects/spring-boot/samples/observability-boot-blog-post/client/src/main/java/com/example/client/ClientApplication.java:[38,16] cannot find symbol
[ERROR]   symbol:   class WebClient
[ERROR]   location: class com.example.client.ClientApplication
[ERROR] /Volumes/Data/projects/spring-boot/samples/observability-boot-blog-post/client/src/main/java/com/example/client/ClientApplication.java:[46,122] cannot find symbol
[ERROR]   symbol:   class WebClient
[ERROR]   location: class com.example.client.ClientApplication
[ERROR] /Volumes/Data/projects/spring-boot/samples/observability-boot-blog-post/client/src/main/java/com/example/client/ClientApplication.java:[52,45] cannot find symbol
[ERROR]   symbol:   class WebClient
[ERROR]   location: class com.example.client.ClientApplication
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Did you make changes to the pom.xml file and forget to commit them? Could you also please update the sample to use the latest sable release rather than snapshots.

Comment From: spring-projects-issues

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Comment From: spring-projects-issues

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.