SpringBoot Need SSLSession/Certificate access in RestTemplateExchangeTagsProvider for required tags

Our RestTemplateExchangeTagsProvider implementation requires access to each request/response certificates (SSLSession / peer certificates) to extract ASG, cluster, application and instance information.

Requesting an update to this signature (or to ClientHttpResponse, or whatever option works):

Iterable<Tag> getTags(String urlTemplate, HttpRequest request, ClientHttpResponse response);

Comment From: wilkinsona

Thanks for the suggestion. Can you expand a bit on how you anticipate that information being made available?

With RestTemplate, SSL configuration is quite a low-level detail. It's managed by whatever HTTP client you're using and is abstracted away by the ClientHttpRequestFactory that's being used. In other words, RestTemplate doesn't really deal with SSL at all so it's not clear to me how the requested information could be made available automatically.

Comment From: jleibund

Yes, I doubt this information could be supplied ubiquitously and it would require HTTP client support (this would be a case-by-case basis). I have extracted this reflectively from two clients thus far in both cases traversing from the ClientHttpResponse not the request.

Client 1 (Apache + our Metatron SSL) chain: org.springframework.http.client.HttpComponentsClientHttpResponse.httpResponse (private) org.apache.http.impl.execchain.HttpResponseProxy.connHolder (private) org.apache.http.impl.execchain.ConnectionHolder.managedConn

From ManagedHttpClientConnection the SSLSession is accessible.

Client 2 (NIWS) - the chain is 8 levels deep (redacted).

In either case, its a little jaw-dropping to me that a simple detail like a peer certificate has been deemed too low-level by each of these OSS projects. An essential reporting requirement for tags is who we are speaking with. Personally, I find the collective set of decision to bury this under layers of indirection and private members (to shield consumers from it?) to be unacceptable and I'm trying to think of any other language where I wouldn't have access to peer certs, in a rest client (RestTemplate or otherwise, IDK) to information about the peer I've connected to. Seems like pretty basic detail to get wrong.

Beyond my own opinion and given what exists now I believe that offering HTTP client providers the ability to supply this through ClientHttpResponse may be the only option:

Optional<SSLSession) getSession();

If accessible, its present, if not we cannot supply basic tags metrics about who we are speaking with.

Comment From: wilkinsona

Encapsulation is one of the main features of Java and is something that the Spring projects embrace strongly. Hiding implementation details and complexity has, in my opinion, been one of the main contributors to Spring's success over the years. From what you've said, it sounds like Spring, and Java to a large extent, may not be a particular good fit for your tastes. I don't think that means that they've got anything wrong. It's more an indication that they might not be the right tool for the job.

From what we've seen in the 3 years since RestTemplateExchangeTagsProvider was introduced, the URI to which the request is being made has been sufficient from a metrics perspective to know to whom the client is speaking. As far I can remember, this is the first time we have seen interest in deriving metrics from the SSL session and Micrometer's issue tracker tells a similar story. This isn't to say that your use case is invalid, but it does suggest that it's quite unusual which hopefully goes some way to explain why it isn't quite as easy as you would like it to be.

For what it's worth, it is possible to access the SSLSession with Apache's HTTPClient without resorting to reflection. You can do so using an HttpResponseInterceptor, retrieving the HTTP connection from the HttpContext using the HttpCoreContext.HTTP_CONNECTION constant. Unfortunately, it's not entirely straightforward to then pass the session from the interceptor to the tags provider but it can be done with a thread local. Putting that together looks like this:

@Bean
public CommandLineRunner runner(RestTemplateBuilder builder, SslSessionRestTemplateExchangeTagsProvider tagsProvider) {
    return (args) -> {
        RestTemplate restTemplate = builder.requestFactory(() -> {
            HttpClient httpClient = HttpClients.custom().addInterceptorFirst(tagsProvider).build();
            return new HttpComponentsClientHttpRequestFactory(httpClient);
        }).build();
        restTemplate.getForObject("https://spring.io", String.class);
    };
}

@Bean
public SslSessionRestTemplateExchangeTagsProvider restTemplateExchangeTagsProvider() {
    return new SslSessionRestTemplateExchangeTagsProvider();
}

static class SslSessionRestTemplateExchangeTagsProvider implements RestTemplateExchangeTagsProvider, HttpResponseInterceptor {

    private final DefaultRestTemplateExchangeTagsProvider delegate = new DefaultRestTemplateExchangeTagsProvider();

    private final ThreadLocal<SSLSession> sslSession = new ThreadLocal<>();

    @Override
    public Iterable<Tag> getTags(String urlTemplate, HttpRequest request, ClientHttpResponse response) {
        try {
            Tags tags = Tags.of(this.delegate.getTags(urlTemplate, request, response));
            SSLSession sslSession = this.sslSession.get();
            // Add tags based on the SSL session
            // tags = tags.and(key, value);
            return tags;
        }
        finally {
            sslSession.remove();
        }
    }

    @Override
    public void process(HttpResponse response, HttpContext context) throws HttpException, IOException {
        ManagedHttpClientConnection routedConnection = (ManagedHttpClientConnection)
                context.getAttribute(HttpCoreContext.HTTP_CONNECTION);
        this.sslSession.set(routedConnection.getSSLSession());
    }

}

I agree that making this more straightforward would be best done via the ClientHttpResponse abstraction which would require a change in Spring Framework. It may also be that I have overlooked something and it's already easier than what I have shown above. I'll ask the Framework team to take a look at this issue and we can take things from there.

Comment From: bclozel

I think @wilkinsona 's proposal is the best course of action.

HTTP client libraries developers hardly surface this information with their APIs and how this aspect is modeled might change from one library to another. RestTemplate is about abstracting those specifics on top of several clients; any advanced support like this should use the "native" API.

As of Spring Framework 5.x, RestTemplate is in maintenance mode: we're still maintaining it but refrain from evolving its API or feature set. I'm afraid that even if we did manage to provide this concept consistently across libraries, we would not evolve the ClientHttpResponse at this stage.

In similar cases, many libraries chose to implement their own metrics support (using micrometer as an optional dependency). This usually allows to provide low-level metrics while still enjoying the benefits of strong encapsulation. From a Spring Framework perspective, I guess we can decline this enhancement request.

Thanks @jleibund for raising this interesting use case!

Comment From: wilkinsona

Thanks, @bclozel. Given the situation with HTTP client libraries and the abstractions built on top of them, I don't foresee any changes to Spring Boot in this area either. Thanks again for the suggestion, @jleibund, but I'm going to decline this one.

Comment From: rstoyanchev

The RestTemplate was created long before concerns such as observability became common and as @bclozel mentioned, the RestTemplate is in a place where it won't see major enhancements.

For our WebClient however there is easier access to the underlying native request through its httpRequest method and then getNativeRequest on the returned ClientHttpRequest. That is about all we can do in any case, since as you pointed out yourself, any attempt to abstract HTTP client specific API types isn't going to get very far.

Comment From: jleibund

Thanks, yes I have already worked on WebClient with @smaldini and had a much better experience accessing what we need (is the key, not encapsulation v none) in the case where DNS resolvers lead us to instances where their identity (ASG, cluster, instance) is contained in metatron certificates and this is the only way to properly tag associated metrics. It worked well, its totally immutable, doesn't resort to thread local hacks, that we are trying to reduce as we move infrastructure toward WebFlux. Thus I'd probably also have to decline the provided workaround.

This question had more to do with the rest of the fleet that isn't on WebClient and won't migrate to it for maybe years.. or until we offload all egress traffic to envoy - which has absolutely no problem accessing this information for tagging. This issue was about how do we deliver complete metrics RestTemplate-based clients in the meantime - before any mass WebClient or envoy migration.

It seems like unless I want to introduce new thread local future problems, the best course is what we're already doing.