Spring Digest Auth support in WebClient

Spring's WebClient allows specifying Basic Auth credentials quite simply:

webClient.get()
        .uri(someEndpoint)
        .headers(httpHeaders -> httpHeaders.setBasicAuth(someUserName, somePassword))
        ...

However, there doesn't seem to be a way to configure the WebClient to perform Digest Auth (RFC 7616).

A lot of servers rely on Digest Auth, and given that WebClient supports the less secure Basic Auth protocol, it would make sense to add support to the client's capabilities. Note that this feature has been requested in other places, for example on StackOverflow.

Thanks for considering this feature request!

Issue originally posted in the Spring Security project (https://github.com/spring-projects/spring-security/issues/7861).

Comment From: rstoyanchev

This is a reasonable request although digest is more involved than basic authentication, and also some HTTP libraries like the Jetty client have built-in support, so we'd likely leave this to be done at that level, and at the Spring Framework level make sure it can be plugged in.

In regards to Netty, I see no built-in support. However there is one external library that might be a good place to start if you need to do this. I did not try it but from a quick look, I think it could be plugged into Reactor Netty and the WebClient like this:

HttpClient client = HttpClient.create().doOnRequest((request, connection) -> {
    connection.addHandler(new HttpObjectAggregator(1048576));
    connection.addHandler(new NettyHttpAuthenticator("...", "..."));
});

ReactorClientHttpConnector connector = new ReactorClientHttpConnector(client);
WebClient client = WebClient.builder().clientConnector(connector).build();

NettyHttpAuthenticator expects (aggregated) FullHttpRequest and FullHttpResponse. Hence the HttpObjectAggregator above, and if the client sends synchronous values or Mono (but not Flux) the above could work. Further on, fully aggregated request and response should not be needed I believe, since the interaction involves just status and headers, so NettyHttpAuthenticator could probably be modified to use HttpRequest and HttpResponse and then it would work more generally.

Those are suggestions to try if you're stuck for something.

Comment From: bclozel

It seems that the RFC allows those headers to be added as trailing headers if the content encoding of the response body is chunked. In that case, we wouldn't need to buffer the whole response but accumulate the digest while writing the response.

This asks two questions: * do client and servers generally support trailing headers? * WebFlux doesn't support trailing headers and I'm not sure all supported servers do.

Comment From: PyvesB

Thanks for the responses and thoughts so far!

We do have an upcoming use case where we want to do send a body which is a Flux of DataBuffers, using chunked encoding.

If my understanding of the above is correct, this won't be possible with the current state of NettyHttpAuthenticator? I'll open an issue on that project's repository as well to gather the author's thoughts on the matter.

Comment From: rstoyanchev

@PyvesB I think the way NettyHttpAuthenticator is written it is a bit limiting, but it might not be too much of a change required if you follow my comment.

Comment From: vzhn

NettyHttpAuthenticator designed for one specific case: connect to IP camera over RTSP protocol. Usually IP camera does not close connection after sending 401 response. NettyHttpAuthenticator uses this fact to make things simpler. Sorry, I did not mention it in README. I am afraid that we need a new approach.

The digester state must be preserved across requests. Maybe using some mapping like this: host -> [nonce, realm, nonceCount, ...].

Some parts of NettyHttpAuthentificator could be reused: like Digester. Just now i've added couple of tests. Note: Digester lacks some functionality like userhash, charset and support of digested "username, realm, password" tuple. Some work is need to be done.

Also there is a server for debugging: Jetty with Digest auth. This could be used for unit tests.

Comment From: PyvesB

I built a small proof of concept internally integrating with the non-netty bits @vzhn's netty-http-authenticator. I'm operating at the Webflux level rather than the Netty level, and have added an onErrorResume call which performs the request again with a 401 is returned from the initial call.

My code roughly looks like the following:

webClient.get()
    .uri(someEndpoint)
    .retrieve()
    .onErrorResume(WebClientResponseException.Unauthorized.class, e -> {
        String wwwAuthenticateHeader = e.getHeaders().getFirst(HttpHeaders.WWW_AUTHENTICATE);
        HttpAuthenticator httpAuthenticator = new HttpAuthenticator("USERNAME", "PASSWORD"); // should be extracted to a field
        authenticator.challenge(new ChallengeResponseParser(wwwAuthenticateHeader).parseChallenge());
        String authorizationHeader = httpAuthenticator.generateAuthHeader("GET", someEndpoint);
        return webclient. ... //perform request again with authorizationHeader
    })

Will refine the approach when we actually get round to implementing this properly, and will share any insights/code that may be useful to others. 😉

Comment From: pszemus

Hi, I came across this issue when I was trying to find how to use digest authentication in reactive WebClient. @PyvesB Have you been able to perform a proper digest auth with your code?

Comment From: PyvesB

@pszemus indeed I have. The code in my previous message should hopefully help you get started. The production solution I've ended up with is slightly nicer, my code keeps state of the digest auth challenges returned by the server using a me.vzhilin.auth.DigestAuthenticator field and makes use of the doOnError and retryWhen reactive operators to update the challenge and reattempt the request on a 401.

Comment From: pszemus

@PyvesB My working example looks like this:

final RequestBodySpec getRequest = webClient
        .get()
        .uri(uri);
getRequest 
        .retrieve()
        .bodyToMono(String.class)
        .onErrorResume(WebClientResponseException.Unauthorized.class, error -> {
            log.info("Unauthorized. Logging in...");
            String receivedAuthenticateHeader = error.getHeaders().getFirst(HttpHeaders.WWW_AUTHENTICATE);
            DigestAuthenticator authenticator = new DigestAuthenticator(config.getLogin(), config.getPassword());
            try {
                authenticator.onResponseReceived(new ChallengeResponseParser(receivedAuthenticateHeader).parseChallenge(), error.getRawStatusCode());
            } catch (ParseException e) {
                return Mono.error(e);
            }
            String authorizationHeader = authenticator.autorizationHeader(HttpMethod.PUT.name(), uri.getPath());

            return getRequest 
                    .header(HttpHeaders.AUTHORIZATION, authorizationHeader)
                    .retrieve()
                    .bodyToMono(String.class);
        });

What optimizations would you recommend?

Comment From: PyvesB

Your working example looks like a good start from what I can tell.

The main optimisation that you'll want to address is to keep track of the last received digest auth challenge. Indeed, the protocol allows reusing the same nonce value as long as the nc count is incremented properly, which netty-http-authenticator does for you correctly underneath the hood.

To do so, a basic approach would be to create some sort of DigestAuthenticator bean that you initialise with username/password and then inject in the class performing the HTTP requests. You can use that object to send the Authorization header upfront, which will avoid hitting the systematic 401 when you perform the request:

String authorizationHeader = authenticator.autorizationHeader(HttpMethod.PUT.name(), uri.getPath());
final RequestBodySpec getRequest = webClient
        .get()
        .uri(uri)
                .header(HttpHeaders.AUTHORIZATION, authorizationHeader);
getRequest 
        .retrieve()
        .bodyToMono(String.class)
        .onErrorResume(WebClientResponseException.Unauthorized.class, error -> {
            log.info("Unauthorized. Logging in...");
            String receivedAuthenticateHeader = error.getHeaders().getFirst(HttpHeaders.WWW_AUTHENTICATE);
            try {
                authenticator.onResponseReceived(new ChallengeResponseParser(receivedAuthenticateHeader).parseChallenge(), error.getRawStatusCode());
            } catch (ParseException e) {
                return Mono.error(e);
            }
            String authorizationHeader = authenticator.autorizationHeader(HttpMethod.PUT.name(), uri.getPath());

            return getRequest 
                    .header(HttpHeaders.AUTHORIZATION, authorizationHeader)
                    .retrieve()
                    .bodyToMono(String.class);
        });

Note that on the very first request, authenticator.autorizationHeader will return null, so you may need to handle that - I don't know how the header method behaves when given null.

My second recommendation would be to do a refactor along the following lines:

Mono.fromSupplier(() -> webClient.get()
                    .uri(uri)
                    .headers(httpHeaders -> addAuthenticateHeader(httpHeaders, uri)))
                .retrieve()
        .bodyToMono(String.class)
                .doOnError(Unauthorized.class, u -> extractAuthenticateAndUpdateAuthenticator(u, uri))
                .retryWhen(Retry.max(1).filter(t -> (t instanceof Unauthorized)));

Comment From: sebphil

Hi! I'm having the same issue and would love to see a full digest auth client implementation in Spring WebClient.

In the meantime, I followed the ideas previously mentioned and came up with a solution that involves a FilterFunction. This enables the handling of Digest Auth as a crosscutting concern and lets you keep your actual webservice call code clean.

This filter makes use of @vzhn 's netty-http-authenticator to handle the parsing and construction of the various headers involved in Digest Auth.

Here is the gist: Digest Auth in Spring WebClient

Comment From: bclozel

See #33640