Spring Security Feature Request - Expose Reactor Netty HTTP Client Configuration

Summary of Issue:

The Spring security OAuth2 filter leverages Reactor Netty HTTP Client to send request to IDP. In cloud environment, there are many different network components with timeout setting will drop off idle HTTP connections. Netty HTTP Client by default use connection pool which will retry twice with the connections in pool and then throw out connection reset by peer exception. In that case Spring Security OAuth2 flow will get fail.

Feature Request:

Reactor Netty has configuration to disable/enable connection pool and set up idle timeout, etc. Also regarding the above mentioned issue, Netty will have an enhancement on connection pool retry algorithm with two options FIFO or LIFO. We would like Spring Security expose those configurations so any application leverage Spring Security can customize HTTP Client it uses, similar to what Spring Cloud Gateway does,

https://cloud.spring.io/spring-cloud-gateway/reference/html/appendix.html

https://github.com/spring-cloud/spring-cloud-gateway/blob/master/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/config/HttpClientProperties.java

The exposed configurations suggested, spring.security.httpclient.pool.type (Enabled/Disabled ,etc) spring.security.httpclient.pool.max-idle-time

Once Spring Security has Reactor Netty 0.9.5 built in, then expose Netty Client Pool configuration on FIFO/LIFO. So Spring Security Application can select LIFO and Netty HTTP Client then will create new HTTP connection after unsuccessful retry by LIFO algorithm.

Comment From: jgrandja

@hanscrg Spring Security does not provide/expose application configuration properties. This is typically provided by Spring Boot. I realize Spring Cloud Gateway does the same as you have referred to, however, we would not provide the same in Spring Security. FYI, the properties defined under spring.security.oauth2.* is provided by Spring Boot NOT Spring Security.

Also, the various ReactiveOAuth2AccessTokenResponseClient implementations provide a setWebClient(WebClient), so you can configure a custom WebClient and supply it to one of those implementations, eg. WebClientReactiveClientCredentialsTokenResponseClient.

I'm going to close this issue as application configuration properties are typically provided by Spring Boot.

Comment From: jgrandja

@hanscrg Here is an example on how to supply a custom WebClient to WebClientReactiveClientCredentialsTokenResponseClient composed within a ReactiveOAuth2AuthorizedClientManager:

@Bean
WebClient webClient(ReactiveOAuth2AuthorizedClientManager authorizedClientManager) {
    ServerOAuth2AuthorizedClientExchangeFilterFunction oauth =
            new ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager);
    return WebClient.builder()
            .filter(oauth)
            .build();
}

@Bean
ReactiveOAuth2AuthorizedClientManager authorizedClientManager(
        ReactiveClientRegistrationRepository clientRegistrationRepository,
        ServerOAuth2AuthorizedClientRepository authorizedClientRepository) {

    // TODO Customize
    WebClient webClient = WebClient.builder().build();

    WebClientReactiveClientCredentialsTokenResponseClient clientCredentialsTokenResponseClient =
            new WebClientReactiveClientCredentialsTokenResponseClient();
    clientCredentialsTokenResponseClient.setWebClient(webClient);

    ReactiveOAuth2AuthorizedClientProvider authorizedClientProvider =
            ReactiveOAuth2AuthorizedClientProviderBuilder.builder()
                    .clientCredentials(builder -> builder.accessTokenResponseClient(clientCredentialsTokenResponseClient))
                    .build();
    DefaultReactiveOAuth2AuthorizedClientManager authorizedClientManager =
            new DefaultReactiveOAuth2AuthorizedClientManager(
                    clientRegistrationRepository, authorizedClientRepository);
    authorizedClientManager.setAuthorizedClientProvider(authorizedClientProvider);

    return authorizedClientManager;
}

Comment From: odbuser2

This hasn't been resolved in Spring Security, Spring Boot, or Spring Cloud Gateway. I'm documenting some of the versions and referencing the one of the initial issues to avoid confusion:

It is resolved in reactor-netty 0.9.5.RELEASE. https://github.com/reactor/reactor-netty/issues/938

https://github.com/reactor/reactor-netty/releases/tag/v0.9.5.RELEASE Reactor Netty 0.9.5.RELEASE is part of Dysprosium-SR5 Release Train

Dysprosium-SR4 (not 5) is part of spring-security 5.2.2 https://github.com/spring-projects/spring-security/releases/tag/5.2.2.RELEASE

Dysprosium-SR5 should be in spring-security 5.3.0 spring-security 5.3.0 is slated for spring-boot 2.3.0

As far as I know, this issue has not been opened in spring-boot. @hanscrg do you plan on opening one in spring-boot? If it's part of spring-boot, spring-boot should use the correct defaults to prevent an error from occurring as well as exposing the underlying components for further customization.

In the current state, this is a showstopper when using Spring Cloud Gateway with OAuth2 (with reactor-netty underneath). There are various places to place the workaround but it really needs to be fixed in one of the Spring projects. @hanscrg was bounced around the projects a bit until it was recognized as an issue to resolve first in reactor-netty and then leveraged in one of the Spring libraries.

Comment From: rwinch

@odbuser2

Thanks for taking the time to follow up. @jgrandja provided a way in which you can configure Spring Security with custom Reactor Neety settings in https://github.com/spring-projects/spring-security/issues/7985#issuecomment-590902612

Does this allow you to do what you need to accomplish?

Please note that there is a difference in what you want to do and how you want to do it. Since Spring Security provides a setter for how you achieve the customization, you could map this to properties in your code while another user who wants to achieve the how another way can achieve the how in another way.

Spring Security will not provide properties that map to these settings. If you are looking for using properties to customize the Reactor Netty setttings, this has not been solved. You would need to create a ticket in Boot or Cloud. However, it is not a showstopper because there is a simple way to achieve the same result. You can also easily provide your own properties mapping to customizing the Bean that is created.

Comment From: odbuser2

@rwinch I was primarily documenting what had happened and what may need to happen (open a new issue in Spring Boot). It is currently a showstopper since reactor netty 0.9.5 is not used by Spring Boot. Even when it is, @jgrandja's suggestion needs to be incorporated into Spring Boot, otherwise it's still a workaround (an appropriate one).

Comment From: clavinovahan

@jgrandja, thanks for your suggestion on custom WebClient for OAuth which is very helpful. Also, @odbuser2, thanks for all the effort to a decent solution regarding this issue. I totally agree with you. I am trying to override the lib dependency and leverage Netty 0.9.5 and set leasing strategy and see how it goes. If it does not work, I will try @jgrandja's solution. But even it works it still look like a workaround. I am looking forward to Spring Boot configuration as a final solution.

Comment From: rwinch

@odbuser2

It is currently a showstopper since reactor netty 0.9.5 is not used by Spring Boot

Spring Boot 2.2.5 was released and uses Reactor Dysprosium-SR5 which uses Reactor Netty 0.9.5.

@hanscrg

But even it works it still look like a workaround. I am looking forward to Spring Boot configuration as a final solution.

You need to log an issue to Spring Boot for such a request. Generally, I don't view this as something I would recommend (there cannot be properties for every feature) but it is up to the Boot team.

Comment From: clavinovahan

@jgrandja, @rwinch, I tried @jgrandja's suggestion to include two Beans in our SpringBoot application as below, the customization is trying to disable Netty Pooled Connection and create new connection everytime, but it does not seems to be effective. The logs still shows PooledconnectionProvider used.

[reactor-http-epoll-6] DEBUG r.n.r.PooledConnectionProvider:254 - Channel cleaned, now 0 active connections and 4 inactive connections

What could be wrong the code below? It seems Spring Security still uses its own weblclient, not the customized one below.

@Bean
    WebClient webClient(ReactiveOAuth2AuthorizedClientManager authorizedClientManager) {
        ServerOAuth2AuthorizedClientExchangeFilterFunction oauth =
                new ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager);
        return WebClient.builder()
                .filter(oauth)
                .build();
    }

@Bean
    ReactiveOAuth2AuthorizedClientManager authorizedClientManager(
            ReactiveClientRegistrationRepository clientRegistrationRepository,
            ServerOAuth2AuthorizedClientRepository authorizedClientRepository) {

        TcpClient tcpClient = TcpClient.create(ConnectionProvider.newConnection());

        WebClient webClient = WebClient.builder()
                .clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
                .build();

        WebClientReactiveClientCredentialsTokenResponseClient clientCredentialsTokenResponseClient =
                new WebClientReactiveClientCredentialsTokenResponseClient();
        clientCredentialsTokenResponseClient.setWebClient(webClient);

        ReactiveOAuth2AuthorizedClientProvider authorizedClientProvider =
                ReactiveOAuth2AuthorizedClientProviderBuilder.builder()
                        .clientCredentials(builder -> builder.accessTokenResponseClient(clientCredentialsTokenResponseClient))
                        .build();
        DefaultReactiveOAuth2AuthorizedClientManager authorizedClientManager =
                new DefaultReactiveOAuth2AuthorizedClientManager(
                        clientRegistrationRepository, authorizedClientRepository);
        authorizedClientManager.setAuthorizedClientProvider(authorizedClientProvider);

        return authorizedClientManager;
    }

Comment From: jgrandja

@hanscrg

It seems Spring Security still uses its own weblclient, not the customized one below.

The configuration looks correct to me. Have you tried putting a break point in WebClientReactiveClientCredentialsTokenResponseClient to ensure the associated WebClient is the one that is configured?

I'm suspecting the log entry is coming from another path in the code.

Comment From: clavinovahan

@jaffadog, I put a very simple demo project on github below to show what happens. Could you please check what I am doing wrong? The project is easy to compile and run with instructions on the project page.

https://github.com/hanscrg/Sample-SpringCloudGateway-UAA

Comment From: jgrandja

@hanscrg I took a look at your sample and the issue is that the WebClientReactiveClientCredentialsTokenResponseClient is not used for oauth2Login(). You should configure WebClientReactiveAuthorizationCodeTokenResponseClient instead, as oauth2Login() implements the authorization_code grant as per the OpenID Connect spec.

Remove your existing authorizedClientManager and webClient @Bean and add the following:

@Bean
ReactiveOAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest> authorizationCodeTokenResponseClient() {
    TcpClient tcpClient = TcpClient.create(ConnectionProvider.newConnection());
    WebClient webClient = WebClient.builder()
            .clientConnector(new ReactorClientHttpConnector(HttpClient.from(tcpClient)))
            .build();

    WebClientReactiveAuthorizationCodeTokenResponseClient authorizationCodeTokenResponseClient =
            new WebClientReactiveAuthorizationCodeTokenResponseClient();
    authorizationCodeTokenResponseClient.setWebClient(webClient);

    return authorizationCodeTokenResponseClient;
}

Comment From: clavinovahan

@jgrandja, as you suggested it works fine. Thanks a lot!