Spring Cloud Config Eureka+CloudConfig via HTTPS cannot be used by a CloudConfig Client without setting JVMArgs

I've a Eureka+CloudConfig server running on localhost:8443. You can find an example on https://github.com/trickert76/eureka-discovery-and-cloud-config-service

This repo is configured to be used under Wildfly - so please change the application.yml line 51 and remove 55. defaultZone: https://${eureka.instance.hostname}:${server.port}/eureka If you start the service it runs under https://localhost:8443/eureka and https://localhost:8443/config. The dashboard is visible under https://localhost:8443/dashboard

Then use a client like https://github.com/trickert76/eureka-ssl-client.

If you start the client, it will stop, because during bootstrap there is no certificate to connect to https://localhost:8443/eureka

If you wouldn't use bootstrap, then you could uncomment the SslConfiguration to change the Eureka-Client to provide a way to connect to the running service (like Config). I found no other way but adding the JVMargs to give the Eureka/Cloud-Config client a way to trust the Eureka/Cloud-Config HTTPS server.

So edit the build.gradle bootRun task and change the property (a relative path works during bootstrap).

But later after the Spring context is up, the Tomcat also uses the JVM arg and wants to configure itself. That doesn't work (why ever, that's not the point). The only proper solution is to use Undertow instead of Tomcat.

I know about https://github.com/spring-cloud/spring-cloud-config/issues/148 - but disabling SSL verification isn't a solution and adding later in production hundreds of certs to the system trust store isn't a solution too. It would have side effects if the system needs an upgrade, etc.

I think, it would be very helpful, if the underlying HTTP client used by Eureka and Cloud Config could be configured directly - and give them (already during bootstrap) a hint, where to find a different trust store (instead of the JVMargs or system default).

Because the config under spring.cloud.config.discovery is part of "spring-cloud-config" I would expect some properties like that from server.ssl...

Comment From: ryanjbaxter

I am having a hard time deciphering what the exact problem is. Is it just that the eureka client can't connect to the eureka server because it does not trust the SSL certificate?

Comment From: trickert76

If I run the Eureka client in standalone mode, I don't have a problem, because there is a Bean, I can override. But together with Cloud Config, I need to start the Eureka client during bootstrap and then -yes- the Eureka client can't connect. I'm aware, that this could be a Eureka problem too, but the configuration that the Cloud Config should use a Eureka instance is under spring.cloud.config.discovery - so I'm not sure, how to give Eureka the hint, which Trust store to use.

Comment From: ryanjbaxter

So the problem is that when the config client reaches out to the Eureka server to figure out the config server URL it fails because the Eureka client does not trust the SSL certificate?

Comment From: trickert76

Correct

Comment From: ryanjbaxter

Have you tried importing the certificates into the JVM?

Comment From: trickert76

no, but I'm pretty sure, it would work. But that's not a solution for our problem. The "installation" is made across multiple (hundreds) of customers and all of them have different opinions about changing system settings (and adding a self-signed-certificate to a systemwide available trust store is nothing, they would prefer).

Comment From: ryanjbaxter

Unfortunately the only way to address the problem is to either import the certificate or use a trusted certificate by default.

Comment From: trickert76

That's bad, because I thought Eureka+Cloud Config is a "prefered" base setup for a microservice infrastructure and it works well without SSL. But using SSL should be as simple as without SSL.

I see, that using Undertow instead of Tomcat works, because the Undertow ignore's whyever the JVM arguments.

dependencies {
  compile("org.springframework.boot:spring-boot-starter") {
    exclude module: "tomcat-embed-el"
  }
  compile("org.springframework.boot:spring-boot-starter-web") {
    exclude module: "spring-boot-starter-tomcat"
  }
  compile('org.springframework.boot:spring-boot-starter-undertow')
  ...
}

I'm not sure, if this could help or if this would be a "hack".

Comment From: ryanjbaxter

There is nothing we can really do from a Spring Cloud perspective. We support Undertow so there is nothing wrong with using it if it works for you. But then I question where it is getting the trusted list of certificates from....

Comment From: trickert76

I found another "solution". that works for together with the default Tomcat, but it's not nice. Maybe you've an idea:

I've added a bean via META-INF/spring.factories: org.springframework.cloud.bootstrap.BootstrapConfiguration = myclient.config.SslConfiguration

The SslConfiguration is from another post here, like:

  @Bean
  public DiscoveryClient.DiscoveryClientOptionalArgs getTrustStoredEurekaClient(SSLContext sslContext) {
    DiscoveryClient.DiscoveryClientOptionalArgs args = new DiscoveryClient.DiscoveryClientOptionalArgs();
    args.setSSLContext(sslContext);
    return args;
  }

  @Bean
  public SSLContext sslContext() throws Exception {
    return new SSLContextBuilder().loadTrustMaterial(trustStore, trustStorePassword.toCharArray()).build();
  }

Then the ugly part - because of the second DiscoveryClientOptionalArgs in the spring context there comes an exception. I removed the default bean manually and it works like. I can define in truststore my custom TrustStore, that I need and everything is ok.

  @Bean
  public static BeanFactoryPostProcessor registerPostProcessor() {
    return (ConfigurableListableBeanFactory beanFactory) -> {

      BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;

      for (String beanDefinitionName : registry.getBeanDefinitionNames()) {
        if (beanDefinitionName.equalsIgnoreCase("discoveryClientOptionalArgs")) {
          BeanDefinition beanDefinition = registry.containsBeanDefinition(beanDefinitionName) ? registry.getBeanDefinition(beanDefinitionName) : null;
          if (beanDefinition != null) {
            log.info("Removing Singleton - discoveryClientOptionalArgs");
            if (registry.containsBeanDefinition(beanDefinitionName)) {
              registry.removeBeanDefinition(beanDefinitionName);
            }
          }

        }
      }
    };
  }

From my point if view, you can close this issue, except you have a better idea how to avoid the "discoveryClientOptionalArgs" bean to be registered

Comment From: AsinRay

I have trace the code to have a look at how and when the config client's RestTemplate is create, and get an idea : the config client is not supported https by default. the code in org.springframework.cloud.config.client.ConfigServicePropertySourceLocator.java line 82, called getSecureRestTemplate(properties) which locate in line 250~267 , private RestTemplate getSecureRestTemplate(ConfigClientProperties client), and it's param ConfigClientProperties has no ssl properties at all. ConfigServicePropertySourceLocator has a method public void setRestTemplate(RestTemplate restTemplate) to set a user defined RestTemplate , But how can i inject the RestTemplate in a bootstrap context? BTW, the bean ConfigServicePropertySourceLocator is initialized in org.springframework.cloud.config.client.ConfigServiceBootstrapConfiguration .

Comment From: biergit

We had the same problem and solved it by creating a custom auto configuration that has a method like

@Autowired
public void init(ConfigServicePropertySourceLocator locator) {
  locator.setRestTemplate(createSelfSignedStrategyRestTemplate());
}