Hello,
is there anything programmed about an integration between Spring Cloud Config and AWS Parameter Store? Since it looks like AWS tool for storing configurations, I thought it would be nice having some integration between that and Spring Cloud Config.
Comment From: ryanjbaxter
No we don't support AWS Parameter Store as a configuration backend. However there is nothing preventing you for implementing your own support.
Comment From: mrcasablr
This is something I'm interested in too.
Comment From: shafqatevo
+1 for this support...
Comment From: jimzhanghm
+1. need that support for securing secrets stored in AWS Parameter Store backed with kms.
Comment From: jkuipers
Just had a quick look at the AWS API for the parameter store. Newer versions actually support path-based lookups: https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParametersByPath.html.
That means that instead of only directly implementing PropertySource and being forced to do a single lookup for each property name, the implementation could instead extend EnumerablePropertySource and also support multiple lookups to support shared application properties, overrides for active profiles, etc. Thus it should be possible to do something similar to the Consul support: calculate a bunch of paths based on the application name, active profiles, configured prefix etc. and then, using a PropertySourceLocator, create a CompositePropertySource containing an ordered list of AwsParamStorePropertySource instances, each containing the result of a path-based lookup.
Not sure if supporting various value formats would also make sense here: single key/value instead of whole YAML/Properties-based payloads don't really seem like a good fit for the Parameter Store.
I was looking into this because my team is thinking about starting to use the Parameter Store. If we go that way I'll see what I can contribute for this issue.
Comment From: peruzzo
+1
Comment From: spencergibb
@peruzzo please plus one the original comment using reactions, not a comment.
Comment From: jkuipers
I've implemented something that seems to work quite well, based on what I described. I will at least blog about it and publish the code. A pull request would require some more work to separate the code into an impl and separate starter probably, but I'll see what I can do.
Comment From: jkuipers
Initial code can be found here: https://gitlab8.trifork.nl/jorisk/spring-cloud-aws-param-store
Comment From: ryanjbaxter
@jkuipers if you would like to contribute this to Spring Cloud Config, you will need to open a PR against this repo.
Comment From: spencergibb
Glancing at it, it is not an EnvironmentRepository for config server, but a property source locator, like consul and zookeeper. It does not belong in this project.
Comment From: mrcasablr
@spencergibb It would be great if we can make it easier for developers to use it out-of-the-box and if the code is maintained. Any ideas how to go about this?
Comment From: spencergibb
as it is, it would make more sense as a module in spring-cloud-aws
Comment From: jkuipers
Oh, totally agree: I stumbled upon this issue when I was searching for AWS Parameter Store support in Spring Cloud, and didn't realize that this issue tracker is specifically for the Config Server. I'll see if there's an open issue there already to request AWS Parameter Store support.
Comment From: jkuipers
I'll follow up on this here: https://github.com/spring-cloud/spring-cloud-aws/issues/207
Comment From: adlmez
+1
Comment From: sercastiGL
Pretty sure this issue can be closed now, the feature has been included with Spring Cloud 2.x spring-cloud-starter-aws-parameter-store-config https://cloud.spring.io/spring-cloud-static/spring-cloud-aws/2.0.0.RELEASE/multi/multi__cloud_environment.html#_integrating_your_spring_cloud_application_with_the_aws_parameter_store
Comment From: spencergibb
I don't think so. This is for integration with config server (ie an EnvironmentRepository). What is there is direct without config server.
Comment From: Walliee
If no one is actively working on it, I'd be happy to take a stab at implementing this.
Comment From: rshnaper
@Walliee same here. I actually need this for some prototypes at work and can start/or help out if you haven't started yet. Been looking into the internals already to figure out how things are tied together.
Comment From: Walliee
@rshnaper I haven't started yet; so please go ahead. We can possibly collaborate once you have a PR open?
Here is what I had in mind: - Potentially reuse AWS Parameter Store support from spring-cloud-aws. - It would be nice to be able to support something like spring.cloud.config.token, so that config server can authenticate with AWS Parameter Store on client's behalf.
@spencergibb thoughts? any additional pointers?
Comment From: rshnaper
@Walliee I agree, I was thinking of trying to re-use as much as possible from AWS Parameter Store Support but conform to the search path standards of config-server.
As for authorization, I was thinking of exposing a configuration parameter for a role that the config server can optionally assume before retrieving data from AWS ParameterStore. That way the actual authentication is left to the underlying AWS SDK with its default authentication chain but we can piggy back on it by assuming a specific role.
Comment From: Walliee
So the way it works for Vault today is that the Spring Cloud Config Client passes a Vault token (X-Config-Token header) to the Config Server and the server then uses this on the requests it sends to Vault.
This is so that Config Server doesn't need explicit access to secrets in Vault. You could have multiple apps connecting to Config Server that pass in their own tokens and thus can only access secrets that they have access to.
Since AWS Parameter Store supports storing secrets, I imagine we’d like to preserve the same semantics as Vault.
Comment From: rshnaper
Hmm I can now see why the AWS Parameter Store Support module wasn't merged right away into the Config Server. The way AWS authorization works is that the config server will need to have access to Parameter Store in order to read specific entries.
This is normally done via custom roles thus either the config server is allowed to read all the entries or the client can pass the name of the role that config server will have to assume in order to retrieve specific entries. The problem with the roles is that they're not exactly the same as a client's secret (I'm assuming that is what being passed in X-Config-Token header?).
So if one wants to be super secure then the only option is for the client to fetch the properties directly from AWS Parameter Store just like it's done in AWS Parameter Store Support module. This is because authentication and authorization is done on the client and is not delegated to the config server.
Any thoughts on how to make it more secure apart from passing the name of the role that config server will temporarily assume for fetching specific entries from Parameter Store?
Comment From: Walliee
I wonder if Session Token is more appropriate in this case.
Comment From: rshnaper
Actually both work on the same principle where temporary credentials are returned that a service or user will use to interact with AWS. According to the documentation get-session-token is used only by IAM users to generate temporary credentials while assume-role can be done by any service or user that has a role.
Initially I thought the client may just pass the name of the role to config-server to assume but the more I think about it the more it makes sense for the client to ask for temporary credentials via assume-role (or get-session-token) and pass the temporary credentials off to config server to use for retrieving the items.
I am thinking of making the following implementation: 1. If config server did not receive X-Config-Token header with the request then the default authentication chain will be invoked in order to fetch requested entries. This is assuming config server was set-up to access AWS, otherwise request will fail. 1. If X-Config-Token header is sent with the request then it will contain (Base64 encoded?) temporary credentials that config server will use to retrieve the requested items from Parameter Store
I'm still pondering whether sending temporary credentials over the wire is a good idea. I'll have to look into this a bit more.
Comment From: Walliee
What you are suggesting sounds good to me.
I'm still pondering whether sending temporary credentials over the wire is a good idea.
This is a valid concern/risk but one that is easily mitigated by using TLS.
Comment From: zampettim
I think the X-Config-Token approach should be good. Its basically what the Vault implementation for Config Server is doing. I have a need for this as well, so if there is a repo I can look at and possibly contribute with, I'd be willing to help.