The basics of secret management

The basics of secret management

The need for secret management exists in just about every application that is developed, which is to say, every application needs to be able to run locally and in production at a minimum. In most cases, an application will also need a staging server and ways to dynamically spin up test environments to try out some new idea your team cooked up. This leads every application to have some sort of configuration system as a way to change how the system operates at runtime.

While some configuration options are straightforward, such as which environment you are running in. Other configuration options can be sensitive. If you are following best practices such as the Principle of Least Privilege then not everyone on your team should have access to:

• Third-party API keys
• Database credentials
• Production environment

How can I keep these a secret?

The general idea for managing secrets is to use a secret management solution such as Vault or to use something built into your CI\CD pipelines, as in the case of Gitlab. No matter what solution you choose, each one provides the following.

• A secure backend to store your secret in
• A mechanism to allow you to retrieve your secrets
• A user management system to put restraints on who can access your secrets.
• A way to set the same secret for different environments

Using one of these mechanisms, you can inject your secret into your application usually through an environment variable or a provided API.

How we manage secrets at Architect.io

To keep these a secret, we first need to look at how your application can get access to outside configuration options. To do this, let’s take a look at an architect.yml file.

name: component
description: Human-readable description of my component
parameters:
  google_api_key:
    required: true
    description: Google API Key
services:
  my-api:
    environment:
      GOOGLE_API_KEY: ${{ parameters.google_api_key }}

We have specified a new parameter called google_api_key, that we can use when deploying our app. Architect.io can then pass that parameter into our service as an environment variable.

As an example, we could deploy this application as follows:

architect deploy account/component -p google_api_key=SOMEKEY

Then, for instance, if this was a Node.js service, we could access the passed in parameter as follows:

process.env.GOOGLE_API_KEY // "SOMEKEY"

So we were able to define a required configuration parameter for the deployment of this component and then set it when we deploy the application. From there, our application was able to read an environment variable with the same name to get the value

What happens if I have a lot of secrets?

Most applications tend to have more than one secret, and while we could pass them all in one at a time with the -p flag, this is not scalable. So instead, we can make a file that contains our secrets and pass it in the file instead.

# secrets.yml
account/component:*:
  google_api_key: SOMEKEY

The secret file requires two pieces of information:

• A scope that lets us know who these secrets should be deployed to. In this instance, we only want to pass this information to account/component. No other component can access this secret.
• A set of key values that contain the name of the parameter from the architect.yml file and the value we want it to be.

To use this, we just deploy with the secret file:

architect deploy account/component -s secrets.yml

These secret files can also be used to set up the secrets that developers may need to run locally. Instead of having complex commands we just need to make a developer secrets.yml and distribute it to each developer.

How do I keep my secrets secret though?

While Architect.io does allow you to pass in a secret or secrets file during deploy time, we need to now store our sensitive secrets in some backend that has user restrictions on them. This is where the Architect.io website comes in.

To do this we are going to go to one of our accounts or environments and then access the “Secrets” tab on the side menu. We will then be brought to the following page.

This is the same data we saw in the secrets.yml file above. We have a scope, a key, and a value for each secret. If you are an admin or owner of the account you can view the secret value. Otherwise, if you are a member you can only view which secrets are being set.

Now if we were to deploy our application again, it would use the secret we set up in the UI.

architect deploy account/component

When our application is deployed it will look for the secrets it needs from either the account secrets or environment secrets. If neither contains the secret and it is set as required then the deployment will fail.

What comes next when it comes to secret management

For more information please check out the documentation on configuration management. To try it yourself you can fork one of our templates and attempt to deploy it with parameters set using our secret store.

If you’d like to learn more about dependency-aware continuous delivery with Architect.io, or secret management, we encourage you to check out these resources:

And as always, we’d love to have you follow us for more cool content and product updates. Find our channels on LinkedIn and Twitter!