When working on enterprise projects, it’s common to have to download private dependencies that require authentication to be installed (usually, internal or paid packages).

In modern setups, you’ll most likely use Docker to package your application (or service) and all its dependencies into a standalone image. Typically, building the Docker image is automated through a continuous deployment system.

Do Not Leak Your Secret Keys!

Most forges, including the popular GitHub and GitLab services, provide at least two ways to access private repositories: using HTTP authentication with personal access tokens or using SSH.

Most package managers, including Composer (the PHP package manager), support authentication by these means.

Composer, for instance, can use the following methods:

• Storing the personal access tokens provided by the forge in a file named auth.json
• Storing them in an environment variable named COMPOSER_AUTH
• Relying on SSH authentication

A good security practice is to prevent people who can access your Docker images from also having access to these secrets, and thus to the private repositories they grant access to. This means that the secrets should be used to download dependencies, but should not be accessible to people who have access to the Docker image or to the Git repository containing the source code.

Unfortunately, this is quite difficult to achieve with Docker. For performance reasons, Docker caches the result of each build step in what are called layers.

© Docker

Anyone with access to an image can access all the layers created during the build process. Exploring the content of layers is very easy with tools like dive. That means that if you copy your tokens (e.g. the auth.json file for Composer) to download your private dependencies, then remove them from the image, they will still be accessible to anyone having access to the final image through the layers!

Relying on environment variables or build args is not a better option: their values are also accessible to anyone who has access to the image!

Docker Build Secret Information

Docker 18.09 (2018) and Docker Compose 2.7.0 (2022) have introduced a new feature to solve this common problem: build secrets.

Build secrets allow files containing secrets to be mounted at build time, and guarantee that the content of these files will not be accessible in the final image.

They have also added a new subsystem designed specifically for downloading private dependencies: the ability to use the host system’s SSH access during builds.

This is very convenient because SSH has always been the preferred way to download private Git repositories, and is of course supported out of the box by the popular version control system.

Cloning a Private Git Repository Thanks to SSH Forwarding

Let’s create a Docker image containing a private Git repository but not the credentials needed to download it!

First, make sure that you’re able to download your private repository from your host:

Then, ensure that your private SSH key is added to the SSH agent (Docker will connect to it) by running:

ssh-add

You should now be able to build an image that contains your private repository:

FROM alpine

# OpenSSH client and Git are required dependencies to clone the repository
RUN apk add --no-cache openssh-client git

# Add the SSH public keys of the Git server to the known hosts (here, github.com)
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

# Clone the repository, notice the "--mount=type=ssh" option, which allow this instruction to use SSH forwarding
RUN --mount=type=ssh git clone --depth=1 '[email protected]:dunglas/my-private-repo.git'

Finally, hint Docker that it needs to connect to the default SSH agent of the host:

docker build --ssh default

Tadam! You have cloned a private Git repo without disclosing the secret key!

Downloading Private Composer Packages

Under the hood, Composer uses Git to download private repositories (it can also use other protocols such as HTTP, but this is less practical with Docker and is outside the scope of this article). Applying the same method we’ve seen previously is straightforward.

Here is a composer.json file referencing a private package stored on GitHub thanks to the repositories option:

{
  "require": {
    "dunglas/my-private-package": "dev-main"
  },
  "repositories": [
    {
      "type": "vcs",
      "url":  "[email protected]:dunglas/my-private-package.git"
    }
  ]
}

By the way, if you rely on private packages, consider subscribing to Private Packagist, a nice service created by the Composer team specially dedicated for this use case.

To copy this private repository into the vendor/ directory of the Docker image, use the following Dockerfile:

FROM php:alpine

RUN apk add --no-cache openssh-client git
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

# Install Composer
COPY --from=composer /usr/bin/composer /usr/bin/composer

# Copy the composer.json and composer.lock file
COPY composer.* .
# Install the dependencies and allow Docker to use the SSH credentials of the host
RUN --mount=type=ssh composer install --no-progress --no-interaction

If you already use the Symfony Docker skeleton or API Platform, you can adapt the provided Dockerfiles: add the required packages, add the SSH keys of your forge, and add `–mount=type=ssh` to the existing RUN instruction doing a composer install.

Docker Compose

Docker Compose now natively supports SSH forwarding! Use the new --ssh default flag to let the builder use the SSH connection of the host:

docker compose build --ssh default

GitHub Actions and GitLab CI

Finally, you’ll probably want to build the Docker image directly into your CD pipeline.

To achieve this, the first step is to create deployment keys for your private Git repositories. Deploy keys are special SSH keys that grant read-only access to Git repositories.

First, generate a pair of public and private keys for each private repository. Then upload the public key on the forge and associate it with the private repository:

After that, store the private key as a CI secret:

Finally, each time you start a job building Docker images, copy the private key from the secret variable to SSH and start an SSH agent. To do so, I strongly recommend using the ssh-agent GitHub Action, which automates this process:

# .github/workflows/build-image.yml
jobs:
    build:
        steps:
            - uses: actions/checkout@v2

            - uses: webfactory/[email protected]
              with:
                  ssh-private-key: ${{ secrets.MY_PRIVATE_DEPLOY_KEY }}

            - run: docker compose build --ssh default

You should now be able to safely build Docker images containing private code!

If you need help setting up fast and secure CI/CD pipelines, optimizing your Docker images, or creating platforms on top of Kubernetes, don’t hesitate to contact the great DevOps team at Les-Tilleuls.coop!

If you liked this post, consider sponsoring me on GitHub.