As you can see from the title that’s a lot of configuration going on here. Basically, I am trying to set up a very simple development workflow using very fully customizable (and cheap) configuration.

Why in Windows Azure? I have a running commercial project and paid a subscription so why not also use it for development purposes.

But before diving into the technical details let’s try to visualize it. That’s what I wanted to achieve:

Ubuntu 18.04 and disk storage

Actually, the configuration is quite straight forward. We have Ubuntu 18.04 virtual machine running in Microsoft Azure. 1TB external storage mounted as /apps/data (about the steps to mount it I will talk a bit later). This storage is needed to keep our configuration and data persistent. Hence, I configured it as managed and premium storage. In terms of Microsoft Azure, shortly, managed disk storage means it is highly durable and available, has integration with availability zones (to avoid a single point of failure) and eligible for Azure Backup support. It’s also Premium SSD which deliver high performance and low-latency disk support for VMs.

VM with Docker/Docker Compose and mounted data storage

Check this article to setup and run Ubuntu with Docker in Microsoft Azure. Once Docker installed, follow this article to install Docker Compose as well.

As a next step, we need to setup disk for our VM. Check this article on how to add a data disk to Linux VM. For the instructions to actually mount the disk to Linux VM I used these 3 articles:

My final fstab configuration for the disk looked like this:

UUID=uuid-of-disk /apps/data ext4 defaults,nofail 0 1

Running Jenkins in Docker/Docker Compose with external volume

Now, when we have all configured VM with Docker/Docker Compose, it is time to add our Docker configuration. Our Docker Compose file looks like this:

version: "3.2"services:
  jenkins:
    build:
      context: .
      dockerfile: ./Dockerfile
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - /apps/data/jenkins-data:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
    restart: always

The most important and interesting part here is how we setup volumes. But about this a bit later. Let’s see Dockerfile that we use to build an actual image:

FROM jenkinsci/blueoceanUSER rootRUN addgroup jenkins && adduser -D -G jenkins jenkinsUSER master

It’s based on jenkinsci/blueocean and what we do here is a little trick to deal with user permissions.

As you know we will use persistent volume to keep our Jenkins data and it will be stored in mounted storage. The thing is actual files in the volume is stored in a host machine and used in Docker containers. To avoid permission errors you need to have proper user and group configuration. And that’s why in Dockerfile we add a new user called jenkins with jenkins group. Official image from is also using this user. More about this you can read here.

And in the host machine, for the directory /apps/data/jenkins-data, we have jenkins:jenkins user and group. To do this you can use the following command:

chown -R jenkins:jenkins /apps/data/jenkins-data

Hence, when you mount this directory to the container (see docker compose conf) you have the same user and group and as a result, you do not get permission denied error.

In Ubuntu 18.04 to create this user and group you can use following commands:

groupadd jenkins
useradd -G jenkins jenkins

And finally to our docker container use the following command

docker-compose up -d

restart: always configuration in docker compose will make sure that it is always running after system or docker reboot.

Summary

I tried very hard to keep this article short and to the point. I highly recommend checking articles on Docker volumes and how they work. Please check following links for this:

https://success.docker.com/article/different-types-of-volumes
https://docs.docker.com/storage/volumes/
https://docs.docker.com/docker-for-azure/persistent-data-volumes/
https://docs.docker.com/storage/bind-mounts/
https://stackoverflow.com/questions/34357252/docker-data-volume-vs-mounted-host-directory

Also, once you installed Docker in Linux machine make sure to check post-installation steps here: https://docs.docker.com/install/linux/linux-postinstall/.