Surgically Busting the Docker Cache

What is ‘Busting the Cache’?

If you’ve ever spent any time building Docker images, you will know that Docker caches layers as they are built, and as long as those lines don’t change, Docker treats the outputted layer is identical

There’s a problem here. If you go to the network to pick up an artefact, for example with:

RUN curl https://myartefactserver.local/myjar.jar > myjar.jar

then Docker will treat that command as cache-able, even if the artefact has changed.

Solution 1: –no-cache

The sledgehammer solution to this is to add a --no-cache flag to your build. This removes the caching behaviour, meaning your build will run fully every time, no matter whether the lines of your Dockerfile change or not.

Problem solved? Well… not really. If your build is installing a bunch of other more stable artefacts, like this:

FROM ubuntu
RUN apt-get update -y && apt-get install -y many packages you want to install
# ...
# more commands
# ...
RUN curl https://myartefactserver.local/myjar.jar > myjar.jar
CMD ./run.sh

Then every time you want to do a build, the cycle time is slow as you wait for the image to fully rebuild. This can get very tedious.

Solution 2: Manually Change the Line

You can get round this problem by dropping the --no-cache flag and manually changing the line every time you build. Open up your editor, and change the line like this:

RUN [command]  # sdfjasdgjhadfa

Then the build will But this can get tedious.

Solution 3: Automate the Line Change

But this can get tedious too. So here’s a one-liner that you can put in an alias, or your makefile to ensure the cache is busted at the right point.

First change the line to this:

RUN [command] # bustcache: 

and then change your build command to:

perl -p -i -e "s/(.bustcache:).*/\1 $RANDOM/" Dockerfile && docker build -t tag .

The perl command will ensure that the line is changed to a random number generated by the shell.

There’s a 1/100,000 chance that the number will repeat itself in two runs, but I’m going to ignore that…

---

If you like this, you might like one of my books:
Learn Bash the Hard Way
Learn Git the Hard Way
Learn Terraform the Hard Way

Get 39% off Docker in Practice with the code: 39miell2

---