Optimize Dokku Deployment Speed for Ruby on Rails Apps with Dockerfile
source link: https://www.tuicool.com/articles/hit/MFZvaqI
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Dokku lets you setup Rails hosting infrastructure on a simple VPS without much dev ops experience. Although it is easy to get started, a default config might result in very slow and unreliable deployments. In this blog post, I will describe how I’ve improved my Dokku based Ruby on Rails (NodeJS with Yarn and Webpack) application deployment speed by over 400% using a Docker image Dockerfile.
Getting started with Dokku
If you are not familiar with Dokku, you should check out one on myprevious blog posts to get up and running quickly. Once you have a simple Rails app hosted on buildpacks based Dokku setup, you can follow rest of this tutorial to significantly reduce deployments speed.
But first, let me explain why default Dokku deployment tends to be so slow.
“There’s a Heroku buildpack for that…”
Every non-trivial Rails app has multiple system level dependencies. Dokku provides support for them using so-called buildpacks
, the same approach that Heroku is using.
My new project
was using the following .buildpacks
file:
https://github.com/gaffneyc/heroku-buildpack-jemalloc.git https://github.com/heroku/heroku-buildpack-nodejs.git https://github.com/heroku/heroku-buildpack-ruby.git
- Ruby - a default buildpack for Rails apps
- NodeJS - required for Rails apps using Webpacker and Yarn for frontend dependencies
- Jemalloc - a must-have for modern Rails apps, out of the box reduces memory usage by ~20%
I was also using Chrome Puppeteer buildpack for a moment but eventually switched to Browserless.io on a separate CPU optimized VPS .
But why buildpacks are so slow?
Buildpack is a list of commands that are executed during a deploy. I am not too much of a Docker expert to understand exactly what’s going on under the hood, but let’s take a look at a sample console output:
Downloading and installing node 8.11.4... ... Downloading and installing yarn (1.10.0)... ... Compiling Ruby/Rails ...
As you can see by default buildpacks are not too smart about caching and every single deployment downloads and recompiles some of the application dependencies. No wonder it is slow, and probably not the best way to use your application’s VPS CPU and RAM.
Digital Ocean monitoring shows CPU usage spikes during deployments
Speed up the deployment with Dockerfile
There is an alternative solution. You can use a custom Docker image instead of multiple buildpacks. The official Dokku documentation
mentions it very briefly and describes as a "Power User"
feature.
In practice it’s just a matter of adding one config file and running a couple of bash commands. Switching to Dockerfile reduced deployment time of Scraper App from over 8 minutes to less than 2.
I will explain how to set this up for a sample Ruby on Rails app using Ruby 2.5.1, NodeJS 8.x.x LTS with Yarn 1.9.4.
Use an official Ruby Docker image
You need to start with adding a Dockerfile
file to the root of your application folder with the following contents:
FROM ruby:2.5 # Install NodeJS and Yarn RUN apt-get update RUN apt-get -y install curl RUN apt-get install -my gnupg RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list RUN apt-get update && apt-get -qqyy install nodejs yarn && rm -rf /var/lib/apt/lists/* # Install Ruby Gems and node modules COPY Gemfile* /tmp/ COPY package.json /tmp/ COPY yarn.lock /tmp/ WORKDIR /tmp RUN bundle install --jobs 5 --retry 5 --without development test RUN yarn install RUN mkdir /app WORKDIR /app COPY . /app ENV RAILS_ENV production ENV RACK_ENV production # Execute the Procfile CMD ["bin/run-dev.sh"]
Copying Ruby gems and node modules files to /tmp, effectively caches them reducing deployment time.
It installs NodeJS and Yarn on top of the official Ruby 2.5 Docker image
Don’t forget to precompile the assets by adding it as a predeploy step in app.json
file:
{ "name": "My Rails app", "scripts": { "dokku": { "predeploy": "bundle exec rake assets:precompile", "postdeploy": "bundle exec rake db:migrate" } } }
Now you just need to remove the .buildpacks
file if you were using it before and remove one config variable:
dokku config:unset --no-restart DOKKU_PROXY_PORT_MAP
When you do a git push to dokku remote your Ruby on Rails app will use a Dockerfile instead of buildpacks:
In practice assets precompilation takes ~90% of deployment time. All the Dockerfile steps are cached and executed almost instantly after the initial deploy.
Use a custom Docker image with preinstalled dependencies
Alternatively, you could use my Ruby Jemalloc/NodeJS/Yarn buildpack ( [Disclaimer] I am not a dev ops pro. Tips/PRs on how this image could be improved are welcome. ). It has an additional advantage of using Ruby binary compiled with Jemalloc for reduced memory usage and NodeJS with Yarn is already in place:
FROM pawurb/ruby-jemalloc-node-yarn:latest COPY Gemfile* /tmp/ COPY package.json /tmp/ COPY yarn.lock /tmp/ WORKDIR /tmp RUN bundle install --jobs 5 --retry 5 --without development test RUN yarn install RUN mkdir /app WORKDIR /app COPY . /app ENV RAILS_ENV production ENV RACK_ENV production CMD ["bin/run-dev.sh"]
You could also build and publish Dockerfile image yourself but that’s outside the scope of this tutorial.
Caveats
ENV
variables defined by dokku config:set
command are not available during Dockerfile based deployments build time. bundle exec rake assets:precompile
predeploy step will launch your Rails app process, and things could fall apart if some of the required ENV
variables are missing.
You should set them using this command:
dokku docker-options:add build '--build-arg AWS_SECRET_KEY=12345' dokku docker-options:add build '--build-arg AWS_ACCESS_KEY=67890'
It’s not a perfect solution because requires you to duplicate config but works for my use case and must be done only for nonoptional variables.
Useful commands
Here’s a list of commands that might come in handy if you get stuck along the way:
docker ps // display docker containers docker image prune // remove unused docker images docker stats // display CPU/memory usage of running docker containers docker kill $(docker ps -q) // stop all docker containers docker rm $(docker ps -a -q) // remove not running docker containers docker system prune // general cleanup
Summary
Playing directly with Dockerfile images is a bit lower level than using a default Dokku buildpacks based approach. For me, the speed and reliability of Dockerfile powered deployments was more than worth the effort.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK