A Complete OpenShift Cluster on Vagrant, Step by Step

tl;dr

Following on from my Kubernetes post here, I have automated an OpenShift Origin cluster using the same tools.

Video

Here is a video of the whole process.

It gets (relatively) interesting later on, as a lot of the process is Vagrant starting up and yum installs failing on bad mirrors. Also, Ansible needs to be run several times for it to work (I suspect due to resource limitations, see Gotchas below).

Architecture

Here is a layout of the VMs. The host uses the landrush plugin to allow transparent DNS lookup from the host, and between boxes.

The code is here:

Run Yourself

You will need at least 6.5G spare memory (maybe more) on your host. Even then it may struggle to provision in a timely way.

Do get in touch if you think you can help improve it.

Tech Used

• Vagrant (Virtualbox)
• ShutIt
• Ansible

I am interested in porting to libvirt also. Please get in touch if you want to help.

One of the big problems with running OpenShift in production is the complexity of each environment. You can have test, UAT and prod environments, but sometimes you want to quickly spin up a realistic environment for development or

At that point you’re usually offered an ‘all-in-one’ or single-command setup, which, while very convenient, doesn’t represent the reality of the system you’re running elsewhere.

This is less didactic than the Kubernetes post (the steps to set up take a good while to run even if you’re using ansible…) but still has its uses.

Because this is in vagrant and is automated, it gives you a reliable, fast, and realistic representation of a real live infrastructure. This comes in very handy if you’re trying to determine the memory usage of etcd, the effect of tuning some config variables, or failover scenarios.

Gotchas

Here are some of the things I had to overcome to make this work. They’re fairly instructive:

• OpenShift requires DNS to work. Landrush to the rescue!
• Getting the memory and CPU-sizing right so it would fit on my 12G server. master1 does the heavy ansible lifting, so that gets 2G and 4CPUs
• The ansible playbook runs three times before it works. I suspect this is because of the memory limitations. I’ll find out when my 32G tower arrives… Or can anyone help me improve it?
• centos/7 VMs don’t allow password logins so I had to tinker with sshd
• Setting up keys across all the servers so ansible could work
• Vagrant networking and ansible. Yuk.

Learn More

My book Docker in Practice:

Get 39% off with the code: 39miell