Kubernetes on Google, Azure and AWS Compared

There have been many comparisons done between these cloud hosted Kubernetes providers already. However, probably none as honest as this one.

Below is a screenshot of the Google sheet comparing GKE, AKS and EKS. You may notice that some of the cells have comments in already. These comments link to the place I got the information from.

If anything is incorrect please add a comment to the sheet or drop me a message below.

This is going to be a brutal comparison and I’m happy to change my mind if somebody can present some factual evidence.

As it stands today I’ve personally used EKS, and AWS in general a lot. I’ve used GKE a bit but only with my own personal credits doing Kubernetes The Hard Way and spinning up a very quick GKE test cluster a while ago. I’d like to use it more but we’re on AWS at work.

Azure is something I’ve avoided since using it for a few months last year. I was working as a Microsoft partner so it was unavoidable back then. Parts of it are alright but the user experience coming from an Amazon background is worlds apart.

I’ll split my recommendation up into a few categories based on scenario.

Greenfield Projects

I’ve been trying to weigh up whether to advise people to review all of the services across Google, AWS and Microsofts clouds. Kubernetes is a single component of a larger system and other factors will come into play. However, there really isn’t much difference between what each cloud offers nowadays.

Marketing teams may claim otherwise but when you look at what most companies need in the way of features they all pretty much have it covered.

So let’s simplify things and say you only want to run Kubernetes. The answer is clear. Go with GKE every time.

Why? It’s cheaper, faster and better in almost every single measurable way than the competition.

You may look at the 3 minutes vs 20 minute cluster creation time and think it’s not a big deal. How many times do you really build a cluster anyway? Well, not very often if they take 20+ minutes and sometimes fail because you will change your entire workflow to fit around it.

If, however, clusters only take 3 minutes to create you will find yourself using disposable test infrastructure and working in an entirely different way.

Networking is the other reason. Google is miles ahead of everyone here. Similar story with HA and scaling.

I won’t call out every row from the spreadsheet as a reason as I think the data speaks for itself.

We’re already locked into AWS

You probably don’t have much of a choice. Kubernetes on its own isn’t reason enough to do a cloud migration.

Hopefully, you’ll see this comparison and know that life could be worse. EKS is still new and the Amazon teams iterate quickly. I doubt Amazon will ever match Google for network speed or VM boot times but your life will still be tolerable.

It would be nice if they could enable the default Admission Controllers on the masters. Also, I have no idea why cluster creation is so slow. I’ve measured cluster build times at work and they always take around 20 minutes to build to the point I can run integration tests on them.

The other annoyance is the lack of worker management. Hopefully the EKS guys can package this up like how GKE does it.

An interesting thing I noticed was that EKS supports i3.metal instances so this is the only Kubernetes solution that supports bare metal workers. This may be useful for some people.

My company is looking at Azure

Again, not really much you can do. My advice is to setup a demo with AKS and GKE and try to convey the developer experience to the decision makers.

Show them how some things on Azure need to be done in a clunky web UI, other things need Powershell and other random stuff uses the CLI. Try to get people to account for the slowness of operations and how that effects the design of DevOps pipelines and automation in general. Yes, you can make it work, but why make life hard for yourself.

It’s possible you’re locked in with something like Active Directory as a requirement which means you’re probably out of luck.

I’m being serious when I say this: if the company I’m working for decided to migrate to Azure I’d find a new job. By the way, this wouldn’t have always been the case. Back in my youth I managed Windows servers, I fixed broken TFS installations and I walked into server rooms and replaced tapes every week.

Nowadays I want to programmatically control infrastructure. It needs to be fast and bug free so that I can build cool automation on top. Working on something like Azure, especially after having worked on AWS for years, would be extremely depressing.

To all of you who are moving from on-premise into Azure it could be a step up in terms of agility so I wish you the best of luck.

So there you have it! This site needed a comparison of Kubernetes options that run in the cloud. My recommendation is go with Google GKE whenever possible. If you’re already on AWS then trial EKS but it doesn’t really give you that much currently. You may be better off looking at Kops or some other cloud installer until they add managed workers and other integrations.

Additional Reading

More information about each cloud offering from Kuberiter : https://resources.kuberiter.com/k8scpcchart

Another good comparison although a bit out of date now: https://blog.hasura.io/gke-vs-aks-vs-eks-411f080640dc

The guys at Caylent provide some nice bullet points: https://caylent.com/kubernetes-managed-service/

Good cost comparison article: https://www.replex.io/blog/the-ultimate-kubernetes-cost-guide-aws-vs-gce-vs-azure-vs-digital-ocean