Spaced Out & Smiling

Spaced Out & Smiling

I am lucky in that I have a few side projects which have grown up to become full fledged companies. One of them builds little tools to detect different types of financial crime.

As part of the development cycle I run a test suite. The test suite is pretty extensive as we follow the Test Driven Development & Behaviour Driven Development approach. For every feature in the project there are often a half dozen tests. Its a big suite.

After almost every change to the code i tun the tests. All of them… Being able to run the full suite everytime helps me to pick up issues early and iterate quickly. The faster the tests run, the more i can get done.

I mostly work on Macs and we recently purchased a 16” MacBook Pro with an M1 Pro CPU for me to do my development on. It’s a beast of a thing, with 10 cores and more RAM and Flash Storage than i know what to do with.

The test suite is embaressingly “parallel”. Each test can run on its own core so the more cores we have the faster the tests will run. My current 10 core Mac runs them much faster than my 4 core Mac etc.

Graviton

Given i develop on a ARM Mac, i’d like to deploy to an ARM server. Right now i have to build and deploy using a cloud server or an older Intel MacBook i keep in a draw. Not super elegant!

Amazon have been trying to solve this issue (and many others) by developing a line of ARM servers call Graviton. I recently gave one a try. I was curous to see how the very fastest 64 core graviton server compared to my 10 core 16” MacBook Pro and my 8 core M1 Mac Mini.

The test was reasonably simple. I installed node v16 on both my MacBook and the cloud instance, installed our app and then ran the full end to end test suite.

The full suite executes 1196 unit test and 350 end to end tests. The unit test are run in series on one core, while the BDD tests are run in paralell on all the remaining cores.

I wanted to get an idea for if the graviton cores where quick enough to run our workload, but also too see if it would be worth running our build and deploy scripts in the cloud.

 Results

The results are really interesting.

Lets start with the M1 Mac mini. It’s a nice baseline as it’s the Mac i was developing on up untill recently. The Mac mini completed the tests in 45.13 seconds across 8 cores.

The M1 Pro MacBook compared well with the M1. The additional faster cores in the MacBook Pro delivering a score of 31.84 seconds across the 10 cores. Around 31% faster than the M1 Mac mini

The AWS Graviton instance (c6g.metal to be specific) delivered a score of 14.63 seconds across its 64 cores. Around 68% faster than the Mac mini.

I found all three options pretty impressive for different reasons.

The Mac mini is impressive because of the value. From a CPU perspective at least, the 8 core Mac Mini cost about £700 to own forever. Whereas the fastest ARM server AWS makes costs £13,831 just to rent it for one year. The Mac mini is 3x slower, but almost 20x cheaper. Thats pretty amazing value!

The M1 MacBook is impressive on the performance per core & value. Roughly speaking each Apple CPU core is performing twice as quickly as each AWS Graviton core. It’s 1.5x slower than the AWS server, but it’s ~6x cheaper. Thats remarkable value and makes me very excited for higher core count ARM Macs in the future. A 24 or 32 core could be astonishingly quick!

Finally the Graviton impressed me for the outright pace. Having 64 cores to split the task up allowed it to deliver the best result of the night and take the crown as the fastest CPU ever to run the test suite.

Final thoughts

This was a fun little experiment with some neat practical uses. The Graviton platform is more than fast enough for us to move our production workload too. It’s not fantastic value, but it wont break the bank either.

With the Graviton processes being “good enough” i can also move my whole development pipeline to be ARM end to end. That means i can use the fast Mac on my desk to write code, but also to build the docker container and run the release scripts. That’s a really nice milestone in the apps development and means i can finally retire the Intel Mac which is gathering dust in the draw.

All in all, a fun afternoon of experimentation!