AMD EPYC Performance Testing… or Don’t get on the wrong side of SystemD

Ever since AMD released their EPYC CPU for servers I wanted to test it, but I did not have the opportunity until recently, when Packet.net started offering  bare metal servers for a reasonable price . So I started a couple of instances to test Percona Server for MySQL under this CPU. In this benchmark, I discovered some interesting discrepancies in performance between  AMD and Intel CPUs when running under systemd .

The set up

To test CPU performance, I used a read-only in-memory sysbench OLTP benchmark, as it burns CPU cycles and no IO is performed by Percona Server.

For this benchmark I used Packet.net c2.medium.x86 instances powered by AMD EPYC 7401P  processors. The OS is exposed to 48 CPU threads.

For the OS I tried

• Ubuntu 16.04 with default kernel 4.4 and upgraded to 4.15
• Ubuntu 18.04 with kernel 4.15
• Percona Server started from SystemD and without SystemD (for reasons which will become apparent later)

To have some points for comparison, I also ran a similar workload on my 2 socket Intel CPU server, with CPU: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz . I recognize this is not most recent Intel CPU, but this was the best I had at the time, and it also gave 48 CPU Threads.

Ubuntu 16

First, let’s review the results for Ubuntu 16

Or in tabular format:

Threads Ubuntu 16, kernel 4.4; systemd Ubuntu 16, kernel 4.4;

NO systemd

Ubuntu 16, kernel 4.15

1 943.44 948.7 899.82 2 1858.58 1792.36 1774.45 4 3533.2 3424.05 3555.94 8 6762.35 6731.57 7010.51 12 10012.18 9950.3 10062.82 16 13063.39 13043.55 12893.17 20 15347.68 15347.56 14756.27 24 16886.24 16864.81 16176.76 30 18150.2 18160.07 17860.5 36 18923.06 18811.64 19528.27 42 19374.86 19463.08 21537.79 48 20110.81 19983.05 23388.18 56 20548.51 20362.31 23768.49 64 20860.51 20729.52 23797.14 72 21123.71 21001.06 23645.95 80 21370 21191.24 23546.03 90 21622.54 21441.73 23486.29 100 21806.67 21670.38 23392.72 128 22161.42 22031.53 23315.33 192 22388.51 22207.26 22906.42 256 22091.17 21943.37 22305.06 512 19524.41 19381.69 19181.71

There are few conclusions we can see from this data

1. AMD EPYC CPU scales quite well to the number of CPU Threads
2. The recent kernel helps to boost the throughput.

Ubuntu 18.04

Now, let’s review the results for Ubuntu 18.04

Threads Ubuntu 18, systemd

Ubuntu 18, NO systemd

1 833.14 843.68 2 1684.21 1693.93 4 3346.42 3359.82 8 6592.88 6597.48 12 9477.92 9487.93 16 12117.12 12149.17 20 13934.27 13933 24 15265.1 15152.74 30 16846.02 16061.16 36 18488.88 16726.14 42 20493.57 17360.56 48 22217.47 17906.4 56 22564.4 17931.83 64 22590.29 17902.95 72 22472.75 17857.73 80 22421.99 17766.76 90 22300.09 17773.57 100 22275.24 17646.7 128 22131.86 17411.55 192 21750.8 17134.63 256 21177.25 16826.53 512 18296.61 17418.72

This is where the result surprised me: on Ubuntu 18.04 with SystemD running Percona Server for MySQL as a service the throughput was up to 24% better than if Percona Server for MySQL is started from a bash shell. I do not know exactly what causes this dramatic difference—systemd uses different slices for services and user commands, and somehow it affects the performance.

Baseline benchmark

To establish a baseline, I ran the same benchmark on my Intel box, running Ubuntu 16, and I tried two kernels: 4.13 and 4.15

Threads Ubuntu 16, kernel 4.13, systemd Ubuntu 16, kernel 4.15, systemd

Ubuntu 16, kernel 4.15, NO systemd

1 820.07 798.42 864.21 2 1563.31 1609.96 1681.91 4 2929.63 3186.01 3338.47 8 6075.73 6279.49 6624.49 12 8743.38 9256.18 9622.6 16 10580.14 11351.31 11984.64 20 12790.96 12599.78 14147.1 24 14213.68 14659.49 15716.61 30 15983.78 16096.03 17530.06 36 17574.46 18098.36 20085.9 42 18671.14 19808.92 21875.84 48 19431.05 22036.06 23986.08 56 19737.92 22115.34 24275.72 64 19946.57 21457.32 24054.09 72 20129.7 21729.78 24167.03 80 20214.93 21594.51 24092.86 90 20194.78 21195.61 23945.93 100 20753.44 21597.26 23802.16 128 20235.24 20684.34 23476.82 192 20280.52 20431.18 23108.36 256 20410.55 20952.64 22775.63 512 20953.73 22079.18 23489.3

Here we see the opposite result with SystemD: Percona Server running from a bash shell shows the better throughput compared with the SystemD service. So for some reason, systemd works differently for AMD and Intel CPUs. Please let me know if you have any ideas on how to deal with the impact that systemd has on performance.

Conclusions

So there are some conclusions from these results:

1. AMD EPYC shows a decent performance scalability; the new kernel helps to improve it
2. systemd shows different effects on throughput for AMD and Intel CPUs
3. With AMD the throughput declines for a high concurrent workload with 512 threads, while Intel does not show a decline.