Google engineers on Thursday posted initial "request for comments" patches on their KVM-CPUFreq driver that is part of their effort to improve the dynamic voltage and frequency scaling behavior and task placement within KVM-based virtual machines. This effort is leading to big improvements in raw performance and performance-per-Watt for tasks running within Linux VMs.

David Dai of Google wrote yesterday on the kernel mailing list:

This patch series is a continuation of the talk Saravana gave at LPC 2022 titled "CPUfreq/sched and VM guest workload problems". The gist of the talk is that workloads running in a guest VM get terrible task placement and DVFS [Dynamic Voltage and Frequency Scaling] behavior when compared to running the same workload in the host. Effectively, no EAS [Energy Aware Scheduling] for threads inside VMs. This would make power and performance terrible just by running the workload in a VM even if we assume there is zero virtualization overhead.

We have been iterating over different options for communicating between guest and host, ways of applying the information coming from the guest/host, etc to figure out the best performance and power improvements we could get.

The patch series in its current state is NOT meant for landing in the upstream kernel. We are sending this patch series to share the current progress and data we have so far. The patch series is meant to be easy to cherry-pick and test on various devices to see what performance and power benefits this might give for others.

With this series, a workload running in a VM gets the same task placement and DVFS treatment as it would when running in the host.

As expected, we see significant performance improvement and better performance/power ratio. If anyone else wants to try this out for your VM workloads and report findings, that'd be very much appreciated.

The idea is to improve VM CPUfreq/sched behavior by:
- Having guest kernel to do accurate load tracking by taking host CPU arch/type and frequency into account.
- Sharing vCPU run queue utilization information with the host so that the host can do proper frequency scaling and task placement on the host side.

The benchmark results are very promising such as when running Android within a VM on a Chromebook that PCMark experienced a 19% boost in performance for its weighted total or as much as 45% faster for photo editing. The Roblox performance benchmark was up by 57% with KVM-CPUFreq while power efficiency improved by 26%. More details on this KVM-CPUFreq work via this RFC kernel patch series now under discussion.