Greetings! We’re pleased to announce that SPEC has released the SPECvirt® Datacenter 2021 benchmark, a new multi-host benchmark for measuring the performance of a scaled-out datacenter. As a founding member of the SPEC Virtualization committee, VMware contributed to its design, development, testing, and support on VMware vSphere® 6.7 and 7.0 platforms.

The SPECvirt Datacenter 2021 benchmark uses real-world and simulated workloads to measure the overall efficiency of virtualization solutions and their management environments. It provides a methodical way to measure a virtualization platform’s performance in a dynamic virtualized datacenter environment. It models typical, modern-day use of virtualized infrastructure, such as VM resource provisioning, cross-node load balancing (including management operations such as VM migrations), and VM power on/off.

First result published by Lenovo

Lenovo has published the world’s first SPECvirt Datacenter 2021 benchmark result. It utilizes VMware ESXi ™ 7.0 U2a and VMware vCenter Server® 7.0 U2b on four ThinkSystem SR665 servers and the ThinkSystem DE6000H Hybrid Storage Array. The Lenovo solution achieved the following score:

SPECvirt Datacenter-2021 5.952 per host @ 4 Hosts

The primary metric is the sum of the normalized throughput score of all tiles, which are shown in figure 1 below, divided by the number of hosts in the system under test. The workload mix features light, medium, and heavy workloads. Individual workload scores are weighted accordingly to prevent undue influence of any one workload on the primary metric. Each individual workload score must meet the quality-of-service criteria defined for each workload as well as any other validation that the workloads require.

SPEC virt_sc® 2013 vs. SPECvirt Datacenter 2021

SPEC virt_sc 2013 remains available and measures the performance of single server consolidation scenarios. Differences between the two are:

SPEC virt_sc 2013  SPECvirt Datacenter 2021 

Single host  Clusters in multiples of four 

User-defined software stack and virtual machine resource settings  Requires SPEC’s CentOS VM template distributed as OVF. Deploys VMs through hypervisor API 

Stresses host hypervisor scheduler under load  Stresses management server resource scheduler under load (DRS), as well as individual host schedulers

Complex workload VM configuration and setup  Requires no configuration beyond setting system resources. Eliminates any performance variation of the OS and software stack. Allows for hypervisor tuning only. 

Benchmark implementation

SPECvirt Datacenter 2021 compares performance and scalability of various server platforms and storage solutions, helping to make appropriate hardware choices and measuring platform performance on an ongoing basis. At the time of release, the benchmark supports VMware vSphere and Red Hat® Virtualization platforms.

Figure 1. Multi-tile and harness configuration

SPECvirt Datacenter 2021 benchmark feature overview:

• Multi-host benchmark – Minimum of four hosts required, scales in increments of four
• Datacenter operations model – Multi-workload benchmark measures performance of hypervisor infrastructure, including how the hypervisor manager controls resources
• Five real-world and simulated workloads –
  • OLTP database, based on HammerDB benchmark
  • Hadoop/Big Data cluster, based on BigBench benchmark
  • Simulated departmental mail server
  • Simulated departmental web server
  • Simulated departmental collaboration server
• VM resource management – Handled by the hypervisor manager, including scheduling policies. Workload VMs are powered on or deployed from template during the benchmark.
• Ease of use – Single pre-configured template VM to set up harness and workloads. Guest OS/software is pre-tuned, and the workloads are pre-installed. The harness uses the LiveView GUI (see below) to monitor the measurement while in progress.

Figure 2. The LiveView user interface

You can learn more about the benchmark and see the blog entry. To learn more about the Lenovo result, see Lenovo’s announcement and SPECvirt Datacenter 2021’s results page.