Steve Worley of the System x Performance and Analysis Benchmarking group at IBM just released a paper that we here at the Intel Ethernet Virtualization team are pretty excited about.
This paper, entitled Effect of SR-IOV Support in Red Hat KVM on Network Performance in Virtualized Environments discusses how Steve used the SPECvirt_sc2010 benchmark to test performance when using SR-IOV and when not using SR-IOV.
They ran a variety of workloads; here is a quote from the document detailing the workloads:
The benchmark uses several workloads representing applications that are commonly consolidated into a virtualized environment. Scaling is achieved by running additional sets of virtual machines, called “tiles,” until overall throughput reaches a peak. Each tile consists of six different virtual machines:
• Infraserver – Serves file downloads directory for web workload, runs web backend simulator
• Webserver – Runs Web workload
• Mailserver – Runs Internet Message Access Protocol (IMAP) workload
• Appserver – Runs application server workload
• Dbserver – Runs database as backend to application server workload
• Idleserver – Runs a poll workload
Because each tile represents six VMs, a system running 10 tiles would consist of 60 VMs.
The article itself does not indicate which SR-IOV Ethernet controller is used for the test, however the detailed configuration document that it links to does. The Intel® 82576 Gigabit Ethernet Controller was used for these tests.
The results of the tests are very interesting. It shows that SR-IOV definitely reduces CPU overhead and that by doing so, it can increase QoS due to the fact that the CPU utilization is reduced.
The following chart is a graphical representation of the data published within the paper by IBM:
SR-IOV Data Results
Note that there is no data for 13 Tiles in non-SR-IOV mode – this is because the CPU utilization was so high that the test run could not be completed. The details of each of the tests can be found here.
So What Does This Mean
Good question. It does not mean that the SR-IOV enabled network path is a replacement for the Emulated path for all workloads. In fact, if you dig into the details of the test, IBM seems to have picked and chosen specific workloads to use SR-IOV while leaving the others on the Emulated path.
The current ecosystem for SR-IOV today has its limitations – one of the major ones is that live migration is not currently supported (though we are working with the industry on this right now). For this reason alone, SR-IOV may not be suitable for all virtualized workloads.
However, as these test results show, there are definitely some cases for some customers where the performance benefits gained by using SR-IOV outweigh the current limitation associated with it.
For more information, I suggest you carefully read the document by Steve Worley entitled Effect of SR-IOV Support in Red Hat KVM on Network Performance in Virtualized Environments, and take a close look at the SPECvirt_sc2010 Result information.
BTW… If you take a look at the other test results posted on the SPECvirt_sc2010 site, all of the systems are using Intel® Ethernet Controllers for their primary benchmark network;
- Intel® Ethernet Server Adapter X520 10GbE dual-port with SR-IOV support
- HP* NC365T 4-port Server Ethernet Adapter that uses Intel® Ethernet 82580 Controller.
- Intel® Gigabit ET Adapter that uses the Intel® Ethernet 82576 Controllers with SR-IOV.
Curious about what SR-IOV is, and how it works? We have a blog on that!