If there is one thing that has stayed consistent in the computing industry over time, it's that performance doesn't stand still. As our computing platform processing, I/O, and memory speeds continue to accelerate, it is important to remember a little thing called latency.
Often in the Ethernet world throughput is the 1st and last performance metric of choice. 1 Gigabit and 10 Gigabit are the numbers that inspire thoughts of increased performance, and improved computing power. However, it's important to note that, in many applications, the transaction latency over the wire is really the key to unlocking high performance at the system level. One of the primary reasons that some organizations have turned to Infiniband and other I/O technologies for HPC and clustering in the past has to do with their desire to achieve very low latencies, not necessarily increased throughput. If you look at a historical standard Gigabit Ethernet connection, you may see latencies that are around 125μs. This may have been ok in the past, but as improvements at the application level as well in the system hardware and CPU take hold, legacy Ethernet won't be good enough for HPC and clustering environments.
The interesting, and often overlooked fact with Ethernet is that the latency characteristics are improving as the industry moves from 1 Gigabit to 10 Gigabit. The faster throughput on the wire comes along with lower latency to some extent, but in addition, there have been several improvements in interrupt handling that drastically improve overall latencies when comparing legacy 1Gigabit to 10Gigabit. With a basic 1st generation Intel® 10Gigabit CX4 card you can now see latencies approach 25μs without any special tuning.
What's even better is that Intel's 10 Gigabit networking silicon also has further enhancements for improving latency by introducing some new specialized Low Latency Interrupt (LLI) filters in the silicon. These filters provide the hardware with a quicker reaction time to network packets that meet certain customizable criteria. The filters can be tuned to have a rapid response to certain packet and traffic types. With these kinds of LLI filters in place, latencies can be reduced further by another ~50% to ~14μs.
Going forward with 10 Gigabit there are new technologies and designs that can help push latency even lower to the sub-10μs threshold to keep Ethernet very competitive as a fabric not only from a cost and throughput perspective, but also from the perspective of latency.
And while lower latency is certainly important, the last piece that was really missing from the Ethernet performance puzzle was not just low latency, but deterministically low latency. The key is that the worst case packet latencies for many applications are relevant and very important. By application thread affinitization, the individual data thread can be piped directly between a network queue and a CPU core. By more evenly distributing the networking workload between CPU cores in a predictable fashion, you get a deterministic kind of latency that does not stray far from the average assuming CPU cores do not get oversubscribed. Average latency of ~14μs is good, but the fact that you can get this with reasonable determinism is a key for many applications and usages.
Now, lower, deterministic latency is not just a theoretical benefit for certain niche applications. Decreasing latency and improving overall latency characteristics while increasing throughput directly benefits the transaction rates that can be achieved with real world applications. As an example of the improved performance is the latest Reuter Market Data Systems (RMDS) benchmarks done by STACResearch on the 4-way Intel® Xeon E7450 (Dunnington) using the Intel® 82598EB 10 Gigabit AT Dual Port networking adapter. The testing showed the Highest Point-to-Point Server throughput to date on a single server in testing done by STAC. And total updates per second reached over 15 million. Financial Service industry administrators: I can see you drooling...
Latency and throughput numbers are great to talk about, but at the end of the day, real world application performance on real systems is the key. While there will always be a small subset of the high end server market that needs the absolute lowest latencies provided by Infiniband; 10 Gigabit Ethernet is gaining ground while maintaining its place as the default fabric of choice for multiple applications and traffic types. I believe the best is yet to come as newer, faster, and more responsive technologies continue to roll out.
Ben Hacker
