It’s not just about energy-sipping systems—it’s also about your storage footprint

Most of us are familiar with the concept of green IT: increasing energy efficiency across the enterprise to trim costs and optimize resources. While you hear a lot about servers helping to reduce energy usage, not as much is said about storage. Intel and the storage industry are working together to provide green storage solutions, too.

For the storage community, every system has to be cost-effective as well as performance-driven, which means energy efficiency is a key consideration. It starts at the processor level, where the Intel® Xeon® processor 5500 series is extending the boundaries of energy efficient performance.

Many storage system providers have picked up on the Intel Xeon processor 5500 series since it was introduced last March. For example, the HP StorageWorks XP10000* Disk Array and 3000 Enterprise* Virtual Array are based on the new processors. Schooner Information Technology appliances leverage quad-core Intel Xeon 5500 processors and half a terabyte of Intel® X25-E flash memory. The bottom line for the Schooner appliances is an 80 percent decrease in power and cooling requirements versus ordinary servers.

But green storage isn’t just about power consumption at the processor or system level. An equally important green strategy is to reduce the overall storage footprint, and a number of technologies are available to help IT organizations implement this strategy.

Virtualization is driving huge data center energy savings by greatly reducing the number of physical machines in the data center. As Bob Fine, director of product marketing at Compellent, pointed out at the 2009 Storage Networking World conference last spring, many large enterprises realize that they’re approaching a cap. “They can only get a certain amount of power in their data centers and see virtualization as a way to reduce their power requirements,” says Fine. “Instead of building new data centers, they can stay in the ones they have, saving millions of dollars in the process.”

Many IT managers tell Intel that storage can be a big gating factor when it comes to scaling virtual environments. The Intel Xeon processor 5500 series uses Intel® HT Technology within each processor core, doubling the number of threads that can be processed at the same time. This option permits more efficient workloads and enables storage servers to virtualize more applica­tions. Intel HT Technology is also more energy efficient than traditional threaded processing.

Compellent and Hitachi Data Systems (HDS), both users of the Intel Xeon processors, recommend reducing the storage footprint in other ways as well. “Limit the amount of content you need to store by using technologies like data deduplication,” advises Asim Zaheer, vice president of product and competitive marketing at HDS. “Also, don’t have wasted capacity or wasted systems—that’s where tiered storage and virtualization come into play.” 

Compellent’s Fine sees tiered storage as especially important when using expensive disk resources like solid-state drives (SSD). By limiting SSD to the top tier, a company could save on drive costs and increase storage efficiency. “Only the active data would sit on SSD, and all the inactive data would go onto a tier-three SATA drive,” says Fine. “Since SSD drives are about 10 times the cost of Fibre Channel, it’s very important to gain those kinds of efficiencies.”

Isilon Systems, another user of Intel processors, has a pay-as-you-grow model for its clustered storage products that makes it easier to avoid over-provisioning and wasting power. If a customer needs to add more performance, Isilon can provide nodes with Intel processors and memory, but no storage. If the customer requires capacity only, Isilon sells nodes with just disks. In addition, Isilon uses ColdWatt power supplies, which it says are about 30 percent more efficient than traditional power supplies.

As Intel works with the storage industry to deliver more energy-efficient and high-performance storage solutions, we’d like to know what IT organizations are doing to implement green storage technologies in the data center. If you work in IT and have fresh perspectives to make your organization more efficient, you’re invited to share your ideas  here.   

Prior to the Intel Xeon X5500 Server Platforms*, measuring server power was done via expensive equipment and could only be performed in a discrete fashion.  Unless you had tons of monitoring equipment to mash-up your power data - it was a tedious process.  Now, using Intel DCM and Node Manager - you can pull multiple servers worth of power info to make some important power decisions in your datacenter.

First of all, you need to baseline your workload.  If you're confident that you can replicate workload patterns then you've got a starting point.  Otherwise, it's usually a good idea to start monitoring and looking for some cyclical patterns and/or common data points (time, power, thermals, etc) to keep track of.

In this scenario (like in my last blog) we're using a SQL workload which can be modified to run the CPU at high levels for a relatively set amount of time.  The base workload runs for 7 min 30 seconds, as shown in the Intel DCM screencap below.

In this test case: Idle power for the 4 servers is 782W, and under load - the power increases to 1174W - which is a delta of 392W.  This power increase occurs when work is given to the server and the P/T states react to the workload and increase power/voltage to the system to increase performance.  Exactly what we've been used to seeing even since EIST was introduced several years ago.

Now, what I'll show you is something that may be very interesting in scale... I will power cap the servers by 20W each, and set the Intel DCM Power Policy to only allow 1095W for the 4 servers in the rack.

What is awesome here is that we can still finish the workload in the same 7 minutes 30 seconds.  So essentially, we have saved 80W of power for each set of 4 servers and still get the same amount of work completed!  In a large datacenter this can be HUGE in energy savings.

Let's do some quick math:  20W power savings per serer x 10,000 servers = 20kW power savings and you still get the work done.  I hope I just helped some of you server admins get some new ideas on your next "I need a raise" talk with your manager

*your mileage may vary, so test your own workloads and report out!

Every day, Intel® technology and platforms help companies solve business problems and challenges. Here are a few of the growing number of stories and reasons for choosing Intel processors and technology.

Winning: Humana – Healthcare product and services company

Humana continues to refresh its infrastructure with more powerful, energy-efficient technologies. For Humana, technology is vital for providing information and a full array of health benefit services to members. To replace an outdated facility, the company worked with Intel to design a state-of-the-art data center with a compact, energy-efficient infrastructure that could deliver flexibility and scalability.

Read about it here

The results:

·          The Intel processor–based virtualized environment helps IT deploy new services quickly and ensure high availability.

·          Humana added 25 percent more servers in 56 percent of the previous space while decreasing data center power consumption by 16 percent.

Winning: Emerson Electronics

Emerson reshapes its IT infrastructure for future growth, consolidating approximately 135 data centers down to four using Intel® technology–based servers

Read about it here

The results:

·          3,600 physical servers are eliminated by virtualizing on Intel processor–based blade servers, for 18:1 consolidation worldwide

·          Power-saving processors help make Emerson’s new global production data center in St. Louis 31 percent more energy efficient than traditional data centers

Winning: Türkiye Finans Katılım Bank

Leading Turkish Financial Institution Drives Better Growth and Services with Intel®Technology. Türkiye Finans Katılım Bank makes use of the online Intel Xeon processor-based Server Refresh Savings Estimator

Read about it here.

The results:

Intel® Xeon® processor-based Server Refresh Savings Estimator¹ sets expectations clearly, predicting 80 per cent reduction in power/cooling requirements, and a 30 per cent increase in system performance already realized. With only 20 per cent of capacity currently utilised, bank has significant headroom for business expansion

Winning: Oracle IT

Oracle uses Intel® Xeon® processor 5500 series–based systems with Intel® Intelligent Power Node Manager to increase rack density and propel business growth. Refreshing its existing dual socket, quad-core servers on a three- to five-year schedule to increase processing capability and energy efficiency, but had no significant power management in use in the data centers.

Read about it here

The results:

More processing capability can fit within the data center power envelope because Oracle can actively manage power consumption for individual servers and applications.

·          Energy savings of 35 percent are projected with Intel Intelligent Power Node Manager, for reduced operating expenses

·          50 percent more servers per rack saves data center space and enables more growth while keeping costs low

Winning: DataPipe®

DataPipe® retains a competitive edge by designing a new facility and refreshing existing data centers with cutting-edge technology that can deliver outstanding processing performance for a broad range of customer applications. Low-voltage Intel® Xeon® processors help DataPipe create a dense, energy-efficient infrastructure for managed IT services.

Read about it here

The results:

                 New Intel Xeon Processors Provide a Foundation for Cloud Computing. With the Intel Xeon processor 5500 series, DataPipe is creating

                a robust virtualized server environment, Stratosphere™, for hosting customer applications.     

These are dog years for servers.   Pretty much every year Intel introduces a new Xeon processor.  Those who have heard the story recognize this as the Tic Tock model.  On Tic years the manufacturing process is updated, on Tock years the chip architecture is updated.  Every year customers get a boost in performance, and often a cut in power.  Typically this boost is in the 50% neighborhood, enough to make it worth the upgrade, and still achievable by engineering teams on a two year cycle.  Except, we are in dog years.

The Nehalem – Xeon 5500 – processor broke all prior boundaries on single generation performance gain.  Delivering two to three times the compute capacity of the Xeon 5400 (Harpertown) generation.  This is a big change, probably a once in a lifetime change – unless that quantum thing happens in my lifetime.  Roughly a 10X performance boost in less than 5 years.

During this same five years we have seen virtualization technology go from a lab project – something for test and dev – to mainstream data center process.  In 2005 it would have been heresy to suggest virtualizing the corporate ERP.  At that point virtualization overhead on the server could be as high as 25% and the entire server was needed to do “real work”.  Fast forward to today.  Virtualization technology in both the hypervisor and processor have reduced overhead to only a few percent, AND servers are 10X faster.  Not only can you virtualize the ERP, you are irresponsibly wasting resources if you do not.  Unless your ERP demands have grown 10X in 5 years, your ERP alone won’t even make a new Xeon 5500 system sweat.

If this advancement wasn’t enough, the announcements last month from Intel about the coming Xeon 7500 (4+ socket) processor were amazing.  All the benefits of the Xeon 5500, but on steroids.  The  new biggest leap ever.  With up to eight cores and four memory channels per socket, this is a monster.  Your ERP system will be barely a blip in perfmon.  It isn’t unreasonable that an entire data center for a SMB business could be virtualized onto one of these beasts.  And, how big is a Xeon 7500 server?  My guess is about the size of a breadbox…

I’m a bit late in relaying my thoughts from Intel’s Developer Forum (IDF), but there was definitely some excitement around virtualization and high performance networking that I wanted to get the word out about!

In the past I’ve shared some details about SR-IOV and the advantages you can gain by being able to present virtual LAN hardware to each Virtual Machine (VM), effectively avoiding the Hypervisor when presenting virtual devices to each VM.  The advantage of being able to do this is clear:  The less interaction in the networking stack there is from the hypervisor, the less processing overhead is required for the system process the data.

That’s all good because if you have a dual 10 Gigabit adapter, you can segregate those two physical pipes into perhaps 16 virtual pipes that get exposed to 16 VMs.  By segregating these LAN pipes at the hardware level with SR-IOV instead of using Hypervisor switching, the performance gains in both CPU utilization as well as maximum total throughput can be very large.  There were several demos at IDF with various configurations, but reductions in CPU utilization of 40% were possible coupled with dramatic improvement in throughput!

But there is unfortunately one minor complication that I didn’t mention in my last post on the topic of SR-IOV.  There is the little fact that when VMs move between physical boxes (a usage that is highly desired and commonplace these days) you run into some problems with this SR-IOV capability.  When the hypervisor owned the network hardware abstraction, the performance was worse, but the functionality was better because you could seamlessly migrate from one box to another and the virtualization application would handle the details.  But with SR-IOV, a new layer needs to be added so that the direct hardware connection between the VM and the LAN hardware can be moved to a new box.

The really exciting part of IDF demos that I saw was the demonstration not just of the SR-IOV functionality on multiple hardware and virtualization configurations, but that these demonstrations also showed updated software from two virtualization vendors allowing mobility of the VMs while supporting SR-IOV! 

There was a demo on Dell systems showing this fully functional SR-IOV implementation with Citrix’s Virtualization suite.  There were two separate demonstrations on Dell systems, with VMWare displaying their new Network Plug-In Architecture (NPIA) solution that allows for the migration of SR-IOV connected VMs seamlessly between servers.

For those hungry for more detail, I’ve included the three SR-IOV demonstration videos here:

The first is the Citrix demonstration on Dell and Intel hardware of SR-IOV with VM mobility:

These next two are two videos are demos on Dell and Intel hardware with VMWare and their NPIA software implementation.

Each virtualization demo shows the massive performance benefits under various workloads when moving from Hypervisor based LAN segregation to SR-IOV implementation.  But most importantly, each demonstration proves out the capability to migrate VMs between physical hardware.  The only system hardware requirement is that the server itself supports VT-d.  If the networking hardware in the newly migrated-to box supports SR-IOV you get better performance, and if not, the solution falls back on the legacy Hypervisor virtualization.  Backwards compatibility is maintained!

I didn’t get firm details on when this full support for SR-IOV and migration will be available in Citrix and VMWare’s releases, but the demos looked pretty clean, and hopefully these suites will be available soon with this new functionality.  The LAN and Server hardware ecosystems are ready today, and it looks like the software vendors are just around the corner.  Virtualization momentum continues!

While virtualization was the big takeaway for me from IDF, there were also several other interesting demos for us networking hounds.  I’ve linked a couple videos of them below for anyone still thirsting for more of the latest networking technology and performance details!

The first video is a demonstration of Intel’s 82599 10 Gigabit Ethernet-based adapter card with Fiber Channel over Ethernet (FCoE) support.  Storage and Ethernet together at last!

The second video is a demonstration of Intel’s NetEffect 10 Gigabit Ethernet card publishing 1 million messages per second in a simulated NYSE floor trading scenario.  Oh yeah, only 35uS of latency.  That is fast.

So although I am two weeks after IDF, I hope some of you got a little taste of the networking excitement that took place.   Industry wide, hardware and software vendors alike are delivering ultra high performance low latency applications for the financial services industry, as well as mainstream performance increases for virtualization.  The performance and technology beat moves forward.  Exciting times!

--

Ben Hacker

I wanted to follow-up on the pre-IDF blog I wrote and what I and Sean conveyed regarding comprehensive IO optimization for enterprise cloud (based on virtualization infrastructure). A blog I owed to those who could not attend the IDF session.

In the last blog we identified 4 important vectors that drive I/O evolution.

1)      Balanced system that maps to the increases in CPU performance

2)      Scalability

3)      Unified fabric

4)      Security

In my view I state it an evolution as I feel that is the natural state things will head towards in the (near) future.

In a cloud environment you would anticipate automation and policies determine the consolidation possible on a system. If SSDs get broader adoption and virtualization performance increases due to hardware assists, I/O and fabric could become the bottleneck for the degree of consolidation and efficiency as it cannot map to the increased data rates from the storage and CPU performance.

Ways to address this is either to reduce or eliminate the overheads in the I/O stack caused by software emulation of devices in the VMM. VMDq is an example of a technology that can reduce the overhead or offload some of the VMM tasks through hardware assists in the NIC. Direct assignment with PCI-SIG SR-IOV support is a way to eliminate the overheads by by-passing the VMM. With SR-IOV, a single device can be divided into many logical devices known as Virtual Functions (like a pair or independent transmit receive queue). Each virtual function can be directly assigned to a VM using Intel VT-d thereby bypassing the VMM. This can work with Live VM Migration too. At IDF we showcased 4 demos of prototype SR-IOV software solutions running on Intel Xeon 5500 based hardware with prominent VMM vendors like VMware, Citrix and Redhat that have different hypervisor technology. The networking demos showcased working live migration with SR-IOV and VT-d based direct assignment. Direct assigned VMs could be even moved to an emulated mode and brought back to direct assigned mode. Intel has not only been working with software providers but also with other hardware vendors like LSI to demonstrate this capability. These technologies are as important to storage as networking particularly as SSDs gather steam. You can learn more from these blogs below on the demos.

Demo with Intel Xeon 5500 based Dell servers

An analyst view on the LSI solution demonstrated

If those multi 1GbE cables (that make your fabric look like pasta) be replaced by 10GbE and if SR-IOV and VT-d be used for performance, then it answers both the I/O performance and scalability requirements for a flexible datacenter.

Beyond those VT-d provides greater protection by allowing I/O devices to access only the memory regions allocated to them, and SR-IOV allows VMs to access only their portion of the device and restricts access to other Virtual Functions (owned by other VMs) on the I/O device or the entire device itself. Better security through better isolation.

Last but not the least of the requirements is the unified fabric. When IT can use a single I/O device for storage or for LAN traffic, the rigidity associated with provisioning of servers with some number of HBAs and some number of NICs is reduced. The I/O capacity becomes fungible and flexible. FCoE and iSCSI are key technologies in this direction. Adding capability to monitor QoS and shape the traffic makes it a good match for flexible datacenter.

Many of the technologies I discussed above (VT-d, SRIOV, FCoE, iSCSI) are here today… software such as Red Hat Enterprise Linux is already delivering the solution. In my view just a matter of time that ecosystem builds further and hardware is well tuned.

With these in perspective how do you see your datacenter shaping up?

In my previous post Fall IDF: Is Italian Pasta the Actual Inspiration for Server Virtualization? I talked about the evolution of Server I/O virtualization. I mentioned a few demos and invited you to check them out. But I didn’t give any details about the demos…

Well, it's the 3rd and last day of IDF today and the demos have running now for 2 days. You have one moreday to check them out! So let me describe them quickly.

Dell has been a great partner for these demos. We’re showing 2 demos together in booths 709 and 711 in the Virtualization Community, using Dell’s R710 servers, based on the Xeon 5500 platform, and using Intel 82599 (Niantic) with Virtual Machine Direct Connect (VMDc). The 1^st demo is with VMware and their Network Plug-In Architecture (NPA) technology.

When you visit the demo, check out the great CPU utilization as well VMotion* among heterogeneous server configurations!

The 2^nd demo is delivered with Citrix, showing scalable direct assignment by using XenServer with VT-d and SR-IOV support. The overall performance is really great and live relocation of virtual machines is working nicely.

Another demo (booth 707) is delivered by Red Hat, featuring RHEL 5.4 with KVM (shipping SW with VT-d and SR-IOV support!), Neterion with their 10GbE NIC, all running on an Intel Xeon 5500 server. Look out for the performance value shown for scalable direct assignment!

And finally, a storage (not LAN) demo! Using the same combination of VMM and server, in booth 517, LSI is showing the value of scalable direct assignment for a RAID controller. The performance boost is fantastic!

Check out these demos at the IDF Showcase... I’d love to hear your impressions!