I wanted to follow-up on the pre-IDF blog I wrote and what I and Sean conveyed regarding comprehensive IO optimization for enterprise cloud (based on virtualization infrastructure). A blog I owed to those who could not attend the IDF session.

In the last blog we identified 4 important vectors that drive I/O evolution.

1)      Balanced system that maps to the increases in CPU performance

2)      Scalability

3)      Unified fabric

4)      Security

In my view I state it an evolution as I feel that is the natural state things will head towards in the (near) future.

In a cloud environment you would anticipate automation and policies determine the consolidation possible on a system. If SSDs get broader adoption and virtualization performance increases due to hardware assists, I/O and fabric could become the bottleneck for the degree of consolidation and efficiency as it cannot map to the increased data rates from the storage and CPU performance.

Ways to address this is either to reduce or eliminate the overheads in the I/O stack caused by software emulation of devices in the VMM. VMDq is an example of a technology that can reduce the overhead or offload some of the VMM tasks through hardware assists in the NIC. Direct assignment with PCI-SIG SR-IOV support is a way to eliminate the overheads by by-passing the VMM. With SR-IOV, a single device can be divided into many logical devices known as Virtual Functions (like a pair or independent transmit receive queue). Each virtual function can be directly assigned to a VM using Intel VT-d thereby bypassing the VMM. This can work with Live VM Migration too. At IDF we showcased 4 demos of prototype SR-IOV software solutions running on Intel Xeon 5500 based hardware with prominent VMM vendors like VMware, Citrix and Redhat that have different hypervisor technology. The networking demos showcased working live migration with SR-IOV and VT-d based direct assignment. Direct assigned VMs could be even moved to an emulated mode and brought back to direct assigned mode. Intel has not only been working with software providers but also with other hardware vendors like LSI to demonstrate this capability. These technologies are as important to storage as networking particularly as SSDs gather steam. You can learn more from these blogs below on the demos.

Demo with Intel Xeon 5500 based Dell servers

An analyst view on the LSI solution demonstrated

If those multi 1GbE cables (that make your fabric look like pasta) be replaced by 10GbE and if SR-IOV and VT-d be used for performance, then it answers both the I/O performance and scalability requirements for a flexible datacenter.

Beyond those VT-d provides greater protection by allowing I/O devices to access only the memory regions allocated to them, and SR-IOV allows VMs to access only their portion of the device and restricts access to other Virtual Functions (owned by other VMs) on the I/O device or the entire device itself. Better security through better isolation.

Last but not the least of the requirements is the unified fabric. When IT can use a single I/O device for storage or for LAN traffic, the rigidity associated with provisioning of servers with some number of HBAs and some number of NICs is reduced. The I/O capacity becomes fungible and flexible. FCoE and iSCSI are key technologies in this direction. Adding capability to monitor QoS and shape the traffic makes it a good match for flexible datacenter.

Many of the technologies I discussed above (VT-d, SRIOV, FCoE, iSCSI) are here today… software such as Red Hat Enterprise Linux is already delivering the solution. In my view just a matter of time that ecosystem builds further and hardware is well tuned.

With these in perspective how do you see your datacenter shaping up?

…on my way to a customer meeting, and the thought dawns on me about why the car I’m getting into is a relatively new, clean 2008 compact car and not a 1966 Chevy Impala, which probably has enough steel to dramatically distort the earth’s local magnetic field.  Well the reasons are fairly simple:

• Newer cars are more reliable and require less maintenance - cars in the shop don’t make the rental car agency money, and don’t make customers happy if they break down
• Newer cars are typically more fuel efficient - that ’66 Impala’s gas mileage might be quoted in gallons per mile J
• Newer cars typically fall under a manufacturer warranty

As with rental cars, servers aren’t much different.  It’s all about keeping your business running smoothly, minimizing your operating costs, and keeping your customers happy.  While I’m guessing not many of today’s data centers have the server equivalent of a ’66 Impala in them, there are probably a bunch ready to be removed from the rental car fleet.

Think about it on your next business trip, and check out the benefits of refreshing servers that are only 3 or 4 years old with the Xeon® ROI estimator tool (link:  www.intel.com/go/xeonestimator).

First there was the Multi-Billion Dollar Automobile “Cash for Clunkers” program that I wrote about back in early August. Then in late August we started reading more about the planned $300M state-run rebate programs for consumer purchases of new ENERGY STAR® qualified home appliances. Appliance categories eligible for rebates include: central air conditioners, heat pumps (air source and geothermal), boilers, furnaces (oil and gas), room air conditioners, clothes washers, dishwashers, freezers, refrigerators, and water heaters.

The government wants to make cars and homes more energy efficient, while helping to support the nation’s economic recovery.  But what about making Data Centers more efficient?

A couple of years ago the US Environmental Protection Agency reported that the energy consumption associated with data centers had doubled between 2000 and 2006, reaching some 60 billion kWh in 2006, roughly 1.5% of the entire US energy use. The EPA says this is expected to double again by 2010.  The same authors of that report previously calculated that US servers currently use the same level of electricity as all color TVs in the country combined. 

So this got me thinking…which industries have done the most to increase output per energy unit and which products also offer the most attractive paybacks when you invest in them.  The findings were interesting to say the least.  Let’s first look at the sectors creating more energy-efficient products over the last 30 years*.

• Autos – 1978 (14.3 MPG), 2008 (20 MPG): Energy Efficiency gains = 40%
• Airlines – 1978 (22.8 Revenue passenger MPG), 2008 (50.4): Energy Efficiency gains = 121%
• Agriculture – 1978 (0.63 units of output per unit of energy use), 2008 (1.46): Energy Efficiency gains = 132%
• Steel Mfg – 1978 (63 lbs of steel per MBtu), 2008 (167 lbs): Energy Efficiency gains = 167%
• Lighting – 1978 (Incandescent light bulb – 13 lumens per watt), 2008 (Compact Fluorescent Bulb – 57 lumens per watt): Energy Efficiency gains = 339%
• Computer Systems – 1978 (1,400 instructions per second per watt), 2008 (40,000,000 instructions per second per watt): Energy Efficiency gains = 2,857,000%

*Source:  “A Smarter Shade of Green,” ACEEE Report for the Technology CEO Council, 2008.

Next let’s look at some big ticket energy efficient products that offer the most attractive paybacks on their investments. (Note: Buying a hybrid automobile wouldn’t make this list below in terms of rapid payback, hence not included.)

• Central Air Conditioning: 4.7 year payback per Energy Star Savings Calculator (I used Los Angeles as Location)
• Freezer: 4.3 year payback per Energy Star Savings Calculator
• Refrigerator: 2.8 year payback per Energy Star Savings Calculator
• Commercial Ice Machine: 2.1 year payback per Energy Star Savings Calculator
• New Intel Xeon 5500 based Server: 9 months payback per the Xeon ROI estimator at www.intel.com/go/xeonestimator. (I modeled refreshing from older 2005 Single-Core servers, getting 10:1 server consolidation ratio.)

IT industry far exceeds others at increasing output per energy unit… and Intel servers also offer a faster payback on investment than other energy efficient products (including Energy Star Products).  Yet there is not government stimulus package to help encourage these purchases in energy efficiency. Simply, this is the most energy efficient investment that the government won’t help you make.

I would be curious to hear what you think.

bryce

Given the increasing costs to power and cool data centers, IT and Facility managers of enterprise and cloud data centers want to maximize capacity utilization and reduce total cost of operations. However this has been challenging given the lack of fine grained instrumentation and visibility into server and rack level power consumption resulting in over-provisioning of racks and costly expansions despite unused capacity.

With Intel® Xeon® 5500 series, we introduced Intel® Intelligent Power Node Manager that provides server level power monitoring and policy based power capping. Since power and cooling constraints exist at rack, row, room and PDU level, we also released Intel® Data Center Manager to enable ISVs to realize Node Manager benefits at data center level with reduced investment.

Intel® DCM software development kit (SDK) provides power and thermal monitoring and management for servers, racks and groups of servers in data centers using Intel® Xeon instrumentation. Management Console Vendors (ISVs) and System Integrators (SIs) can integrate Intel® DCM into their console or command-line applications and provide high value power management features to IT organizations.

IT organizations facing power and cooling challenges can benefit from:

• Increased rack density within space, power and cooling constraints through power control
• Reduced capital costs by right-sizing power and cooling infrastructure based on actual power
• Reduced operations costs by eliminating worst-case head-room during provisioning

Intel® DCM features include:

• Built-in policy based heuristics engine maintains group power and dynamically responds to changing loads
• Designed as an SDK to integrate into existing management software products
• Scales to thousands of nodes support in large data centers
• Manages across system vendors through use of standard IPMI and DCMI Power Management support
• Supports integration with Smart PDU meters for servers without Node Manager to provide a unified view

If you are at IDF in San Francisco Sep 22-24, 2009, stop by following sessions to learn more about Intel® DCM and see proof-of-concepts and meet with end users who benefited from this.

• PDCS003 - Cloud Power Management with Intel® Micro-architecture (Nehalem) Processor-based Platforms
• ECTS0004 - Improving Data Center Efficiency with Intel® Xeon® Processor Based Instrumentation

There are also several demos in the showcase area that showcase the technologies from multiple partners.

I will be at IDF co-presenting PDCS003 - stop by after the class with your questions and thoughts about Intel® DCM.
See you at IDF.

--Susmita

Intel's RK Hiremane & Sun's David Caplan discuss Xeon 5500 blade servers virtualization ROI

Join experts from Intel, Sun Microsystems, and Ziff Davis Enterprise on August 20 for an informative eSeminar, where you will learn:

• How Sun’s Network Express Module technology works
• How easy it is to achieve high availability and near-instant failover
• How to reduce network cabling by a factor of 10:1
• How to simplify network and storage management.

Intel Intelligent Power Node Manager is a new technology that is available with the Xeon 5500 Series Platforms released earlier this year.  Many of you have asked me questions via Twitter (@Toadster) about "How can I use Node Manager?" - so I wanted to present some simple use cases to simplify the explanation of Node Manager and how you can best use the technology in your own enterprise.

First of all, let's explain the growth problem at hand.  As servers shrink in size, the density of each server 'footprint' is growing from a power perspective... a few years ago, a single 42U rack could hold about 21 servers (estimating 2U servers) - and usually hosting one or two apps/servers per physical server, depending on if you had single or dual-socket servers.  In modern datacenters, that same 42U rack can hold 42 servers (1U each) with 2P per server - so you have an immediate density increase of 2X the # of servers, and 2-4X the number of sockets - which can equate to 16X the number  processor threads per rack...  one good thing is that Intel has been developing newer technologies to keep the TDP of each CPU roughly the same over the same time period between processor updates... where you used to have 2 or 4 cores, you now have 8 to 16 cores at the same thermal envelope!

Knowing how much power your platform uses is a key factor in populating racks and rows in your datacenter.  Prior to Node Manager technology, most Datacenter Managers would base rack population on 'nameplate' power - or the (W) rating on your power supply.  That's the 'max' power utilized by the platform, and what the PSU is rated for (worst case).  See the image below...

As you can see - using Intel Intelligent Power Node Manager technology, you can view your system's power utilization in real-time using Intel Datacenter Manager and the administrator can implement the power caps to ensure your server rack stays within your required power limits.  By utilizing the 'actual' power limits instead of nameplate power, you can increase your rack density thereby increasing your ROI, and decrease your TCO!  Lets face it - everyone loves saving money!

Many of us are familiar with this next scenario... it's summertime, and the power company is announcing that the power grid is under strain.  Personal homes start having their A/C cut-off to save the power grid from brown-outs...  now your enterprise can help reduce those risks as well!

Over the next few weeks, I hope to post more blogs/videos:

1. Single Node Power Monitoring & Management
2. Group/Rack Power Monitoring & Management
3. Thermal Monitoring & Management

Please provide some feedback, and post your questions and ideas for upcoming blogs!

We created a server refresh ROI estimator tool to help IT managers make sense of the significant OpEx savings they can achieve by making targeted investments in new server hardware. In my previous blog when we introduced the ROI tool back in April 2009, I talked about the capabilities of the estimator and the benefits of server refresh.  In the first 3 months, we have had nearly 4,000 users of the ROI estimator and of those users almost 800 users have printed reports to share with others in their organizations. The feedback we have received from users has been very encouraging. 

• CIO for major US hospital: “This would help my IT staff justify the financial value of the technology investment they are proposing. This has been a barrier to freeing up capital internally”
• IT Manager for major US bank: “I used to have regular funding for technology refresh projects. It was a given for my budget.  However, with the increased constraints on capital, I now have to justify this type of spending”
• Technology Sales Consultant: “This tool helped me work better with my customer to gain a deeper understanding of their server environment and allowed us to jointly identify high ROI investments to improve their infrastructure”

I have also heard many constructive suggestions for improvement.  As a result, we have continued to evolve the tool based on feedback from users.

Tool Training – How to Use: We heard that the benefits of using the Savings Refresh Estimator spanned many functional roles, making us realize that the use models for this type of tool and what users were looking for would vary dramatically from person to person.  This has challenged us to look at ways to streamline the user interface (something we continue to work on) for different users and analyses.  In the interim, we are in the process of developing a video training guide to help users understand how to use the tool to get maximum benefit.  We have a pdf training guide today that can help you get started now.

PowerPoint Output: What would we do without powerpiont? J We received feedback on the desire to make the output of this tool more sharable inside IT organizations and with business partners in a powerpoint format as a way to communicate the opportunity and benefits for server refresh investment.  So, we now have a powerpoint output option in the reports section that breaks down the benefits of server refresh for a variety of audiences from executive staff to facilities to finance.  Everyone inside your business can benefit from server refresh and now you can show them how.

Secure Analysis: We received feedback that many users wanted access off-line either as a way to use in meetings when connectivity was challenged or to protect internal data from exposure online.  We now have the ability for you to run the tool on your laptop to support these use models.

More … More … More Functionality. We heard lots of requests and ideas to expand the level of functionality and analysis capabilities.  We have to balance scope, complexity Keep these requests coming.  The following changes are incorporated into today’s estimator.

• Virtualization to Virtualization Refresh Scenario – now included
• Virtualization Loading: Can edit and change VM/server new and old
• Custom Performance Data – enter you own performance data to better model what you expect to see in your biz
• Depreciation Cycle – no longer fixed at 4yrs .. can adjust
• Memory Sizing: information added to allow user analysis
• Processor Description: allows user to cross reference data to other more familiar terminology.

Accuracy / Approach: We have also heard some feedback challenging us on different ways to look at refresh scenarios, especially as we learn more about how people are looking at and using virtualization and sizing their environments after refresh.  Sizing is a very customer-centric and application specific task that is difficult to model in a one-sized fits all.  We won’t be able to model every sizing situation, but are planning some future enhancements intended to help you self-evaluate. 

I want to thank everyone in the community for their input on this tool and helping us to deliver a better product over time.  Keep the ideas coming.  Feel free to respond with comments here.

Chris

twitter: @chris_p_intel

I recently had the opportunity to sit down with Intel's Chief Virtualization Architect Rich Uhlig to discuss the new usage models and virtualization technologies in Intel new Xeon 5500 series platform. Rich and I have been friends and colleagues for several years and the video of our discussion is attached and can be viewed on Youtube. The conversation sparked some interesting questions from my colleagues, friends and children which I thought I would share with a wider audience.

First the questions from my son's (I have three boys...yes this means that my wife has the patience of a saint):

Dad, what is virtualization? Does that mean you can take people and computers and teleport them to new places, like Star Trek? Did Intel invent virtualization? Why do you think it is so cool? When I grow up, can I be virtualized?

My Answer:

Slow down.....slow down...let me try to answer the questions one at a time.

Virtualization is the ability to increase computer, network and storage utilization with multiple operating systems or logical machines, called virtual machines. This allows Dad and his friends to use more of their computers with different applications and devices. Using virtualization allows Dad and his friends to save money, save power and increase efficiency.

Response (My three son's in unison):

Boring! I thought you said your job is cool. Your such a geek......(trailing off and looking at their iPods)

My response:

Guys, hold on...let me explain. Virtualization technology IS cool. While it wasn't invented by Intel, we have worked with an industry of incredibly gifted engineers, architects and designers to create new ways for people to use their computer technology....and the best part is we are only in the beginning. By the time you are an adult you will have the opportunity to use virtualization technology in ways we are only beginning to imagine. Think of virtualization as a journey and evolution of computer technology for Dad and his friends to maximize the use of the computers that we buy/build. Hopefully, with more innovation and computer technology advances you will be able to create a virtualization layer that will allow you and your digital identity to "teleport" to new places in a virtual cloud. You won't be "virtualized" but you will be able to create your digital environment wherever there is a machine that can understand your commands. That is pretty cool. Think of it this way, you can save and play your Nintendo Wii, Sony Playstation or XBox profiles on any machine, any where in the world that can download your profile.

Response (from my 13 year old):

You mean I can play EA's Madden Football 24 hours a day with my friends, even when we are on vacation and you want me to see some historic landmark, like the Lincoln Memorial?

My response:

Well...yes but not exactly what I had in mind. (aargh!)

A recent question from my friend from a former job on Virtualization:

I hear the new Intel chip, Nehalem (formerly known as the Intel Xeon 5500 series), is the best product you guys have released in a long time, What makes the product so good, is it the virtualization technology that you work on?

My response:

     Virtualization technology provides increased instrumentation and flexibility for the Intel Xeon 5500 series platform but it is only one a host of fantastic features which make this product the best we have ever released. For Data Center managers, increased efficiency is an every day part of life. Nehalem offers increased performance, increase memory capacity, a new Quick Path Interconnect (which acts like a NUMA switch fabric on silicon, remember that cool product we launched in 1997 at Sequent Computers?) and a 2nd generation of virtualization capabilities that deliver native virtualization instruction capabilities for VMWare, Microsoft, Citrix and a host of Xen providers. It is a truly a breakthrough server product. With this new architecture and design characteristics we are able to meet the needs of a platform of new Virtualization usage models including: Rapid Application Deployment, High Availability, Virtual Desktop Infrastructures and Server Consolidation. It is a very exciting time...

My friend's response:

     Very cool. I miss working on hardware innovations...sounds like you guys at Intel are up to something special. Should I buy the stock?

My response:

     Thanks. Intel is a great place to work and we are doing some very cool product innovations. Do we always have to talk about stock price?

Finally, a recent question from a dear colleague:

     What happens if virtualization technology is deployed on every platform that Intel ships? Won't business and consumers need less devices? Won't users no longer have an insatiable demand for compute, network and storage resources?

My Answer:

    Funny you should ask that question. Rich Uhlig, Fernando Martins, Rick Olha, RK and I have debated this exact question for years. The answer is simple. Virtualization increases demand for more resources than ever before. In fact, until the recent economic downturn virtualization technology was cited by a Citigroup analyst as the key driver to Server growth in 2H 2007. For the first time in over 10 years the markets average selling price was increasing. Why? Because users could do more with every server they purchased. Virtualization actually facilitiates more usages on more application development and production environments than ever before. As we increase the performance of the instruction sets and Intel Microarchitectures we increase the capabilities that virtualization can impact for new usage models, while preserving some the legacy compatibility that users require for 32-bit application workloads. Simply stated, "we can do more with less!"

Next question (by the way this was a skeptical Intel exec.):

     Doing more with less is fine...but what about our volumes for server products? what happens when virtualization is prevalent across all of Intel CPU and Platform offerings?

My response:

    Flexibility and control are critical to all of our customers regardless of form factor. Is there anything worse then buying a new server, PC or handheld and having application compatibility errors? No. Do we really believe the world wants to become software compatibility specialists everytime Microsoft releases a new operating system? What about Dell, HP, Lenovo, IBM, Acer, Nokia, Motorola, LG, Samsung, RIM and HTC? It has taken us over 10 years of research, testing and product development to get here. Virtualization is a "Hot Topic" today and will be in the future because it makes a positive difference in our customers lives both financially and efficiently. Our job is deliver the greatest silicon products the world has ever seen, over and over and over again. Virtualization allows us to do that AND preserve the investments our customers and software partners make in developing their own operating environments. What is cooler than that? Virtualization facilitates innovation, consumption and utilization, our customers are telling us this everyday. Innovation is critical to this process, enabling our software colleagues is a must and opening up the discussion is part of the process.

Her response:

     Well, I guess you are pretty passionate about virtualization?

My response:

     I hope so...that is why you hired me.

Have a listen, enjoy the video and join the discussion of Rich and I. For us, Virtualization is a very Hot topic, that we have thought is Cool for a very long time.