These are dog years for servers.   Pretty much every year Intel introduces a new Xeon processor.  Those who have heard the story recognize this as the Tic Tock model.  On Tic years the manufacturing process is updated, on Tock years the chip architecture is updated.  Every year customers get a boost in performance, and often a cut in power.  Typically this boost is in the 50% neighborhood, enough to make it worth the upgrade, and still achievable by engineering teams on a two year cycle.  Except, we are in dog years.

The Nehalem – Xeon 5500 – processor broke all prior boundaries on single generation performance gain.  Delivering two to three times the compute capacity of the Xeon 5400 (Harpertown) generation.  This is a big change, probably a once in a lifetime change – unless that quantum thing happens in my lifetime.  Roughly a 10X performance boost in less than 5 years.

During this same five years we have seen virtualization technology go from a lab project – something for test and dev – to mainstream data center process.  In 2005 it would have been heresy to suggest virtualizing the corporate ERP.  At that point virtualization overhead on the server could be as high as 25% and the entire server was needed to do “real work”.  Fast forward to today.  Virtualization technology in both the hypervisor and processor have reduced overhead to only a few percent, AND servers are 10X faster.  Not only can you virtualize the ERP, you are irresponsibly wasting resources if you do not.  Unless your ERP demands have grown 10X in 5 years, your ERP alone won’t even make a new Xeon 5500 system sweat.

If this advancement wasn’t enough, the announcements last month from Intel about the coming Xeon 7500 (4+ socket) processor were amazing.  All the benefits of the Xeon 5500, but on steroids.  The  new biggest leap ever.  With up to eight cores and four memory channels per socket, this is a monster.  Your ERP system will be barely a blip in perfmon.  It isn’t unreasonable that an entire data center for a SMB business could be virtualized onto one of these beasts.  And, how big is a Xeon 7500 server?  My guess is about the size of a breadbox…

I wrote a few weeks ago about the end of the mini generation.  This time I thought I would dig out some data to support my case.  My personal anecdotal evidence is what I am hearing from my customers.  They are looking at replacing hundreds of legacy Unix servers with new high performance Xeon boxes.  I am not talking about a one-for-one replacement, but using virtualization to replace 5 to 25 of these older unix boxes with each Xeon 5500 server.  The economic incentives here are pretty staggering.

Why now?

I see multiple reasons

1)  Ecosystem maturity.  Enterprise class tools for virtualization, Linux, high availability from VMware, KVM, Xen, RedHat, Suse and others

2)  Performance.  The performance of 200-2005 vintage sparc and ultra-sparc boxes is easily replaced by Xeon – saving power, space, and potentially licensing.

3)  Applications readiness.  Applications like Oracle are now “made for linux” and do great on X-86 platforms

4)  Staff.  You have the expertise in Linux on Xeon, this is a growing area, capitalize on it.

5)  Economics.  There is real savings to be had in licensing, power, space, staff, sanity( sanity savings is subjective).

I hopped out to tpc.org to look at some benchmarks.  Benchmarks are notoriously awful as measures of actual performance, but they do work – mostly – as a comparison of relative performance.

There isn’t a lot of Sparc data, and much of it is old, but if you are looking at replacing some aging 4+ year old Unix hardware, that may be just what you need.  (with respects to Bryce’s cash for clunkers blog).

For TPC-C the most recent Sparc result I found was from 2003. Running Oracle Database 10g EE on Sun Solaris 8 on 64 single cores of Fujitsu SPARC64 - 1.3 GHz processors, they delivered 595702 tpmC at $12.43/tpmC (tpc.org)

So if “this old machine" is setting in your landscape, gulping power and support costs, you could replace it today by running Oracle Database 11g SE1 on Oracle Linux 2 quad core Intel Xeon Processors X5570 2.93GHz  delivering 631766 tpmC at $1.08/tpmC (tpc.org)

The ROI on this must be about 10 minutes! Ok maybe that is a bit quick, but this is a data base! Export, Import, ta-da!  What are you going to do with all that extra rack space and power?

Replace a 64 socket platform with a 2 socket platform.  Amazing.  this could be 1U, or even a blade.  You could put it under your desk.  There have got to be some examples of older sparc and power boxes sitting in the landscape. Let me know what you have.

-Ken

Sunsets can last a while, but in the end the sun will go down.  I talk to a lot of companies and listen to a lot of data center managers.  Customers trust their AIX-Power and Solaris-Sparc platforms.  These are solid platforms and deliver good features and reliability, but, if these managers could get the sense of security, performance, and reliability with Linux / Intel Xeon platforms, they would move tomorrow.  It is simple economics.

The reality is that customers are making this move, and being successful.  The hardware reliability on Intel platforms today is amazing.  Intel recently announced that their next generation of Xeon(Nehalem) EX based servers will support Machine Check Architecture.  This brings high end Xeon X86 servers into the RAS family previously reserved to proprietary RISC & mainframe platforms.  Intel Xeon already eclipses the performance of proprietary RISC processors both on a per processor basis and a per dollar basis.  It is reasonable to say Xeon can deliver better performance, better value, and equal or better reliability.  The only hurdle left is the software.

Linux has come a long ways.  It is no longer a university OS, run by geeky dudes in black T-shirts emblazoned with the quadratic formula.  It is mainstream and solidly supported.  Linux is the primary development and delivery platform for Oracle.  Other OS environments are ports, delaying support and innovation.  Linux is used by major financial companies.  Linux is available in solid and well supported distributions with a 20 year history of enterprise business.  Linux experts are broad community, worldwide, and growing in number.  Linux is economical vs proprietary RISC.

In an era of big budgets and conservative (don’t make any changes) philosophies, businesses will always stick to their proprietary RISC systems.  That era is over.  Sticking to your RISC systems may seem like the safe move, but failing to examine the opportunities for better performance and lower cost with Linux on Intel Xeon platforms is business negligence.  Business negligence is seldom rewarded. 

Performance, Price, Infrastructure, Ecosystem, TCO, RAS – when the decision factors are examined it becomes clear -  we are in the RISC twilight and the sun will set on Sparc-Solaris and Power-AIX.

"We are virtualizing".  I hear that at every customer, every day.  I am not sure where virtualization is on the hype curve, but i don't think it is anywhere near slowing down.  I am very glad to be past the "Dilbert" and "in flight magazine" era.  Customers seem to have a really solid command of what they want to virtualize and why they want to virtualize. ( not to imply that all the questions have been answered )

The latest Intel servers - Xeon 7400 processor series in the 4 socket family, and the incredible Xeon 5500 (Nehalem) processor series in the 2 socket family - deliver more than sufficient capacity for sweeping data center virtualization.  i.e. very few enterprise applications are to big for a VM on one of these platforms.

I hear three reasons from customers for virtualization. ( in order of emphasis )

1) To improve efficiency.  Most enterprise servers are only about 10% utilized ( and many of these are old, slow, inefficient servers)  Applications are partitioned onto individual servers for archaic historical reasons.  Combining these on powerful modern servers can dramatically reduce footprint, power, server costs and licensing costs.

2) To improve flexibility.  Virtualization allows "servers" ( think VMs ) to be easily moved from one platform to another - for sizing - for maintenance - for almost any reason.  With the Intel Flex Migration technology and recent versions of VMware ESX - customers can pool Intel servers across multiple generations and families.  Live migration from your Xeon 5100 processor based server to a cozy VM on a 4 socket Xeon 7400 based 24 core server.

3) To improve reliability.  Virtualization provides a vehicle for managing hardware failures, allowing near instantaneous fail-over in the event of a server loss.

Virtualization has moved out of the lab and become a "best know method" for doing IT right.

Intel points to three focus areas for servers.  Efficiency, Performance, and Virtualization.  I think virtualizaiton's place in this triad is fleeting.  It only remains  because changes are still being made to the platform to support virtualization.  Soon virtualization will become just another part of the stack- like the operating system.  Nobody claims their processor is optimized for running an operating system...  Even today choosing the best processors for virtualization is more about efficiency and performance than about virtualization features.  Fortunately - as I do work there - Intel has a solid lead on both efficiency and performance.

Last year I did a series of entries on the opportunity to avoid brick and mortar ( or steel and drywall in most cases ) data center expansion.  The math was pretty compelling.  If you have depreciated enterprise servers in production, and at 10% utilization today, and the current "state of the art" Xeon servers will easily give you 5x or more performance gain.  In this scenario, you can get 50x the capacity in the same space and power.  My formula was pretty conservative - 5x performance, 5x utilization with virtualization, and 2x the rack density by measuring actual power and using high efficiency designs.

Intel's Core i-7 variant for Xeon ( code named Nehalem) only enhances this opportunity.  The Xeon 5500 series processor launches in March 2009, and although performance benchmarks are still pretty scarce, the buzz in the industry is that this is the biggest leap in Intel Xeon performance in many years. 

Consider the impact - if this processor doubles current Xeon 5400 performance, that could yield 100x the compute capacity in the same footprint and power.  Before you get out the sledge hammer you really should evaluate server refresh.  An aggressive server refresh with state of the art Intel Xeon based systems can deliver the performance and capacity for business needs, AND avoid capital data center expansion.  In this economy, avoiding real estate expansion should make you a hero at work.

If this is not enough, take a look at optimizing your cooling ratio.  see whitepaper_energy efficiency in the data center.pdf or Reducing Data Center Cost with an Air Economizer.  This could give you another few thousand watts per rack, to increase density and push your multipliers even higher.

Virtualization is the big thing, everybody is doing it - just read the in-flight magazine to see why you should be virtualizing your data center... While it is true that Virtually everyone in the fortune 500 has begun to virtualize their data center, it is also true that most servers are still not virtualized.

i.e. The data center landscape is still mostly an opportunity. The software is mature, there are multiple viable solutions, but there are still many questions about how "best" to proceed.

As an enterprise engineer working with enterprise customers, I am inevitably asked where the sweet spot is. The reality is, there isn't one. Or "It Depends". In general larger ( 4 socket servers) provide an edge in efficiency as there are more shared components - board, memory, power supplies, etc. Large servers can also provide more head room if most of your VMs are low utilization, but any of them can spike way up. The launch of Intel's six core Xeon 7400 series based servers ( and their record breaking virtualization performance) have added to the interest - is it time to go big?

What does it depend on?

• How big are your VMs? Machines today are quite powerful. We have seen a 10X growth in compute capacity in just the last 6 years. The application that filled 37% of your 2003 vintage server won't even make a dent in a modern Xeon based server. i.e. Most VMs are much smaller than your server 2 socket or 4 socket. There are still tasks - like decision support that scale as big as your machine will go, but with average enterprise utilization down around 12% ( on old hardware) most physical machines fit tidily inside a VM.

• How spiky are your VMs ( in resource demand - compute, memory, network)? By doing some resource profiling, you can understand where your servers fit best.

• How many VMs do you want on each PM(physical machine)? You can put more on 4 socket hardware ( efficiency) but have greater redundancy on a bunch of 2 socket hardware ( depth).

Fortunately you do not have to solve this linear programming problem before you start. In reality the tools are making it easier you solve. Using your favorite VMM manager (choosing this is another discussion). With Intel's VT Flex Migration Technology you can pool together 1, 2, 4 socket current and future generation Xeon platforms and move the workloads ( automatically, or manually) to optimize your resource utilization.

At a recent event the presenter, making reference to Pee Wee's Playhouse, said "virtualization is the word of the day". Of course, all of us older-yet not quite mature individuals had to cheer every time someone said the V word. For you yougsters, I am sure an internet search will tell you more than you ever wanted to know about Pee Wee and the word of the day.

Virtualization is everywhere. If you have been avoiding it, i recommend *this* well constructed summary as a background guide to everything you should already know.

From my perspective two major trends are driving the maturity of virtualization. First, on the software side - there are now multiple players. Yes, VMware is the market leader, but there are credible and demonstrable solutions available from Xensource, Microsoft, SWsoft, Virtual Iron, and others. Virtualization software is increasingly differentiated by the management tools and solution breadth, not the ability to virtualize.

The second significant trend is the change in hardware platforms. Both Intel and AMD have incorporated extensive features into their processors to support and simplify virtualization. Intel has extended this integration to their chipsets and network adapters with Intel virtualization technology for devices and Intel virtualization technology for Connectivity.

Virtualization has become the principle tool in the *data center* survival toolbox. No enterprise data center optimization can be effectively executed without the big V. This is sometimes referred to as virtualization 2.0,,, but like the web and many other 2.0 things,,, it is much more of a continuum between simple usage models - consolidate small servers , and advanced usage models - dynamic load balancing.

I met with three enterprise architects in the last week, all were looking at virtualization as the foundation for their dynamic "utility-esque" compute platforms. To quote the chief architect at a major bank - "the most efficient and affordable server I run is a VM on a Xeon platform". Managed virtualization can deliver efficiency, affordability, and flexibility. At this point you are either actively rolling out virtualization or you are not paying attention.

This is part three - the implication being that it is a sequel to part one and part two. It is. That said, each of the sections have their own messages and may or may not help your data center. The first part talked about the benefits of bringing in the latest hardware. Intel has been delivering performance increases at a pace beyond "Moore's Law". Getting rid of old, slow, inefficient servers can give you 2-12 times the capacity instantly. The second "episode" talked about getting everything you can from each server. Use virtualization and consolidation to make sure your servers are full and busy. The most efficient bus is a full bus ( this is a metaphor, I am talking about the big yellow things carrying students, not the circuitry in the box )

My focus in part three is on density. My operating premise is that the data center manager wants to get everything out of the current data center and avoid, or at least defer, construction of a new data center. If your in the data center construction business, this is not for you.

To get the most out of our data center we want to pack every server we can power into the space. You can do this by executing three actions. 1) Use every watt, 2) Build the right servers, and 3) Optimize HVAC. In many cases twice the servers can be crammed into the existing rack space even without adding power. If you are able to redirect your hvac power savings to your racks, your results could be even better.

So, we potentially got 5x capacity from new quad core servers, 5x capacity from boosting utilization with consolidation, and 2x capacity with higher density. My math says 5x * 5x * 2x = 50x the capacity ( in the same space and power!) video