Prior to the Intel Xeon X5500 Server Platforms*, measuring server power was done via expensive equipment and could only be performed in a discrete fashion.  Unless you had tons of monitoring equipment to mash-up your power data - it was a tedious process.  Now, using Intel DCM and Node Manager - you can pull multiple servers worth of power info to make some important power decisions in your datacenter.

First of all, you need to baseline your workload.  If you're confident that you can replicate workload patterns then you've got a starting point.  Otherwise, it's usually a good idea to start monitoring and looking for some cyclical patterns and/or common data points (time, power, thermals, etc) to keep track of.

In this scenario (like in my last blog) we're using a SQL workload which can be modified to run the CPU at high levels for a relatively set amount of time.  The base workload runs for 7 min 30 seconds, as shown in the Intel DCM screencap below.

In this test case: Idle power for the 4 servers is 782W, and under load - the power increases to 1174W - which is a delta of 392W.  This power increase occurs when work is given to the server and the P/T states react to the workload and increase power/voltage to the system to increase performance.  Exactly what we've been used to seeing even since EIST was introduced several years ago.

Now, what I'll show you is something that may be very interesting in scale... I will power cap the servers by 20W each, and set the Intel DCM Power Policy to only allow 1095W for the 4 servers in the rack.

What is awesome here is that we can still finish the workload in the same 7 minutes 30 seconds.  So essentially, we have saved 80W of power for each set of 4 servers and still get the same amount of work completed!  In a large datacenter this can be HUGE in energy savings.

Let's do some quick math:  20W power savings per serer x 10,000 servers = 20kW power savings and you still get the work done.  I hope I just helped some of you server admins get some new ideas on your next "I need a raise" talk with your manager

*your mileage may vary, so test your own workloads and report out!

Why migrate? Why now?

There has never been a better time to migrate your proprietary RISC servers running UNIX(R) to Intel(R) Xeon(R) processor-powered Dell™ PowerEdge™ servers running Red Hat(R) Enterprise Linux(R) Why? Four compelling reasons. First, cost, cost, and cost again. This industry-standard platform can reduce your capital expenditures as well as your operational costs for a lower total cost of ownership (TCO). Second, choice and flexibility. Because you’re not locked into proprietary technologies, you have substantial choices that keep you nimble and agile no matter how your business needs evolve. Third, simplicity. The Red Hat-Dell-Intel platform just works. And acquiring all the products and services you need from one source–Dell–reduces the complexities of both technology procurement and support.  Finally, performance. In these challenging economic times, migrating from RISC and UNIX to a Red Hat-Dell-Intel solution is an easy and fast way to accomplish more with less, bringing true value to your business.

Power Your Enterprise

Because Red Hat Enterprise Linux is optimized for the Intel Xeon processor on which Dell PowerEdge servers are based, you can support your business’s most demanding  challenges. For starters, Red Hat Enterprise Linux 5.3 takes advantage of the Intel Xeon processor 5500 series to deliver more than twice the performance compared toprevious generation Intel processors.1 Because Red Hat Enterprise Linux incorporates Intel’s energy efficiency enhancements, such as integrated power gates and automated low power states to support low-latency changes among power states, you can lower power consumption during off-peak times. This has the additional benefit of reducing datacenter cooling requirements. You achieve previously unattainable scalability with support for up to 255 central processing units (CPUs) and one terabyte of memory. And Red Hat Enterprise Linux supports Intel Hyper-Treading Technology to enable advanced parallel computing.

Learn More

To learn more about migrating from a proprietary RISC /UNIX platform to a Red Hat, Dell, Intel solution navigate to http://www.redhat.com/intelligence/, then click on White Papers.

     As I’m new to The Server Room, I offer this brief introduction:  I am a marketing manager in Intel’s Software and Services Group – looking after Intel’s collaborative marketing efforts with virtualization solution providers.

     A couple weeks ago, Ken Lloyd blogged about the incredible changes in compute capability and performance brought by the Nehalem microarchitecture – and gave credit to the advances in software, too.  I’d like to take the conversation a step further:  did you know that the launch of VMware™ vSphere 4.0 in April 2009 represented a milestone of collaborative development?  The combination of VMware vSphere and Intel Xeon processor 5500 based systems delivers astonishing performance in part because it is the result of a full cycle of collaboration.

     Intel has a well established rhythm of technology innovation – and a lot of really smart architects who know a thing or two about cpu design – but we get innovative ideas from the outside, too.  Over the years of the VMware alliance, Intel has received (and acted on) many requests for small changes in cpu circuitry…changes that would make virtualizing the cpu easier, more efficient, or add capability.  A whole raft of hardware optimizations for virtualization were included in the Nehalem architecture.  As Intel started to deliver early silicon for Xeon 5500 based platforms, Intel software engineers worked closely with VMware engineers – optimizing vSphere code to take advantage of the new hardware features to improve performance, increase efficiency, and add new functionality.  The results?  Check out this video from the launch of VMware™ vSphere 4.0 to see for yourself what “better together” really means.  And the cycle continues – what can you imagine in the next round of collaborative innovation??

“Imagination is everything. It is the preview of life's coming attractions.” Albert Einstein

Today’s workstation can provide you with a magnificent digital canvas to create tomorrow today.

With workstations powered by two Intel® Xeon® 5500 series processors, engineers have the opportunity to create, shape, test and modify products before they become real. Engineers can now design, visualize and simulate products from the conceptual design phase through the entire manufacturing process. This is done virtually before any investments are made in a prototype.

“Experiment fearlessly.” “Innovation is bloody random.” Tom Peters

Peters, a world renowned author and management consultant, recognized that innovation is more art than science.

Consider this example: Taking innovation to an entirely new level, Boeing, in the late 1990s, employed a process known as algorithmic design to see what designs might be viable to meet a specified hypersonic aircraft design criteria. The algorithmic design process enabled computers to create and test new ideas against the specified design criteria without human intervention. As a result, more models were evaluated in less time, and a vehicle that was counterintuitive to what many engineers may have thought possible was evaluated. Innovation just accelerated.

Intel technology has seen dramatic changes since Boeing first tested the idea of algorithmic design in the last decade. Workstation performance has gone up Dual-processor workstations have yielded to workstations with two processors, eight cores and 16 computational threads. Science or simulation that was never tractable on a workstation before is now standard, and it is getting faster.

“I confess that in 1901 I said to my brother Orville that man would not fly for fifty years.” Wilbur Wright

You think all you need is an entry-level workstation with a single Intel® Xeon® processor.                       After all , you only do CAD—right?

However, as you begin to adopt modern workflows and realize the dramatic impact that simulation-based engineering or digital prototyping can have on your product development cost and schedules, you realize that the cost of the second processor and additional memory necessary to support digital prototyping was far less expensive than the cost of multiple physical prototypes and the associated time to produce them. Instead of investigating hundreds of digital prototypes, you only have time to look at a single physical prototype and ask: What if I …?

Those “what ifs” could have been played out on a dual-processor Intel Xeon processor 5500 series-based digital workbench faster, and your time and cost of physical prototypes could have been significantly reduced.

There’s a video going around from one of Intel’s top external customers.  Before you see this (video linked below) I wanted to position this correctly.  I caught up with Mr. X at an undisclosed coffee shop and got his approval to share publicly the messages that we would have rather had him go out with. Those messages are as follows:

Mr. X’s 4 year old servers were a burden on his organization, he spent all of his budget on just maintenance, nothing left for innovation.

He looked at his old infrastructure and determined that replacing them with more powerful-energy efficient servers from Intel was a strategic investment.

The New intel Xeon 5500 based servers provided the opportunity for him to innovate again.  He claimed that these new Intel Xeon Processor 5500 (Nehalem-EP) are the best enabler of IT business value that he's seen in years.

They boosted energy efficiency, saved him big $ and extended his facility lifespan – now he doesn’t have to go build a new data center. 

He replaced his old servers in a 9:1 ratio (getting rid of 9 old and replacing with 1 new) that enabled him to cut operational expenditures by 90% …And that savings alone is paying for the investment in these new servers in just 8 months. 

By strategically investing in IT when his competitors hunkered down and cut spending – he is now positioned to grow faster and gain share as the economic upturn arrives.

Ok, now that I’ve had a chance to convey his real messages, you can check out this video.

These are dog years for servers.   Pretty much every year Intel introduces a new Xeon processor.  Those who have heard the story recognize this as the Tic Tock model.  On Tic years the manufacturing process is updated, on Tock years the chip architecture is updated.  Every year customers get a boost in performance, and often a cut in power.  Typically this boost is in the 50% neighborhood, enough to make it worth the upgrade, and still achievable by engineering teams on a two year cycle.  Except, we are in dog years.

The Nehalem – Xeon 5500 – processor broke all prior boundaries on single generation performance gain.  Delivering two to three times the compute capacity of the Xeon 5400 (Harpertown) generation.  This is a big change, probably a once in a lifetime change – unless that quantum thing happens in my lifetime.  Roughly a 10X performance boost in less than 5 years.

During this same five years we have seen virtualization technology go from a lab project – something for test and dev – to mainstream data center process.  In 2005 it would have been heresy to suggest virtualizing the corporate ERP.  At that point virtualization overhead on the server could be as high as 25% and the entire server was needed to do “real work”.  Fast forward to today.  Virtualization technology in both the hypervisor and processor have reduced overhead to only a few percent, AND servers are 10X faster.  Not only can you virtualize the ERP, you are irresponsibly wasting resources if you do not.  Unless your ERP demands have grown 10X in 5 years, your ERP alone won’t even make a new Xeon 5500 system sweat.

If this advancement wasn’t enough, the announcements last month from Intel about the coming Xeon 7500 (4+ socket) processor were amazing.  All the benefits of the Xeon 5500, but on steroids.  The  new biggest leap ever.  With up to eight cores and four memory channels per socket, this is a monster.  Your ERP system will be barely a blip in perfmon.  It isn’t unreasonable that an entire data center for a SMB business could be virtualized onto one of these beasts.  And, how big is a Xeon 7500 server?  My guess is about the size of a breadbox…

Just wrapped up Oracle Open World…sitting at SFO, waiting for a flight back home. 

The event from a Nehalem-EX perspective was a success. Hit important points and accomplished what we had to deliver. 

Hit #1:  Michael Dell, in his key note, delivered Nehalem-EX message beautifully.  2.5x performance improvements coming from 9x memory bandwidth…compared to currently sold technology.  Thank you, Michael. 

Hit #2:  Dell placed Nehalem-EX demo at its Exhibit at Moscone West.  I missed seeing it in person but the Dell friends came to me reporting that the demo attracted a lot of attention from the audience.  Thank you again, Dell. 

Hit #3:  My Nehalem-EX demo at Intel booth was also a success.  The pre-production system ran throughout the event with 64 logical processors fully active with 1TB of Samsung DDR3 memory, running SPECjbb, stressing all the CPUs, cores, and threads.  Occasionally, I injected double-bit error to show off the MCA-Recovery function.  Windows 2008 R2 reported nicely that the system encountered a critical error but the system still running at full speed.  If not with MCA-Recovery function, I would have had blue screen each time I ran that error injection script and would have had to wait for few minutes to have the server come back up online. 

Also, I really liked the demographics of the audience this time.  Compared to the other events I went to this year, I had more conversations with the folks who actually purchase equipments, those who test new equipments at IT shops, and those from Oracle starting to realize that hardware choice does matter when selling Oracle software.  Many people specifically asked when Intel starts shipping Nehalem-EX and which specific OEM models would use Nehalem-EX.  I hope my responses to those folks were legitimate.  ;-)  I also hope Oracle sales folks now have true confidence that the Oracle software stack runs best on Intel, specifically, Nehalem-EX. 

Oracle Open World is said to be the largest IT event.  I believe that.  You don’t get to have lunch at the middle Mission St tarmac very often.  You don’t get to see four digit hotel bills very often for just couple night stay. Despite the fall storm hitting the peninsula dumping loads of water and gust knocking trees down on Tuesday, Intel booth continuously had heavy flow of traffic.  I admire the Intel team putting together our presence and admire the whole industry supporting the event. I also personally learned a lot from the event, meeting people, exchanging knowledge.  Three day booth duty is a tough one but worth it. 

Oh, and to wrap the whole trip up…

Hit #4:  cleared the wait list and getting home earlier with an earlier flight… 

AND…I wish today was Friday… 

With the Intel Xeon 5500 series (Nehalem) based processors, the X5500 chipset and instrumented power supplies, you can start with the most basic use case for Intel Node Manager - monitoring the power usage of your servers.

As you can see in the Intel Datacenter Manager (DCM) screen below - there are multiple servers configured into logical units:  HF2-EIL is the lab that these servers are located in.  Rack 1 and Rack 2 are the physical location of these servers, and each Rack contains 2 servers each.

When you highlight one server (as above in DCM)- you can see the power characteristics over a certain time period.  The time period shown gives you the idle power, max power, and thermal measurement.  The 'hump' in the graph is a SQL workload which creates 'work' for the server and the process runs for about 5 1/2 minutes with no power capping.

Here's a graph of the 2nd server in that rack, performing a similar workload.  As you can see, the 2nd server power usage is different than the first.

The Intel Datacenter Manager SDK console can monitor multiple systems as well.  The next graph, is both of those servers in the rack, which accounts for both servers power usage during the same timeframe.

Finally, here is the final graph, showing the accumulation of all 4 servers, in both Rack #1 and Rack #2.  This shows the maximum power utilized during the workload, the minimum power (idle) and the inlet thermal temperature in the lab.  Something that hasn't been able to be done before without expensive equipment in the datacenter.

My next power based blog will show how power-capping can give you more effience use of your workload power while using Xeon 5500 series platforms.