Everyone is talking “green-energy” and “power-efficiency” these days. Reducing carbon footprint, renewable energy, CFLs, solar power, biking instead of driving, etc… the list goes on forever. Many people are excited to do something to change power consumption, but as a server administrator - are the proper tools in place?

Many of you have probably experienced the power/efficiency example at home. When the summer gets hot - many of us run to the thermostat and set it accordingly. When it's REALLY hot outside, we tend to twist the dial cooler - knowing all along, that our electric bill will most likely be higher at the end of the billing cycle. So, what do we do?

Some of us just live with the higher bills, some of us turn off the A/C and struggle in the heat - but I'd hope that most of us set the thermostat to a 'livable' temperature - it may not be the coolest, but it's enough to do the job and keep the electricity bills at a more moderate level - in a sense, it's a happy medium. In today's modern age, thermostats are programmable - taking a lot of the guesswork out of our hands and automating many of the old day-to-day temperature functions that our parents had to follow... Intel server platforms are evolving in this realm as well!



As a server admin, do you have the tools and technologies to reduce power consumption? There are several avenues addressing this issue, and I suggest reading the post from Lori Wigle on Data Center Efficiency. The datacenter is different from the desktop… server admins aren’t likely to enable sleep states to save energy – but rather, increase utilization on fewer servers to maximize your performance output in relation to your server footprint.

When was the last time you looked at your server’s power footprint? Do you even know how much power you’re using? Some of you may have some power meters and can monitor a server (or a few servers) at a time… but how many of you can monitor a rack or servers or a datacenter?

What if this capability was built into your current generation Xeon server platform? The good news is that modern processors DO have power management capabilities. Based on the ACPI specs:

P0 Performance State While a device or processor is in this state, it uses its maximum performance capability and may consume maximum power. Thereby the processor uses it's maximum power allocation.

P1 Performance State In this performance power state, the performance capability of a device or processor is limited below its maximum and consumes less than maximum power.

Pn Performance State In this performance state, the performance capability of a device or processor is at its minimum level and consumes minimal power while remaining in an active state. State n is a maximum number and is processor or device dependent. Processors and devices may define support for an arbitrary number of performance states not to exceed 16.

Each Pn State is a "notch" in the processor's performance powerband (as seen below)

As these performance notches are set, the processor will lower it's power envelope and reduce the power needed in order to save energy. Just as a note, EIST must be enabled in the BIOS for this performance enhancement to work on your platform.

If you attended Intel’s IDF (Intel Developer Forum) you may have run into a few demos in regards to Datacenter Power Management, my booth showcased 4 current generation Intel Servers based on Bensley/Starlake Xeon DP boards and Xeon 54xx Series (codename Harpertown) Processors.

Here’s a quick video showcasing the demo – and just a note - we’ll be redoing this in a higher-quality format soon – so stay tuned!
Hopefully if you’ve watched the video – you’ve got some questions! The good news is that we have a new website from the Intel Software Network that is focused on Intel® Dynamic Power Datacenter Manager. The site lists the features, system requirements, downloads, and FAQ to get you started!

I’m looking forward to your feedback and questions!

.

Last week, the first part this video series focused on the energy efficiency benefits of 45nm. The 2nd part of this video (below) is focused on the benefits of 45nm for virtualization and the intel processor roadmap including what's next in 45nm processor technology - the Dunnington and Nehalem-EP products

Is this information useful to you? why or why not?

Chris

Have you ever asked yourself that question when you are bombarded with marketing messages from multiple different companies on why choose their products vs. a competitors product?. As a non-Engineer in an engineer centric company, I certainly have thought about this several times and asked myself a very simple question - Why should I choose one architecture type over another offering?

I suppose the best place is to start at the beginning and try and decipher the acronym soup of RISC, x86 etc. I decided to use my ‘old friend’ Wikipedia http://www.wikipedia.org/ to help with this process. What I found was another alphabet soup that I could have researched for hours, but try and simplify it below. I attach my detailed definition findings at end of this blog.

Simply put, RISC (pronounced risk) is a CPU design to use simplified instructions to execute very fast thus providing higher performance. x86 is a generic term that refers to the instruction set of another CPU architecture. So basically both RISC and x86 are types of instruction sets linked to CPU architecture.

So which one should I choose?.
Call me old fashioned, but as a business guy, it always comes down to 3 basic tenets in terms of making a decision
1) I like choice and the ability to pick and choose between multiple suppliers to get the best deal to meet my needs.(and the ability to change supplier without major obstacles)
2) Performance is really important. The higher performance means that I get my work done quicker which reduces the overall cost / improves time to revenue and ultimately improves the productivity of my business
3) System cost and total cost of ownership are key decision points in today’s era which is vastly different from the ‘dot.com’ boom. It is all about managing the bottom line through good decisions around CAPEX and OPEX spending

I applied my decision criteria and quickly found out that there is not a lot of choice from a hardware and operating system perspective with RISC architecture. In fact it looks quite the opposite of choice which always concerns me, call me pro-choice if you like, but I like the ability to move around suppliers!. On the other hand I found x86 to have lots of choice with many hardware vendors to list and a range of operating systems from windows to Linux and Solaris.

Having choice out of the way, I then moved onto performance for my business and looked at published results from many hardware vendors on different websites like http://www.spec.org. what I found was that Intel based systems had a lot of leading results against architectures like SPARC from SUN or Fujitsu and POWER from IBM.

I then looked at price (and being an ex-Accountant in my past career) nearly jumped for joy when I saw that system prices were low for x86 systems compared to the comparable RISC systems.

This analysis helped me understand it better and helped simplify my decision making.

Here is a short video with a little bit more detail. I would be interested in your thoughts and have you had any similar experiences that you would like to share.

This is part three - the implication being that it is a sequel to part one and part two. It is. That said, each of the sections have their own messages and may or may not help your data center. The first part talked about the benefits of bringing in the latest hardware. Intel has been delivering performance increases at a pace beyond "Moore's Law". Getting rid of old, slow, inefficient servers can give you 2-12 times the capacity instantly. The second "episode" talked about getting everything you can from each server. Use virtualization and consolidation to make sure your servers are full and busy. The most efficient bus is a full bus ( this is a metaphor, I am talking about the big yellow things carrying students, not the circuitry in the box )

My focus in part three is on density. My operating premise is that the data center manager wants to get everything out of the current data center and avoid, or at least defer, construction of a new data center. If your in the data center construction business, this is not for you.

To get the most out of our data center we want to pack every server we can power into the space. You can do this by executing three actions. 1) Use every watt, 2) Build the right servers, and 3) Optimize HVAC. In many cases twice the servers can be crammed into the existing rack space even without adding power. If you are able to redirect your hvac power savings to your racks, your results could be even better.

So, we potentially got 5x capacity from new quad core servers, 5x capacity from boosting utilization with consolidation, and 2x capacity with higher density. My math says 5x * 5x * 2x = 50x the capacity ( in the same space and power!) video

Datacenter Power Management: Power Consumption Trend

Jackson He

As the internet services grow and the more users embracing internet - approaching 1 billion connected users, one of the biggest challenges for data-center operators today is the increasing cost of power and cooling as a portion of the total cost of operations. As shown in Figure 1, over the past decade, the cost of power and cooling has increased 400%, and these costs are expected to continue to rise. In some cases, power costs account for 40-50% of the total data-center operation budget. To make matters worse, there is still a need to deploy more servers to support new business solutions. Data centers are therefore faced with the twin problem of how to deploy new services in the face of rising power and cooling costs. In a recent survey of data centers 59% identify power and cooling as the key factors limiting server deployment.

Figure 1: IDC Report of data center cost structure and trend

At the same time with the increased energy cost and awareness of global warming, there is increased regulatory scrutiny around both idle and max power of servers and clients (desktops and laptops). The "green awareness" datacenter is no longer a "nice to have" feature, but of necessity of business operation and environmental regulatory compliance. Figure 2 highlight the world-wide existing and emerging regulations on power and energy consumption. Future datacenters have to be able to clearly measure and proof regulation conformance in order to operate properly.

Figure 2: Existing and emerging energy and power regulations

To sum it up, the power management trends for future datacenters are multifaceted and will not be covered by a single company or a single business segment. They could be summarized in the following areas:

• At environment level: conform to increased government regulations on energy and power and increased power constraint (limited available power) - need innovative ways to conform "green datacenter" regulations, while deliver great values to business.
• At the datacenter level: more computing power is needed with increased demand; emergence of mega datacenter and modular datacenter (datacenter in a container); the overall power and cooling distributions need to match the increased need - new datacenter designs and power/cooling management needed.
• At rack level: higher power density and higher server density per rack is needed to pack more computing power for a given space and cooling; workload balance between racks to increase power efficiency and overall datacenter reliability - need effective rack-level power and cooling monitoring and dynamic management capabilities
• At server level: need lower idle and max processing power, so that platform power consumption trend is more linear with platform performance; dynamically adjust power consumption based on policy and workload - need more server-level instrumentations for power/cooling monitoring and more control knobs to dynamically optimize power and performance.

I hope you agree with me of the overall datacenter power management trends at datacenters in the coming year. These trends pose challenges for each of the areas listed above. These challenges also mean opportunities for innovative solutions to thrive. I'd like to listen to your feedback about these trends. I will talk more about challenges and potential solutions in the upcoming blogs. You are welcome to share your thought of where you believe the datacenter power management is going. Thanks a lot.

I have been working as a full-time performance engineer at Intel for 6 years. I started by benchmarking server products for performance validation and now I focus on the TPC-C and TPC-E OLTP server benchmarks. I have used a variety of workloads in this job and spent time optimizing each level of the performance hierarchy: application, system, and processor. I, like many of you, have learned the "tricks of the trade" the hard way: by trial, error, and success. I'm sharing now, so you can all benefit from the things I've picked up along the way.

Let's start with some general methodologies to follow when tuning performance, whether you do it full-time, as a hobby, or just in your spare cycles after getting your "regular work" done. I will follow up with a more detailed post on each habit individually.

1. Ask the right question: Why are you tuning your platform? What level of performance are you hoping to achieve? What do you (or your users) care most about: raw performance, cost/performance, performance/watt, or something else?

2. Start at the top: The first and easiest part of your application server to tune is the hardware itself. Move on to the software and workload only after you feel confident that you have removed any system-level bottlenecks.

3. Know your Platform: This should be where you begin your system (hardware) tuning. The first thing, which I can't stress enough, is to get a block diagram of your platform. Then study it!

4. Know your BIOS: Server BIOSes these days come with more and more options. Be sure to give your new platform's BIOS a once-over. Pay particular attention to options relating to performance and power.

5. Know your Workload: To quantify performance, you need a workload! Some examples: web server response time, boot time, frames rendered per second, simultaneous connections supported, etc. Understand as much as possible about how the work gets done.

6. Try one thing at a time: Little changes that seem harmless can significantly alter the behavior of your system. Or worse, they can interact with each other to wreak havoc. Always try one change at a time, and for goodness' sake, do habit number 7.

7. Document and Archive: When you change something, log it! For each experiment you do, store your hardware and software configuration, performance level, and any collected data.

8. Use the right tool for the job: There are free data collection tools out there for various levels of the tuning process. System tuning tools include such as Performance Monitor for Windows or Sar for Linux. Application-level tools include Intel ® VTune^TM for both Windows and Linux.

9. Don't break the law: Amdahl's Law, that is. Amdahl's Law tells us the maximum amount of performance improvement we will get from a particular enhancement. Amdahl can help you set your expectations properly and clue you in to when you should be suspicious.

10. Compare apples to apples: Todd Christ reminds us of this habit in the last paragraph of this post. Don't compare the performance of mis-matched systems. If you must do it, know exactly what the differences are: the processor, memory type/speed/vendor, a software component, chipset, etc. Dig into the configuration details!

So now you have the high-level list! Stay tuned to The Server Room for more information about each habit in the coming weeks.

In the second comment around the right time for datacenter refresh, I'd like to look at Costs. Power is covered in the comment from Chris and I covered some comments on Space already in the discussion forum. So what it really boils down to is cost of running your existing datacenter versus the costs of throwing the servers out and replacing them. It is clear also from the other comments, that it doesn't make sense to throw out servers which are utilized in average 15% and have them replaced by new servers, which are 5 times faster and utilize the servers 3%... Great achievement hu?... Server Refresh makes therefore most sense to do only when consolidating the environment. How do I consolidate the environment? By using virtualization. See Helmuts blog and the whole theme next week on that topic.

Therefore let's look at the real cost factors, when refreshing the servers:

• Cost of new hardware: That is obviously a significant capital expenditure and starting at about 2000$ for a reasonable DP server. But the trick is also that a lot of server companies offer financing models which make this an operational expenditure. But key is also to understand, that by consolidating your servers at the same time the depreciation costs of the servers may actually decrease, as you have less hardware to depreciate!
• Maintenance costs: Again, reducing the number of servers running given applications, and at the same time unifying the environment helps significantly to reduce the maintenance costs. This can be a significant step in unifying on a given OS or hardware platform.
• Power consumption: Similar to utilization, it doesn't make sense to just look at the power consumption by server, but at the consumption by performance and therefore I can save about 38% in power bills, on a given workload vs. the previous generation hardware and about a 10^th^ of the power of hardware which is 2-3years old. Again, obviously only, if I do this in combination of consolidating the servers. Trick often is, that those costs are often not taken into consideration, as those are not billed to the IT department but to the facilities group. So it becomes an executive decision to ensure they are looked at!.
• Switching costs. Obviously very hard to measure, as this depends on the environment of the customer. And I talked to the customer who said: "No I will never touch this AS400 system, as it just runs and runs and runs." On the other hand I had a customer who replaced just those AS400 systems and saw huge synergistic effects, because he put the application on a standard based architecture and was able to finally integrate it in the other production system and therefore have one reporting and analytics tool.

I try to make a long story short. This is not something you do very often, but you don't get married every year either. But most of the time it's worth going through the efforts. So thinking about replacing the servers which are older than 2-3years is definitely worth while and often an effort which pays off in the first year!

Watt do you care about more?
the Power Consumption of your servers (watts) or the Power Efficiency of your servers (performance / watt)
... or maybe you prefer the Performance per Watt per SqFt argument

I have spent a lot of my time the last several years discussing this topic with IT professionals around the world - and there are a lot of varying opinions.

I believe that Performance per Watt is a better measure of overall value for the data center and server room.
The power consumed by a server is an important measure, but power only comparisons can be misleading.

Example: If server ‘A' consumes 50W less power than server ‘B', then it can save IT $79 per year per server in power and cooling costs (assumes $0.08 kW/hr power costs and cooling costs equal to power costs). Scale that $79 savings per server across a data center with thousands of servers and it can be a pretty impressive number.

However, if a server with 50W lower power delivers lower application performance ... is the power savings worth it? The answer of course depends ... but generally in my experience the answer is a resounding No.

Example: What if server A (the 50W lower power server) underperforms server B by 33% in performance. This means that you need to deploy more ‘A' Servers to get the same performance as ‘B' Servers. In fact, with a 33% performance advantage, you need only 3 ‘B' servers for every 4 ‘A' servers. The higher performance per Watt delivered by server B reduces acquisition costs, reduces power consumption (less servers) and minimizes space and eases manageability. This example is shown graphically above

What do you think? What power and performance metrics do you look at before purchasing servers
... Lower Power or Higher Performance per Watt?