This is part three - the implication being that it is a sequel to part one and part two. It is. That said, each of the sections have their own messages and may or may not help your data center. The first part talked about the benefits of bringing in the latest hardware. Intel has been delivering performance increases at a pace beyond "Moore's Law". Getting rid of old, slow, inefficient servers can give you 2-12 times the capacity instantly. The second "episode" talked about getting everything you can from each server. Use virtualization and consolidation to make sure your servers are full and busy. The most efficient bus is a full bus ( this is a metaphor, I am talking about the big yellow things carrying students, not the circuitry in the box )

My focus in part three is on density. My operating premise is that the data center manager wants to get everything out of the current data center and avoid, or at least defer, construction of a new data center. If your in the data center construction business, this is not for you.

To get the most out of our data center we want to pack every server we can power into the space. You can do this by executing three actions. 1) Use every watt, 2) Build the right servers, and 3) Optimize HVAC. In many cases twice the servers can be crammed into the existing rack space even without adding power. If you are able to redirect your hvac power savings to your racks, your results could be even better.

So, we potentially got 5x capacity from new quad core servers, 5x capacity from boosting utilization with consolidation, and 2x capacity with higher density. My math says 5x * 5x * 2x = 50x the capacity ( in the same space and power!) video

Datacenter Power Management: Power Consumption Trend

Jackson He

As the internet services grow and the more users embracing internet - approaching 1 billion connected users, one of the biggest challenges for data-center operators today is the increasing cost of power and cooling as a portion of the total cost of operations. As shown in Figure 1, over the past decade, the cost of power and cooling has increased 400%, and these costs are expected to continue to rise. In some cases, power costs account for 40-50% of the total data-center operation budget. To make matters worse, there is still a need to deploy more servers to support new business solutions. Data centers are therefore faced with the twin problem of how to deploy new services in the face of rising power and cooling costs. In a recent survey of data centers 59% identify power and cooling as the key factors limiting server deployment.

Figure 1: IDC Report of data center cost structure and trend

At the same time with the increased energy cost and awareness of global warming, there is increased regulatory scrutiny around both idle and max power of servers and clients (desktops and laptops). The "green awareness" datacenter is no longer a "nice to have" feature, but of necessity of business operation and environmental regulatory compliance. Figure 2 highlight the world-wide existing and emerging regulations on power and energy consumption. Future datacenters have to be able to clearly measure and proof regulation conformance in order to operate properly.

Figure 2: Existing and emerging energy and power regulations

To sum it up, the power management trends for future datacenters are multifaceted and will not be covered by a single company or a single business segment. They could be summarized in the following areas:

• At environment level: conform to increased government regulations on energy and power and increased power constraint (limited available power) - need innovative ways to conform "green datacenter" regulations, while deliver great values to business.

• At the datacenter level: more computing power is needed with increased demand; emergence of mega datacenter and modular datacenter (datacenter in a container); the overall power and cooling distributions need to match the increased need - new datacenter designs and power/cooling management needed.

• At rack level: higher power density and higher server density per rack is needed to pack more computing power for a given space and cooling; workload balance between racks to increase power efficiency and overall datacenter reliability - need effective rack-level power and cooling monitoring and dynamic management capabilities

• At server level: need lower idle and max processing power, so that platform power consumption trend is more linear with platform performance; dynamically adjust power consumption based on policy and workload - need more server-level instrumentations for power/cooling monitoring and more control knobs to dynamically optimize power and performance.

I hope you agree with me of the overall datacenter power management trends at datacenters in the coming year. These trends pose challenges for each of the areas listed above. These challenges also mean opportunities for innovative solutions to thrive. I'd like to listen to your feedback about these trends. I will talk more about challenges and potential solutions in the upcoming blogs. You are welcome to share your thought of where you believe the datacenter power management is going. Thanks a lot.

I have been working as a full-time performance engineer at Intel for 6 years. I started by benchmarking server products for performance validation and now I focus on the TPC-C and TPC-E OLTP server benchmarks. I have used a variety of workloads in this job and spent time optimizing each level of the performance hierarchy: application, system, and processor. I, like many of you, have learned the "tricks of the trade" the hard way: by trial, error, and success. I'm sharing now, so you can all benefit from the things I've picked up along the way.

Let's start with some general methodologies to follow when tuning performance, whether you do it full-time, as a hobby, or just in your spare cycles after getting your "regular work" done. I will follow up with a more detailed post on each habit individually.

1. Ask the right question: Why are you tuning your platform? What level of performance are you hoping to achieve? What do you (or your users) care most about: raw performance, cost/performance, performance/watt, or something else?

2. Start at the top: The first and easiest part of your application server to tune is the hardware itself. Move on to the software and workload only after you feel confident that you have removed any system-level bottlenecks.

3. Know your Platform: This should be where you begin your system (hardware) tuning. The first thing, which I can't stress enough, is to get a block diagram of your platform. Then study it!

4. Know your BIOS: Server BIOSes these days come with more and more options. Be sure to give your new platform's BIOS a once-over. Pay particular attention to options relating to performance and power.

5. Know your Workload: To quantify performance, you need a workload! Some examples: web server response time, boot time, frames rendered per second, simultaneous connections supported, etc. Understand as much as possible about how the work gets done.

6. Try one thing at a time: Little changes that seem harmless can significantly alter the behavior of your system. Or worse, they can interact with each other to wreak havoc. Always try one change at a time, and for goodness' sake, do habit number 7.

7. Document and Archive: When you change something, log it! For each experiment you do, store your hardware and software configuration, performance level, and any collected data.

8. Use the right tool for the job: There are free data collection tools out there for various levels of the tuning process. System tuning tools include such as Performance Monitor for Windows or Sar for Linux. Application-level tools include Intel ® VTune^TM for both Windows and Linux.

9. Don't break the law: Amdahl's Law, that is. Amdahl's Law tells us the maximum amount of performance improvement we will get from a particular enhancement. Amdahl can help you set your expectations properly and clue you in to when you should be suspicious.

10. Compare apples to apples: Todd Christ reminds us of this habit in the last paragraph of this post. Don't compare the performance of mis-matched systems. If you must do it, know exactly what the differences are: the processor, memory type/speed/vendor, a software component, chipset, etc. Dig into the configuration details!

So now you have the high-level list! Stay tuned to The Server Room for more information about each habit in the coming weeks.

In the second comment around the right time for datacenter refresh, I'd like to look at Costs. Power is covered in the comment from Chris and I covered some comments on Space already in the discussion forum. So what it really boils down to is cost of running your existing datacenter versus the costs of throwing the servers out and replacing them. It is clear also from the other comments, that it doesn't make sense to throw out servers which are utilized in average 15% and have them replaced by new servers, which are 5 times faster and utilize the servers 3%... Great achievement hu?... Server Refresh makes therefore most sense to do only when consolidating the environment. How do I consolidate the environment? By using virtualization. See Helmuts blog and the whole theme next week on that topic.

Therefore let's look at the real cost factors, when refreshing the servers:

• Cost of new hardware: That is obviously a significant capital expenditure and starting at about 2000$ for a reasonable DP server. But the trick is also that a lot of server companies offer financing models which make this an operational expenditure. But key is also to understand, that by consolidating your servers at the same time the depreciation costs of the servers may actually decrease, as you have less hardware to depreciate!

• Maintenance costs: Again, reducing the number of servers running given applications, and at the same time unifying the environment helps significantly to reduce the maintenance costs. This can be a significant step in unifying on a given OS or hardware platform.

• Power consumption: Similar to utilization, it doesn't make sense to just look at the power consumption by server, but at the consumption by performance and therefore I can save about 38% in power bills, on a given workload vs. the previous generation hardware and about a 10^th of the power of hardware which is 2-3years old. Again, obviously only, if I do this in combination of consolidating the servers. Trick often is, that those costs are often not taken into consideration, as those are not billed to the IT department but to the facilities group. So it becomes an executive decision to ensure they are looked at!.

• Switching costs. Obviously very hard to measure, as this depends on the environment of the customer. And I talked to the customer who said: "No I will never touch this AS400 system, as it just runs and runs and runs." On the other hand I had a customer who replaced just those AS400 systems and saw huge synergistic effects, because he put the application on a standard based architecture and was able to finally integrate it in the other production system and therefore have one reporting and analytics tool.

I try to make a long story short. This is not something you do very often, but you don't get married every year either. But most of the time it's worth going through the efforts. So thinking about replacing the servers which are older than 2-3years is definitely worth while and often an effort which pays off in the first year!

Watt do you care about more?

the Power Consumption of your servers (watts) or the Power Efficiency of your servers (performance / watt)

... or maybe you prefer the Performance per Watt per SqFt argument

I have spent a lot of my time the last several years discussing this topic with IT professionals around the world - and there are a lot of varying opinions.

I believe that Performance per Watt is a better measure of overall value for the data center and server room.

The power consumed by a server is an important measure, but power only comparisons can be misleading.

Example: If server ‘A' consumes 50W less power than server ‘B', then it can save IT $79 per year per server in power and cooling costs (assumes $0.08 kW/hr power costs and cooling costs equal to power costs). Scale that $79 savings per server across a data center with thousands of servers and it can be a pretty impressive number.

However, if a server with 50W lower power delivers lower application performance ... is the power savings worth it? The answer of course depends ... but generally in my experience the answer is a resounding No.

Example: What if server A (the 50W lower power server) underperforms server B by 33% in performance. This means that you need to deploy more ‘A' Servers to get the same performance as ‘B' Servers. In fact, with a 33% performance advantage, you need only 3 ‘B' servers for every 4 ‘A' servers. The higher performance per Watt delivered by server B reduces acquisition costs, reduces power consumption (less servers) and minimizes space and eases manageability. This example is shown graphically above

What do you think? What power and performance metrics do you look at before purchasing servers

... Lower Power or Higher Performance per Watt?