Home > Intel Communities > Open Port IT Community > The Server Room > Blog > 2007 > November > 21
Currently Being Moderated
4

Continuing on the theme of measuring Data Centre efficiency - power consumption of the facilities and IT load are only one element albeit a large one - that contributes to the overall efficiency of a data centre. Ultimately a DC has to deliver useful workload and the amount of workload that can be achieved within a given physical DC is an increasing challenge. Lowering server power and increasing the cooling effectiveness of a DC are one of several ways to enable more equipment to be installed into an existing facility.

 

General consensus seems to be that the servers in many data centres do not always run a maximum utilisation - many are in the 10-15% utilisation range. This results from many IT shops following a policy of hosting one workload ( application ) per server and sizing the server to support worse case usage of that workload - this leads to low average utilisation of the servers. There are several approaches that can be taken to increasing the server utilisation

 

Consolidating several applications onto the same server that have different mixes of utilisation - this is not perfect as a problem on one application could impact the others on that server causing significant business impact

 

Deploying virtualisation within the DC - this enables multiple OS/App instances to be run on the same server. There are multiple benefits here in that the server utilisation increases whilst the number of servers could potentially be decreased so reducing the overall electrical power consumption of the DC and consequently the utility bill. Another aspect of virtualisation is that to achieve the highest levels of consolidation it is best to deploy the latest generation high perf/low power servers, this can result in the removal of many older generation high power servers from the Data Centre and the deployment of a smaller number of newer more power efficient servers

 

There are circumstances where virtualisation may not be appropriate and it is necesseary to retain one workload per server - in this case an increase in the workload capacity of a DC can be achieved by replacement of older smaller servers with the latest generation high performance servers - this can enable the workload capacity of a DC to be significantly increased without building a new DC, again the side benefit here is that latest generation servers consume less power than the older servers they are replacing.

 

There are many different ways in which the workload capacity ( and hence utilisation ) of a DC can be increased , with care most can also result in a reduction in the electrical power consumed by the DC.

 

Given the right tools the utilisation of servers within a DC is 'relatively' easy to measure, so this element of DC effectiveness can be quantified. There is another major element that I believe contributes to the effectiveness of a DC - that is the processes that are in place to manage the DC and hence the way a DC can respond to the new challenges placed on it by a business unit. Gartner have an infrastructure maturity model that is useful to try and quantify how effective a DC is in responding to business needs and looks at responsiveness, Service Level Agreements, IT processes etc. Currently I do not believe many DC managers are measuring how effective their DC in terms of process and when asked to judge where they sit within a model like Gartner's many IT managers will judge themselves more efficient than they really are.

 

Are there other areas that contribute to the efficiency of a DC - I would be interested in your feedback.



Add a comment Leave a comment on this blog post.
Mar 27, 2008 10:40 AM Guest Tim Walsh  says:

An area where a vast amount of money and energy could be saved is by a concerted IT industry effort to do away with the need for air conditioning of computer rooms and data centres altogether. The exact percentage of computer room and data centre energy consumed by the air con depends on who you talk to, but it is somewhere around 40% to 60% of the total.

 

BT has recently gone public with a statement that it is running its hosting centres with fresh air cooling.

 

"To provide some sense of scale, BT’s data centres, the largest in

Europe, use close to 0.7% of the total power output for Great

Britain. The traditional formula for computer cooling is that for every 1 kilowatt used to actually run a computer, you need 1.2 kilowatts of power to cool it. With fresh air cooling as pioneered by BT, refrigeration becomes unnecessary for a majority of the year, reducing those costs by approximately 85%. BT also runs its servers hotter, extending parameters to between 5 and 40 degrees Celsius, further reducing cooling costs. Presently, BT has 107 data centre sites using fresh air cooling as well as 5600 telephone exchanges."

 

The remarkable findings of a huge Google study, in a white paper publish early last year, show that disk drive failures are INCREASED by lower temperatures, and that the optimum ambient temperature for minimum failure rates is somewhere around 35 to 40 deg. C.

 

http://research.google.com/archive/disk_failures.pdf.

 

In our UK office our company runs our 6 main network servers right in the open office, in specialist soundproofed cabs with fans in them to recalculate the office air. That office has never turned on it's heating in the winter and on a hot summer day they just open the windows. The building has no aircon. They have never had a hardware problem since the servers were put there four years ago.

 

Why do we persist in thinking that servers have to be operated only at 20 C or so? Because radical change of this sort is the hardest thing for an IT manager to execute on. He is judged on uptime, uptime and uptime. The status quo works. The server manufacturers continue to state maximum operating temperatures of 35 C and Sun, for example states a 22 C optimum in its best practice guide. How much of that is based on an unwillingness to risk bad PR and warranty claims by going into untested territory?

 

The server makers have a responsibility to work on testing that territory and improving their recommendations accordingly. If their servers really are intolerant to periodic exposure to higher temperatures, as might happen using fresh air cooling in a country with hot summers, they should get far more R&D resource onto it fix that. Think of the coup of being the first to market with servers that don't need cooling.

 

If people are still afraid of downtime at higher temperatures, they should do like Google does. Get the focus off the possibility of a hardware failure and put it onto super-sophisticated fail-over software and systems that make hardware failures more or less irrelevant to uptime.

 

Our society collectively put a man on the moon nearly 40 years ago. That society now requires that the IT industry drastically cuts its power consumption for reasons plain to all. I submit that if the right companies sat around the right table, and worked collectively, the need for computer room and data centre air con could be virtually eliminated. It cannot be beyond the wit of Man, or the leviathan IT industry, if responsibility is allowed to drive away technical inertia.

 

Tim Walsh

CEO

http://www.kellsystems.co.uk

Apr 9, 2008 1:15 PM David Jenkins David Jenkins    says:

Tim - first off, my apologies for the delay in my response. I was in Shanghai for IDF last week which impacted my ability to respond.

 

Thank you for your outstanding comments and a very insightful look at data center temperatures. Intel certainly shares your interest in lowering energy use in Data Centers. We are actively involved with ASHRAE's TC 9.9 who sets guidelines for Data Center temperature and humidity and is, as we write this, actively pursuing an expansion of the recommended temperature range for high end data centers. The current range is 20-25C and we do expect that to be broadened somewhat.

 

There are two items to consider when cooling outside of the current range. First is that anyone cooling below that range is likely wasting energy unless there is a specific need for it with their current IT equipment, but there is very little in the mainstream that would need these temperatures. Second, running at temperatures warmer than the upper limit may actually cause you to waste energy in a typical data center. It's not widely understood, but generally temperatures between 25C and the 35C you mention will cause the server fans to run much harder as well as increase silicon leakage power. These additional power uses may offset the more efficient cooling system operation that could be gained at these warmer temperatures.

 

The real savings potential in running at the upper limit of the current or expanded recommended range will be in the expanded use of economizers or free cooling. Being able to operate with the cooling plant turned off and employing an air or liquid side economizer can be very efficient from an energy and a cost perspective.

 

Is there even greater savings by being able to run servers with no cooling anywhere anytime? That is a question we continue to evaluate, clearly the servers would cost more and would take up more space to be able to dissipate their heat to the warmer airflow. (and we have not even touched on the reliability impacts of high humidity or the ESD risks with low humidity) the key question is how many more hours per year of "free cooling" (which isn't really free by the way) could be had, and is it worth the server cost and density impact? Cooling in DCs is only partially the chiller, air handlers make up a significant portion of the energy used and they actually use more energy in a free cooling scenario with outside air due to the increased flow restrictions associated with the additional required filtration.

 

We don't have all the answers yet but lets consider a little bit more about the cost of cooling. You suggest that 1.2 kW are needed to cool 1.0 kW of IT load. If we assume an ~80% overall efficiency for power distribution and some load for lighting and other uses, the data center you suggest would have a PUE of 2.4 to 2.5. (PUE = total load / IT load, where total includes IT, Cooling, Power Distribution, Lighting, etc...). While there are some data centers operating in this range, modern well designed DCs WITHOUT additional efficiency features built in are typically in the 1.8 to 2.0 range. Advanced DCs with economizers can have PUEs as low as 1.4... so the cooling load is already a much smaller fraction of the total load.

 

We believe the next step should be a widening of the recommended range and a much greater use of economizers to reduce energy demand. That is something that can be done in data center designs this year with the majority of the potential cooling savings there with the right design. The no-cooling anytime, anywhere server will take us a bit longer...

Apr 9, 2008 2:07 PM Alan Priestley Alan Priestley    says:

On the topic of economizers Intel's own IT department has done some work in the area of data-centre cooling - a couple of links here

 

http://www.intel.com/technology/eep/data-center-efficiency

 

http://www.intel.com/it/pdf/reducing-dc-energy-consumption-with-wet-side-economizers.pdf

 

 

 

This is one of area being looked at by both the Green Grid and the team working on the EU Code of Conduct for Data Centres to provide recomendations as to how data centre managers can design their facilities to better handle the cooling requirements of modern day server infrastructure.

 

In the interim before we get to the nirvana of systems that do not need forced air cooling many of the latest generation servers include power management features that enable system power to be reduced when workload demand is low, also the use of live migration in a virtualised infrastructure can enable workloads to be moved based on server utilisation and for servers to be shut down when capacity is not required - this can help reduce overall cooling requirements of a data centre in periods of low demand.

Sep 10, 2008 9:11 AM Guest shuan  says:

Nice. i also use the computer and i think in the today world information is more useful for all the user and easy way to get the information. and also great thing that intel and other related companies save there data in a small thing like chip board hard drive and etc. i relate from real estate business by the name Merricks Resort, Barbados. i have also alot of clients detail which i save in the computer had drive .and i hope that intel introduce new things in which we can save our data easily.