IT@Intel Data Center Blog

Innovating the Thermal Management of High Density Servers

john.g.miner@intel.com — Mon, 22 Sep 2008 20:09:21 GMT

It's great server weather here in New Mexico today. The current temp is 74f , the high will be ~82f ,the low was 57f ,and the humidity will be ~30% all day. These are all "in range" of the air needed for a densely packed server to breath. Depending on how many servers are packed into a rack, they can heat this 74f degree air by as much as 50f degrees. Removing that heat from the server exhaust air consumes energy and expensive equipment. A key system trade off in data center design is that density reduces the unit cost of deploying and operating servers for most inputs but increases the cost of dealing with high density heat. If we can find ways to address this heat with less energy and cost then we can be more economical, and in today's words, more green.

It was on a day like today, a while back, where Tom Greenbaum, a fellow Intel engineer, and I were brainstorming how to get outside air directly to the servers. Because of Tom's experience in custom air handlers, we were focusing on economizers that are normally used for buildings. Economizers are commonly designed into office buildings, homes and in a simple fashion, most cars, but they are rarely are thought of for data centers. The key concept is that you have access to two air supplies: the outside air and the exhaust air. To be most economical, you want to continually decide "what is the best air to use next?". This takes measuring, deciding and switching. In your car, you are both the measurement device and decider as you press the re-circ button or the flow-through button on the dash. In a highend building air conditioner this is an outside weather monitor, an inside weather monitor, a simple controller and some extra duct work.

In our experiment we used an vane controller in a looped air duct that could blend outside air with the exhaust air to get us the best air for the servers to inhale at the lowest cost. The accuracy and controllability of our monitors and controllers were not as capable as we wished, but they did the job. The perfect controller would have measurements of outside air and exhaust air for: temperature, enthalpy and dew point. It would have policies that you could modify for best economy based on the requirements of your equipment. For example, it should be able to blend exhaust heat into the outside air to hold the minimum temperature at say 55f and no more in winter and it should be able to start incremental cooling loads as the temperature of the coolest of the two supplies rises above the maximum allowed by your equipment, in summer. It would decide to use some exhaust air when the dew point was higher than the outside temperature to control air humidity. You get the point, it would condition the air using all available inputs. Our PoC used DX cooling based units that usually are considered not as economical as water based cooling. But, in this mode of operation, they worked well and reduced complexity. In addition they used no water which is a plus in many desert locations. You can imagine evaporative systems in similar designs that could replace the DX units or work with the DX units for even more "economi-zation".

In the video you will see dust on the back of the servers. We had filtration on the air intakes and a control system that can indicate when the filter needs replacing, but we had a door system that let in dust near the inputs during really windy days. It was a design flaw in our temporary room. Once it got into the system we decided to let it run to see where it went. It collected in the exhaust areas but then created very little risk becasue most of the time we exhausted the air. Something that would not have been true in a closed loop system.

Don Atwood was able to negotiate for and create the production capable configuration for a sufficient number of production servers that were dense enough to run a proof of concept (PoC). These servers run high volume batch computing and are nearly always running above 90% utilized, perfect heaters for the job. It is important to note that this PoC was dependent on the concept of a "Compute Center". A Compute Center is the idea that high density servers can be isolated in their own air space a very short distance from the storage. The storage is left in the classical, and perhaps now more aptly named data center. Where this concept is able to be used, it can help free up traditional close loop environmental control for storage systems. If anyone knows of a great economizer controller, please lets us know. An Atom based design would be a plus

The temporary "compute center" we established and operated would not have be successful without the help of several great engineers contributing insight and innovation. Tom Greenbaum, Marvin Bailey, Steven Bornfield, Natasha Bothe, Greg Botts, Demetruis Ferguson, Ryan Henderson, Dan Links, Don Wright all contributed to learning what is possible during our PoC.

Well, it's 79f outside now and still great server weather, there are hot air balloons in the sky this morning, as we get ready for the Balloon Fiesta here in early October. Hot air balloons, like racks of servers, love this air becasue they can inhale cool air and then heat it to create work that moves people. Finally hot air balloons spend very little energy exhausting the resulting heat out the top. Its a simple model really, it just takes a smart controller.

Video: http://video.intel.com/?fr_story=2d6e0fbbef76b72c6119cc7fe7889bba20cb5192&rf=bm

Paper : http://www.intel.com/it/pdf/Reducing_Data_Center_Cost_with_an_Air_Economizer.pdf

Free Cooling for Data Centers - video and whitepaper

don.c.atwood@intel.com — Fri, 19 Sep 2008 18:20:06 GMT

Hi I’m Don Atwood, author of the newly released white paper and video that discusses our proof of concept (PoC) that tested cooling our Data Center with outside air. The topic of humidity control and if this would work in an ultra high humid climate keeps coming up. Most OEM spec’s allow for a wide range of humidity and it’s our belief that this cooling methodology could be used almost everywhere globally. Our only uncertainty comes around trying this near the ocean with high levels of salty corrosive wet air. We know it would negatively affect the servers at some point but the question is how quickly and is it within our refresh timetable. During a trip to ASIA last week I discussed trying a small scale “near the ocean” PoC to test this theory.. Does anyone thing this would add value to your company?

Intel IT Deploys Virtualization - How we did it!

bill.s.sunderland@intel.com — Thu, 24 Apr 2008 17:58:16 GMT

As this is my first blog on this forum, I'd like to introduce myself. My name is Bill Sunderland and I have been working at Intel for 12 years primarily working on Server Hardware Engineering and the last three years of which I have focused my efforts on Program Managing the Virtualization Engineering release for Intel IT. I have recently published a WP demonstrating the methodology used as described below.

Intel IT planned, engineered, and has begun deploying a virtualized business-computing production environment at several data centers, a rollout that will continue through 2008. Our initiative has already confirmed anticipated virtualization benefits such as faster, more automated deployment. We are initially consolidating older servers running applications that are not mission-critical; we see opportunities to achieve 16:1 consolidation ratios.

Click here to read the WP: Implementing Virtualization in a Global Business-Computing Environment

I would be interested in hearing your experiences and/or questions regarding virtualizing IT environments!

The Three R's of the Data Center

tvb@intel.com — Tue, 22 Apr 2008 21:36:16 GMT

My daughter recently brought home from school a photocopy of the lyrics of Jack Johnson’s “The 3 R’s” (from the Curious George soundtrack), which encourage us to “Reduce, Reuse, Recycle”. This struck me as relevant in some of the recent discussions I have been having about Greenwashing in the Data Center.

A fair amount of our data center strategy deals with driving down costs. We’re trying to spend less money to deliver the same or better results. Along the way, we find opportunities to be green. While I would love to have more meetings that start out with the question of “what can we do to help the environment?” rather than “what can we do to cut costs?”, we do talk about both. This is somewhat similar to consumer-oriented eco-efforts, encouraging people to save the world while saving money: unplug electronic devices when they're not being used, replace your appliances with more energy-efficient ones, etc. I don't know of many people or organizations who wouldn't like to spend less money, and when we can help the environment at the same time it's win-win.

Which brings me back to the song lyrics. Our cost-cutting measures tend to be related to at least two of the three “R’s” – reducing what we consume, many times by reusing what we already have. I’ll spend my next few posts exploring this a bit further, giving some specific examples of our cost-savings initiatives that ultimately contribute to a greener data center and IT infrastructure.

Happy Earth Day...

Part II: What if you invested a dollar and it returned 10?

laurie.g.buczek@intel.com — Fri, 11 Apr 2008 16:54:36 GMT

Doug Garday continues his podcast series with part II, which continues the discussion around a heat recovery system to reduce the total cost of ownership. In this podcast, Doug plugs in numbers that show potential energy cost savings.

http://www.podtech.net/home/5079/roi-of-data-center-heat-recovery

To listen to Part I go to What if you invested a dollar and it returned 10?. View the full brief at Data Center Heat Recovery Helps Intel Create Green Facility.

Is the Data Center Green or is it Efficient?

brently.davis@intel.com — Mon, 10 Mar 2008 18:03:32 GMT

I won't go into a long dissertation, but I would like to hear what the masses are thinking about Green or Efficient efforts for the Data Center landscape.

As you all know Green is taking off -- our world is becoming concerned with the legacy we'll leave for our children and their children. I admire that because it identifies how we're a caring nation in the U.S. as well as a compassionate world.

But I believe we're mixing the messages down at the lower levels; In my opinion, "Green" means giving something back to mother earth. It means offsetting your carbon impact by planting trees (as I learned from one of the earlier companies I worked for ), or it means buying energy from alternative means such as wind power -- the direction Intel and other companies are moving towards. Those are green efforts from my point of view. However "Efficiency," is defined by Encarta's North America dictionary as this.. "The ability to do something well or achieve a desired result without wasted energy or effort."

Those are two different directions as I see them, and companies running programs to enable more efficient Data Centers must understand how to correctly identify their approach.

So, what does everyone else thing about Green vs. Efficient?

Share your opinion Greening Data Centers or Make 'em Efficient?

Intel's Data Center Strategy, explained

tvb@intel.com — Wed, 09 Jan 2008 15:36:59 GMT

If you've attended any of the Intel Premier IT Professional events or if you have been following our Data Center blogs, you're no doubt aware that Intel is in the process of transforming our computing and data processing back end. We're moving from a sprawl of resources spread across over a hundred data centers to a much smaller footprint. We've been deploying grid computing and virtual servers to slow the rate of growth of our computing capacity. We're also changing our operations processes, applying disciplines that were originally developed for our factories to improve the way we manage our data centers. A little over a year into the project, some of our team (Uttam Shetty, Alan Ross, Brently Davis, and I) have put out a white paper to summarize our goals, focus areas, and preliminary results. We've uploaded the paper Transforming a Global Data Center Environmentas a resource, which you can read/download.

More on our Data Center Efficiency initiative

tvb@intel.com — Mon, 17 Dec 2007 22:24:00 GMT

I've posted to this blog a few times in the past, but my "day job" (Data Center Efficiency) has kept me away from blogging for longer than I'd like. My colleague Brently and I were recently in Folsom for meetings, and took some time out to talk about some of the key elements of our data center strategy: working horizontally rather than tailoring solutions for a particular business unit, giving transparent access to a global pool of resources, and improving data quality.

http://www.podtech.net/home/4724/virtualization-and-data-center-consolidation

I'll plan to increase my posting frequency, using this blog as a forum to share some of the things that have worked for us (or that haven't worked) as we reduce our data center footprint. I'd like for this to become a conversation rather than a broadcast, so please let me know if you have any questions or would like more details on any specific part of this initiative.

Intel IT 45nm test results

laurie.g.buczek@intel.com — Mon, 12 Nov 2007 17:48:00 GMT

We wanted to determine whether the adoption of the new Intel 45nm process technology could deliver additional benefits to our data center strategy: Improving virtualization performance while reducing power consumption. Intel IT is using server virtualization to substantially reduce costs associated with underutilized servers such as higher capital, support, maintenance,power and cooling. Watch this video for the recent testing parameters and results found by Rob Carpenter.

For the full details behind the tests and results, read the white paper 45nm Quad-Core Processors for Energy Efficiency

Let us know how you think the 45nm processors would benefit your data center strategy.

Additional white papers on testing we did on the 45nm processors:

Chip Design using 45nm Quad-Core Xeon Processors

Accelerating EDA Application Performance with 45nm Quad-Core Processors

To read the predecessor to the 45nm testing done on the 65nm quad-core, see Rob's blog with video white paper

The specified blog post was not found.

The full paperComparing Multi-Core Processors for Server Virtualization

Lastly, a tribute to Rob Carpenter. Without his work, none of this would be possible: Fond Stories Over Coffee: The Passing of a Friend

Intel IT Data Center Efficiency Initiative - Going Green

brently.davis@intel.com — Tue, 30 Oct 2007 14:57:00 GMT

Like most other companies these days, Intel is facing a growing demand for computing resources. As a result, our computing costs are going up along with that demand. All of these issues prompted us to take a hard look at our data center strategy, to see where we could make it more efficient.

Therefore, we’ve launched into a significant undertaking. We’ve started the process to End of Life (EOL) or consolidate our data centers down to just eight strategic locations. This effort is planned to take us eight years, but we’re working to pull this in sooner. This initiative enables us to reduce costs, improve server and storage utilization, create higher density & more energy efficient data centers, and allows us to keep pace with our company’s rapid rate of innovation. The effort could deliver up to $750M in Net Present Value. View my video blog for more information.

I would be interested to hear your comments, questions and have you share with us what your company is doing to drive efficiency within your data centers.