The Intel(r) Dynamic Power Node Manager technology allows setting a power consumption target for a server under load as described in a previous article. This is useful for optimizing the number of servers in a rack when the rack is subject to a power budget.
Higher level software can use this capability to implement sophisticated power management schemes, especially schemes that involve server groups. The range of control authority for servers in the Nehalem generation is significant. The power consumption of a fully loaded server consuming 300 watts can be rolled back by roughly 100 watts. In virtualized utility computing environments additional control authority is possible by migrating the virtual machines out of a host and consolidating them into fewer host. The power consumption of the power capped host now at 200 watts, can be brought down by another 50 watts, to 150 watts.
The reader might ask about the possibility of constantly running servers in capped mode to save energy. Unfortunately capping entails a performance tradeoff. The dynamic is not unlike driving an automobile. The best mileage is obtained by running the vehicle at a 35 MPH constant speed. This is not practical in a freeway where the the prevailing speed is 60 MPH. The vehicle could be rear ended, or perhaps a more mundane motivation, the vehicle driver drives the vehicle at 60 MPH because she wants to get there sooner. Like a server, the lowest fuel consumption in a running vehicle, at least in gallons per hour, is attained when the vehicle is idling. No real work is done with an idling engine, but at least the vehicle can start moving in no time. Continuing with the analogy, turning a server off is equivalent to storing a car in the garage with the engine stopped.
This document provides an example of the performance tradeoff with power capping. Please look in page 5, Figure 2.
The following example illustrates how group power capping works. The plot is a screen capture of the Intel(r) Data Center Manager software managing the power consumption in a cluster of four servers. The four servers are divided in a cluster of two server sub-groups of two servers each, labeled low-priority and high-priority
The light blue band represents the focus of the plot. The focus can be changed with a simple mouse click. The current focus in the figure is the whole rack. Hence the power plot is the aggregated power for all four servers in a rack. If the high priority sub-group were selected, then the power shown would be the power consumed by the two servers in that sub-group. Finally, if a single server is selected, then the power indicated would be the power for that server only.
There are four lines represented in the graph. The top line is the plate power. It represents an upper bound for the server’s power consumption. For this particular group of servers the plate power is 2600 watts. The servers are identical, and hence rated at 2600 / 4 = 650 watts.
The next line down is the derated power. Most servers will not have every memory slot or every hard drive tray populated. The derated power is the data center’s operator guess about the upper bound for power consumption based on the actual configuration the server. The derated power is still a conservative guess, considerably higher than the actual power consumption of the server. As a rule of thumb, it is ~70% of the nameplate. The derated power has been set at 1820 watts for the rack or 455 watts per server.
Finally, the gold line represents the actual power consumed by the server. The dots represent successive samples taken from readings from the instrumented power supplies.
The servers are running at full power using the SPECpower benchmark. The rack is collectively consuming a little less than 1300 watts. At approximately 16:12 a policy is introduced to constrain power consumption to 1200 watts. DCM instructs individual nodes to reduce power consumption by lowering the set points for Node Manager in each node until the collective power consumption reaches the desired target.
When we instructed Data Center Manager to hold a power cap for the group rack (2), it makes an effort to maintain power at that level, in spite of unavoidable disturbances in the system. The source of the disturbances can be internal or external. An internal disturbance can be the server fans switching to a different speed causing a power spike or dip. Workloads in servers go up and down, with a corresponding uptick or dip in the power consumption for that server. An external disturbance could be a change in the feed voltage or an operator action. In fact at T = 16:14 we introduced a severe disturbance: we brought the workload of the bottom server, epieg3urb07 down to idle.
Note that it takes a few seconds for Data Center Manager to react and to reach the original power level. Likewise, when the bottom server is brought to idle, it also pulled back the power consumption for the group. However, the group power went back to the target power consumption after a couple of minutes. If we look at the plot of the individual servers, we can see Data Center Manager at work maintaining the target power.
The figure above captures the behaviors of the individual servers. Note how DCM allocates power to individual nodes yet it maintains a global power cap. When the server at the bottom is suddenly idled, there is a temporary dip in power server consumption for the group, but it soon recovers to the target capped level. Also note that the power not used by the bottom server is reallocated to the remaining three nodes until they get close to the previously unconstrained level.
