Is it accurate to say that reducing power consumption of your server and data center infrastructure used to be way down on the task list…sort of a "nice to do" type of an item. That is generally not the case anymore. Many data centers are just plain out of power and cooling capacity and the cost of building a new data center is often prohibitive.

Would you like to reduce power consumption of your existing server infrastructure? What items should you consider when purchasing new servers? This video provides insight on how to reduce system power when servers are less utilized or sitting at idle. It also covers some things to consider before make new server purchases.

This is part 2 of a 2 part series. Feedback welcomed. Let me know what you think.

Ethernet has been around a long time. It is a highly reliable and trusted means for interconnecting computing nodes, and above that, it has generally been the most commoditize (read: lowest cost) form of interconnect for quite some time. Broad deployment, administrator trust, and low cost have kept Ethernet as the mainstream fabric for LAN traffic for a long time.

However, despite Ethernet's strong connectivity credentials, it still comes up short in certain applications. Ethernet is what is referred to as a ‘best effort' network. This simply means that in the real world, you will generally get pretty good performance (throughput, latency, lack of dropped packets, etc), but from time to time when there is congestion, packet drops and performance degradation can be quite a nuisance. For many applications, this doesn't matter. If you are using email, browsing the web, or transferring files to a shared drive, the only thing you will notice is a decrease in performance, but everything will still ‘work', and transfer properly. For some applications like storage though, this non-deterministic performance is unacceptable. If packets are dropped, or arrive out of order, storage applications have a nasty tendency to hang or crash.

Because of this limitation of the standard, there have been separate fabrics used for Storage Area Networks (SANs) for quite a while. One of the main fabrics developed and used for high performance SANs is known as Fiber Channel. In order to create a Fiber Channel network, a server and storage target need to support a Fiber Channel Host Bus Adapter (FC HBA) to communicate via the Fiber Channel protocol. In addition, the switches that connect the Fiber Channel infrastructure must also be dedicated Fiber Channel switches; a standard Ethernet router cannot be used.

Once in place, this SAN architecture provides a very high performance, high reliability network that is ideal (and required) for high end storage traffic, but it comes at a cost:

1) Fiber Channel HBAs are generally more expensive than their Ethernet counterparts.

2) You have to have a separate fabric in your network which also adds to your infrastructure (switch costs, and cabling costs) as well as complicates IT management.

3) Servers connected to the SAN now need to have an Ethernet adapter AND a Fiber Channel adapter.

The upside to the additional cost and complexity is of course better performance, but the question has always been "Is there a better way?"

I believe there is a better way, and that Fiber Channel over Ethernet (FCoE) (and importantly, the standards in IEEE that are making it possible) seems to be the logical path to solve the issue of performance on lossless performance on Ethernet, while maintaining Ethernets historical core cost advantages.

‘Best Effort' is not good enough:
The bottom line for today's Ethernet is that it simply can't provide the ‘lossless' behavior that storage traffic demands; but this fact is changing. Below I will summarize at a high level some of the standards being developed in IEEE to improve the performance of Ethernet for storage applications, and how they help to mend some of the issues with Ethernet and how that helps to enable FCoE.

Bandwidth Sharing, Priority Flow Control and Pause:
This capability offers a method to assign priorities to different Ethernet traffic classes. From there, when congestion becomes an issue, traffic can be ‘paused' on a per-priority basis; allowing the lower priority traffic to be halted temporarily while keeping the top priority traffic like storage running smoothly. This per-priority pause capability is really the first basic step in allowing Ethernet to provide some ‘QoS like' Layer 2 guarantees.

Congestion Notification (or Backward Congestion Notification):
In addition to simply pausing individual low priority streams of traffic, congestion notification allows for a communication method to go upstream from the node and notify the offending traffic generator to throttle back its traffic and re-route as necessary. This capability is a key to the longer term development of FCoE because with only the pause capability the congestion is really just pushed up a single node in the network. In order to support FCoE storage across multiple nodes in a network, congestion notification is needed.

Shortest Path Bridging:
This capability is really an optimization for inter-node routing that defines the path within the network between switches. Using traditional spanning tree path algorithms will sometimes result in paths in the network that are non-optimal and incompatible with high performance storage traffic. A new algorithm to determine the shortest path between nodes will help to enable both less congestion in the network as well as fast delivery of critical packets for storage.

DCB Capability Exchange Protocol (DCBX):
This capability goes by several different names depending on who you talk to, but essentially what it will provide is the ability for switches on the network to exchange their capability sets with other nodes of the network. This allows for each switch to understand what others switches near it can use the Congestion Notification, Flow Control, or other features need to support this ‘Lossless Ethernet' capability.

While the list above is not meant to be all inclusive of all the new IEEE development under way for this new ‘Lossless Ethernet' initiative, it should provide a good overview of the general push taking place and how the goal of getting to near lossless performance is going to be accomplished.

Weren't we talking about Fiber Channel?
Astute users will realize that I haven't really addressed the Fiber Channel piece of this. The above features I described only allow for Ethernet to carry certain kinds of traffic (like Fiber Channel) that require very high reliability and performance; but how do you get the Fiber Channel data onto an Ethernet frame?

In today's environment, a Fiber channel initiator on a Server system will place Fiber Channel data onto an FC HBA to send over the SAN to a storage target. All of this data is transmitted over a fiber channel network. Under the FCoE model, what you will need is a Server system that has an FCoE initiator, and on the target side, the switch connected to the target must be able to convert the data from storage target and encapsulate it into Ethernet. Beyond that, the data is transmitted over the Ethernet fabric as normal, but the features that I described above allow for the performance of Ethernet to allow a Fiber Channel application stack to function properly.

This is certainly a capability that Intel has been supportive of. Ethernet is a critical piece of the computing platform, and FCoE provides a potential improvement for datacenter and storage network design. By consolidating the Fiber Channel data onto a single Ethernet wire, end user IT houses can also see several benefits:

1) Reduced the need for two physical network cards in each server. Now, a single NIC will connect to the SAN and to the normal TCP/IP data network.

2) Along with the consolidation in network cards, you also save in terms of cabling. One 10 Gigabit link can replace the old Fiber Channel fiber link and Ethernet links.

3) Reduces power consumption and cooling

4) The commoditized and low cost nature of Ethernet provides additional benefit by converging system I/O onto what will likely be the lowest cost interface over the coming years; 10 Gigabit.

In summary, FCoE may be in its infancy, but the standards in final, or in process. Products are available today, and the value proposition in here. Further performance improvements and cost reductions and the proliferation of 10 Gigabit networks, as well as more choices in the future, will only further the support and interest in Fiber Channel over Ethernet in datacenter SAN applications.

~ Ben Hacker

Links for further information:
http://ieee802.org/
http://www.open-fcoe.org/

Last week, the first part this video series focused on the energy efficiency benefits of 45nm. The 2nd part of this video (below) is focused on the benefits of 45nm for virtualization and the intel processor roadmap including what's next in 45nm processor technology - the Dunnington and Nehalem-EP products

Is this information useful to you? why or why not?

Chris

Following a recent interview I conducted with the Register on a related subject, I was asked to talk more about Intel's current 45nm technology and our roadmap for new technology later this year. Join me in a two part video series where I discuss 45nm and beyond.

Part 1 (below) discusses the technology and benefits that 45nm xeon processors deliver for IT today.

Tune in next week to hear Part 2 - what we have planned for future enhancements to today's xeon products - the Nehalem Processor and Intel QuickPath architecture.

Chris

After coming back from IDF a couple weeks ago, I've had some time to go through the mountains of online material, presentations mostly and a few interesting videos. This video is from Pat Gelsinger's keynote address and features Mendel Rosenblum from VMware. Pat and Mendel discuss new technologies in virtualization and demonstrate "Flex Migration", just hit the play button below to view...


This is very interesting for those IT shops with multiple legacy platforms and new generation servers coming online. We will have more discussion on this topic in the future, and so in the meantime, let us know if you have questions on how this could benefit your datacenter.

While Intel is certainly most widely known for manufacturing our extremely complicated CPUs that are the brain of many computing platforms worldwide; there are several other products and technologies that people at Intel have been involved in for many years which are critical to computing environments everywhere. As a person who has been working in various networking and manageability roles at Intel since 2001, I'd like to take a little time to focus on Intel's history in the Ethernet market since its inception more than 25 years ago and focus a little on where the market might be going in the future.

Below is an image that tries to capture the key highlights of Intel's specific involvement in the Ethernet market over the last 3 decades:

As you can see the Ethernet market has come along way from clunky multi-chip 10Mpbs solutions from more than 25 years ago all the way to Quad Port Gigabit and Dual Port 10 Gigabit designs that are prevalent today.

Moving into the future the Ethernet market is growing increasingly more complicated by the year with new capabilities and features targeted specifically to support server virtualization, infrastructure convergence, enhanced storage technologies, and the continued importance of power efficiency of the overall compute infrastructure. Each of these innovations and changes will have a big impact on the overall structure and design what servers and datacenters will look like in the future. My colleague Ken Lloyd gave his thoughts on how 10 Gigabit technologies will provide I/O convergence and overall cost savings for computer networks in the future and there are clearly lots of interesting things going on right now.

Over the next several months I plan to try to go more in depth on many of the exciting developments taking place in the Ethernet market and to hopefully shed some light on some of the changes that are coming our way.
Stay tuned in the coming weeks!
- Ben

Continuing on the theme of measuring Data Centre efficiency - power consumption of the facilities and IT load are only one element albeit a large one - that contributes to the overall efficiency of a data centre. Ultimately a DC has to deliver useful workload and the amount of workload that can be achieved within a given physical DC is an increasing challenge. Lowering server power and increasing the cooling effectiveness of a DC are one of several ways to enable more equipment to be installed into an existing facility.

General consensus seems to be that the servers in many data centres do not always run a maximum utilisation - many are in the 10-15% utilisation range. This results from many IT shops following a policy of hosting one workload ( application ) per server and sizing the server to support worse case usage of that workload - this leads to low average utilisation of the servers. There are several approaches that can be taken to increasing the server utilisation

Consolidating several applications onto the same server that have different mixes of utilisation - this is not perfect as a problem on one application could impact the others on that server causing significant business impact

Deploying virtualisation within the DC - this enables multiple OS/App instances to be run on the same server. There are multiple benefits here in that the server utilisation increases whilst the number of servers could potentially be decreased so reducing the overall electrical power consumption of the DC and consequently the utility bill. Another aspect of virtualisation is that to achieve the highest levels of consolidation it is best to deploy the latest generation high perf/low power servers, this can result in the removal of many older generation high power servers from the Data Centre and the deployment of a smaller number of newer more power efficient servers

There are circumstances where virtualisation may not be appropriate and it is necesseary to retain one workload per server - in this case an increase in the workload capacity of a DC can be achieved by replacement of older smaller servers with the latest generation high performance servers - this can enable the workload capacity of a DC to be significantly increased without building a new DC, again the side benefit here is that latest generation servers consume less power than the older servers they are replacing.

There are many different ways in which the workload capacity ( and hence utilisation ) of a DC can be increased , with care most can also result in a reduction in the electrical power consumed by the DC.

Given the right tools the utilisation of servers within a DC is 'relatively' easy to measure, so this element of DC effectiveness can be quantified. There is another major element that I believe contributes to the effectiveness of a DC - that is the processes that are in place to manage the DC and hence the way a DC can respond to the new challenges placed on it by a business unit. Gartner have an infrastructure maturity model that is useful to try and quantify how effective a DC is in responding to business needs and looks at responsiveness, Service Level Agreements, IT processes etc. Currently I do not believe many DC managers are measuring how effective their DC in terms of process and when asked to judge where they sit within a model like Gartner's many IT managers will judge themselves more efficient than they really are.

Are there other areas that contribute to the efficiency of a DC - I would be interested in your feedback.