Several months ago, our flagship 10 Gigabit Ethernet (10GbE) server adapters underwent a big change. We retired the Intel® Ethernet 10 Gigabit Server Adapter family name and replaced it with Intel® Ethernet 10 Gigabit Converged Network Adapter family. The adapters themselves didn’t change a bit; they’re the same reliable products that have led us to the top spot among 10GbE adapter suppliers[1].

Why did we rename such a successful product line? Quite simply, Converged Network Adapter (CNA) is a much more accurate description of our 10GbE adapters and the features they offer. As IT organizations upgrade their data center networks, we want to make sure they know that these Intel Ethernet adapters meet not only their Ethernet networking needs, but also their converged networking needs.

For you non-networking folks, a CNA is a 10GbE adapter that supports standard LAN traffic as well as Fibre Channel over Ethernet (FCoE) traffic and iSCSI traffic. Traditional LANs and Fibre Channel (FC) storage area networks (SANs) use completely separate network infrastructures, requiring storage-specific network adapters, switches, and cabling. Converged or unified networks allow LAN and SAN traffic to use or even share a 10GbE fabric, greatly simplifying the infrastructure. CNAs connect servers to these converged networks and eliminate the need for separate, dedicated storage network adapters.

So with all of these great benefits, why didn’t call our adapters CNAs from the start? Answering that one requires a history lesson.

Work on the FCoE standard began in 2007, and the first CNAs appeared several months later. For these early designs, the traditional storage adapter vendors modified their FC host bus adapter (HBA) designs to include the Intel^® 82598 10 Gigabit Ethernet Controller alongside their proprietary FC processors. With the FCoE standard still in draft form and a thin ecosystem, these first-generation adapters were little more than proof of concept vehicles.

Meanwhile, we at Intel were working to enable FCoE on the Intel 82598 10 Gigabit Ethernet Controller – the same controller that provided the Ethernet functionality for those early CNAs. We realized, however, that enabling FCoE on that controller (and adapters based on it) would require releasing our own FCoE software stack. Introducing yet another proprietary FCoE solution into the market would have made life harder for IT, and that’s the last thing we wanted to do. So we launched our new adapters as 10GbE adapters, not CNAs, and set about making FCoE easier for IT to deploy.

In 2008, Intel founded the Open FCoE project and released our FCoE initiator code to the open source community. Our goal was to get the Open FCoE initiator integrated into the Linux* kernel and help accelerate the adoption of FCoE. Any adapter vendor could use that native support to develop a CNA, giving customers more hardware options and allowing them to use a common set of OS-based management tools. The industry had gone through a similar process with the successful integration of iSCSI, another storage protocol, in every major OS and hypervisor.

In March 2009, after modifications from the Linux community, the Open FCoE initiator was integrated into version 2.6.29 of the Linux kernel and soon found its way into major distributions, including Red Hat and SLES.

In early 2011, we announced that our newest 10 Gigabit Controller, the Intel® 82599 10 Gigabit Ethernet Controller and the integrated FCoE initiators in Linux and Windows had been qualified by EMC, Cisco, and NetApp. The Intel® Ethernet Server Adapter X520 family, which is powered by the Intel 82599 10 Gigabit Ethernet Controller, was included in this announcement.

And in August 2011, VMware announced Open FCoE integration as part of the vSphere 5.0 launch. With that launch, the Intel Ethernet Server Adapter X520 and Intel 82599 10 Gigabit Ethernet Controller had FCoE support in every major operating system and hypervisor. We felt it was important for customers to understand that Intel Ethernet 10 Gigabit Server Adapters were full CNAs, so later that year, we decided to rename them, and the Intel Ethernet Converged Network Adapter family was born.

Our 10GbE CNAs are the industry’s top-selling 10GbE adapters. Earlier this year, we expanded the family by adding the Intel® Ethernet Converged Network Adapter X540, our fourth-generation 10GBASE-T adapter. Through our Open FCoE efforts, we have given the industry another option for enabling FCoE – an option that doesn’t depend on proprietary hardware and software, can use standard OS-based tools, and scales with advancements in server architectures.

It has been a long journey getting to this point, but sometimes it takes time to do things right.

If you’d like to learn more about the advantages of Open-FCoE-based solutions, check out this blog post on Open FCoE in VMware vSphere.

[1] Crehan Research, Server-class Adapter and LOM, 2Q12

The Summer IT industry event season has kicked into high gear, and this week I’m at Cisco Live! 2012 in San Diego. Networking products and technologies are, of course, one of the primary focus areas of the show, so it’s no surprise that Cisco’s announcement earlier this week included a number of new networking products. One particularly significant product is the Nexus 5596T switch, a new data center switch and the first 10GBASE-T member of the Nexus 5000 family.

If you’ve read some of my previous posts, you know that 10GBASE-T is 10 Gigabit Ethernet over twisted-pair copper cabling – the stuff that’s installed in nearly every data center today – and that it uses the familiar RJ-45 connector. 10GBASE-T has been a big topic for us here at Intel, especially with the launch of the Intel® Ethernet Controller X540 in March.

Need proof? Here’s a word cloud of my posts for the last year. Other than the common words like “Intel,” “Ethernet,” and “10GbE,” the term I used the most was “10GBASE-T.” See it there on the left?

But I digress. (And I drop in the occasional overused phrase.)

Kaartik Viswanath, product manager for the Nexus 5000 family, was kind enough to take a few minutes to answer some questions about the new switch and Cisco’s views on 10GBASE-T.

BY: Kaartik, thanks for taking the time to answer a few questions today. Tell me about the Nexus 5596T switch and 10GBASE-T module that Cisco announced yesterday.

KV: Sure. We’re very excited about the new Nexus 5596T switch. It’s the first 10GBASE-T member of the Nexus 5000 family, and it’s coming at the perfect time, with 10GBASE-T LOM (LAN on motherboard) connections now being integrated onto mainstream server motherboards. LOM integration will help drive 10GbE adoption, and all those new 10GBASE-T ports need a high-performance, high-port-density switch to connect to. The Nexus 5596T has 32 fixed 10GBASE-T ports, and through the addition of the new 12-port 10GBASE-T Cisco Generic Expansion Module (GEM), it can support up to 68 total 10GBASE-T ports in a two-RU (rack unit) design. Plus, customers can deploy any of the existing GEMs in any of the Nexus 5596T’s three GEM slots.

The Nexus 5596T also includes 16 fixed SFP+ ports, which customers can use to connect to aggregation switches, servers, or Nexus 2000 Fabric Extenders using optical fiber or direct attach copper connections. With the Nexus 5596T switch, our customers have the flexibility to deploy both 1/10GBASE-T Ethernet on Copper and FC/FCoE/Ethernet on SFP+ ports on the same chassis.

BY: Are you hearing a lot of interest in 10GBASE-T from your customers?

KV: Yes, definitely, and I think there are a couple of major reasons for that. First, 10GBASE-T offers the easiest path for folks looking to migrate from One Gigabit Ethernet (GbE). 10GBASE-T uses the same twisted-pair copper cabling and RJ-45 connectors as existing GbE networks, and it’s backwards-compatible with all the 1000BASE-T products out there today. That means you can replace your existing 1000BASE-T switch with a Nexus 5596T and connect to both 10GBASE-T and 1000BASE-T server connections. And as you’re ready, you can upgrade servers to 10GBASE-T.

I think the other big reason 10GBASE-T is so appealing is the deployment flexibility it offers; 100 meters of reach is sufficient for the vast majority of data center deployments, whether it’s top-of-rack, middle-of-row, or end-of-row.  Plus, twisted-pair copper cabling is much more cost-effective than the fiber or direct-attach copper cabling that’s used in the majority of 10GbE deployments today.

BY: Cisco and Intel both support multiple 10GbE interfaces in their products. How do you see 10GBASE-T fitting into the mix?

KV: We’ll support whichever interfaces our customers want to use. However, there are some general guidelines that most folks seem to be following. For longer distances – over 100 meters – SFP+ optical connections are really the only choice, given their longer reach.  But fiber costs really don’t lend themselves to broad deployment. Today, most 10GbE deployments use the top of rack model, where servers connect to an in-rack switch using SFP+ direct attach copper (DAC) connections. DAC reach is only seven meters, but that’s plenty for any intra-rack connections.

10GBASE-T hits sort of a sweet spot because of its distance capabilities. It can connect switches to servers in top of rack deployments, with cables that are less expensive than SFP+ DAC, or it can be used for the longer runs where fiber is being used today – up to 100 meters, of course.

There are cases where SFP+ has some advantages, particularly for latency-sensitive applications or if the customers are sensitive to power consumption, but when it comes to deployment flexibility, costs, and ease of implementation, 10GBASE-T is well-positioned as the interface of choice for broad adoption.

BY: How about Fibre Channel over Ethernet? Does the Nexus 5596T switch support FCoE over 10GBASE-T?

KV: Great question. FCoE is a key ingredient in Cisco’s unified fabric vision, and it’s supported in our 10 Gigabit Nexus and UCS product lines. The Nexus 5596T hardware is FCoE-capable like all of our Nexus 5000 switches, and we’re working on FCoE characterization in our labs. We’ve been working closely with Intel to verify FCoE interoperability with the Intel Ethernet Controller X540.

There’s been a fair amount of discussion in the industry around whether 10GBASE-T is a suitable fabric for FCoE. Our collaboration with our ecosystem partners, including Intel, network cable vendors, and storage vendors, will help ensure there aren’t any issues before we enable the feature on the Nexus 5596T. Assuming everything goes well, we’ll also enable FCoE over 10GBASE-T in our 12-port 10GBASE-T GEM Module as well as our fabric extender line with the upcoming Nexus 2232TM-E Fabric Extender.

BY: Kaartik, a couple of quick final questions for you. Before you joined the Nexus team, you worked on the campus network side of Cisco, correct?

KV: Yes, that’s right. I was the Product Manager in the Unified Access Business Unit, managing the Fixed 10/100 business.

BY: How is the data center networking world different than the campus networking world?

KV: One thing that stands out is the faster speeds of the interconnects in data center switching products. In campus networks, the vast majority of connections are at Gigabit or Fast Ethernet (100Mbps) speeds. Our Nexus product line, by contrast, has switches with 96 10 Gigabit ports and 40 Gigabit and 100 Gigabit uplinks. So the speed of individual links is greater, but a campus network typically connects many more machines than a data center network, as there are more client PCs than there are servers.

Another big difference is the technologies that are supported in each class of product. There’s certainly some overlap, but technologies like Fibre Channel over Ethernet, Data Center Bridging, I/O virtualization – those are mostly confined to the data center world. Similarly, technologies like Power over Ethernet Plus and Universal Power over Ethernet today are more prevalent in campus access type of deployments and are not so common in the data center world.

BY: Thanks for taking the time to chat today, Kaartik. We’re looking forward to continuing our work with Cisco.

KV: No problem. I’m looking forward to it, too.

We at Intel have been talking about 10GBASE-T for a long time now, and it’s great to see the ecosystem continuing to grow with new products like the Nexus 5596T switch. I’d like to thank Kaartik for taking the time to answer these questions for us.

For the latest, follow us on Twitter: @IntelEthernet.

Coming into today’s announcement of the Intel® Xeon® processor E5-2600/1600 product family, Intel has been sharing some striking statistics for computing trends that point to a need to improve server networking, bandwidth, and overall performance.

According to Intel projections, by 2015, the number of networked devices will be in the neighborhood of 15 billion, with more than 3 billion users. Worldwide mobile data traffic alone is expected to have increased 18-fold by that time, driven by a jump in streamed content, mobile connections, enhanced computing of devices, faster mobile speeds and the proliferation of mobile video, according to the recently released Cisco Visual Networking Index (VNI) Global Mobile Data Traffic Forecast for 2011 to 2016.

All of this means IT departments will need to increase data center bandwidth and enable faster data flow to servers if they want to meet the needs of demanding applications and avoid bandwidth meltdowns. Fortunately, a number of I/O innovations are here to help with that endeavor.

As part of the Intel Xeon processor E5 product family launch, Intel announced two significant I/O innovations: the Intel® Ethernet Controller X540 and an important server platform feature, Intel® Data Direct I/O Technology.

The Intel Ethernet Controller X540 is the industry’s first fully integrated 10GBASE-T controller and was designed specifically for low-cost, low-power 10 Gigabit Ethernet (10GbE) LAN on motherboard (LOM) and converged network adapter (CNA) designs. I’ve been dropping hints about this product for a while now, and I’m thrilled to say that 10 Gigabit LOM is finally here.

The Intel Ethernet Controller X540

With the Intel Ethernet Controller X540, Intel is delivering on its commitment to drive down the costs of 10GbE. We’ve ditched two-chip 10GBASE-T designs of the past in favor of integrating the media access controller (MAC) and physical layer (PHY) controller into a single chip. The result is a dual-port 10GBASE-T controller that’s not only cost-effective, but also energy-efficient and small enough to be included on mainstream server motherboards. Several server OEMs are already lined up to offer Intel Ethernet Controller X540-based LOM connections for their Intel Xeon processor E5-2600 product family-based servers. This new controller also powers the Intel® Ethernet Converged Network Adapter X540, a PCI Express* adapter for rack and tower servers.

The Intel Ethernet Converged Network Adapter X540-T2

10GBASE-T solutions based on the Intel Ethernet Controller X540 are backward-compatible with Gigabit Ethernet networks, giving customers an easy upgrade path to 10GbE by allowing them to deploy 10GBASE-T adapters in servers today and add switches when they’re ready. Another major benefit of 10GBASE-T is its ability to use the cost-effective, twisted-pair copper cabling that most data centers use today, meaning an expensive cabling upgrade won’t be required, and 10GBASE-T’s support for cable distances of up to 100 meters provides flexibility for top-of-rack or end-of-row deployments in data centers.

The Intel Ethernet Controller X540, like other members of the 10 Gigabit Intel Ethernet product family, supports advanced I/O virtualization and unified networking, including NFS, iSCSI and Fibre Channel over Ethernet (FCoE). Intel Ethernet controllers and converged network adapters are also optimized for new I/O advancements in the Intel Xeon processor E5 product family. One key related feature that should generate some attention is Intel® Data Direct I/O Technology (Intel® DDIO).

Intel Data Direct I/O Technology

Intel DDIO allows Intel Ethernet controllers and adapters to talk directly to processor cache, avoiding the numerous memory transactions required by the previous-generation systems. Eliminating these trips to and from system memory delivers improvements in server bandwidth, power consumption, and latency.

Intel DDIO is a key component of Intel® Integrated I/O, which integrates the PCI Express* controller directly into the processor to further reduce I/O latency. The level of improvement enabled by these technologies when combined with Intel Ethernet 10 Gigabit Controllers can be pretty extraordinary. Together they can deliver more than three times the bandwidth of a previous generation server, and tests in Intel labs have shown even higher peak performance levels[1].

There’s much more to say about the Intel Ethernet Controller X540 and Intel DDIO, so I’m going to spend my next couple of posts doing just that. Next week I’ll take a closer look at the Intel Ethernet Controller X540 and 10GBASE-T, and the following week, I will dig deeper into Intel DDIO and Intel Integrated I/O.

For the latest updates, follow us on Twitter: @IntelEthernet

[1] (I/O Bandwidth) Source: Intel internal measurements of maximum achievable I/O R/W bandwidth (512B transactions, 50% reads, 50% writes) comparing Intel® Xeon® processor E5-2680 based platform with 64 lanes of PCIe* 3.0 (66 GB/s) vs. Intel® Xeon® processor X5670 based platform with 32 lanes of PCIe* 2.0 (18 GB/s). Baseline Configuration: Green City system with two Intel® Xeon® processor X5670 (2.93 GHz, 6C), 24GB memory @ 1333, 4 x8 Intel internal PCIe* 2.0 test cards. New Configuration: Rose City system with two Intel® Xeon processor E5-2680 (2.7GHz, 8C), 64GB memory @1600 MHz, 2 x16 Intel internal PCIe* 3.0 test cards on each node (all traffic sent to local nodes).

*Please not a version of this blog first apeared as an Intel industry perspective on Data Center Knowledge as Tips for Simplifying Your Cloud Network.

"Ethernet is the backbone of the Cloud."

Bold statement? Not at all. Any data center, cloud or otherwise, depends on its Ethernet network to allow servers, storage systems, and other devices to talk to each other. No network means no data center. Today, as IT departments prepare to deploy internal cloud environments, it’s important for them to consider how network infrastructure choices will impact their cloud’s ability to meet its service level agreements (SLAs). Terms commonly used to describe cloud computing capabilities, such as agility, flexibility, and scalability, should absolutely apply to the underlying network as well.

With that in mind, I’d like to take a look at some recommendations for simplifying a private cloud network. You can consider this post a sort of CliffsNotes* version of a white paper we completed recently; you’ll get a basic idea of what’s going on, but you’ll need to read the full piece to get all the details. It’s a great paper, and I recommend reading it.

Consolidate Ports and Cables

Most cloud environments are heavily virtualized, and virtualization has been a big driver of increasing server bandwidth needs. Today it’s common to see virtualized servers sporting eight or more Gigabit Ethernet (GbE) ports. That, of course, means a lot of cabling, network adapters, and switch ports. Consolidating the traffic of those GbE connections onto just a couple of 10 Gigabit Ethernet (10GbE) connections simplifies network connectivity while lowering equipment costs, reducing the number of possible failure points, and increasing the total amount of bandwidth available to the server.

With 10 Gigabit Ethernet, you can consolidate this . . .

. . . down to this.

Converge Data and Storage onto Ethernet Fabrics

10GbE’s support for storage technologies, such as iSCSI and Fibre Channel over Ethernet (FCoE), takes network consolidation a step further by converging storage traffic onto Ethernet. Doing so eliminates the need for storage-specific server adapters and infrastructure equipment. IT organizations can combine LAN and SAN traffic onto a single network or maintain a separate Ethernet-based storage network. Either way, they’ve made it easier and more cost-effective to connect servers to network storage systems, reduced equipment costs, and increased network simplicity.

Maximize I/O Virtualization Performance and Flexibility

Once you have a 10GbE unified network connecting your cloud resources, you need to make sure you’re using those big pipes effectively. Physical servers can host many virtual machines (VMs), and it’s important to make sure bandwidth is allocated and balanced properly between those VMs. There are different methods for dividing a 10GbE port into smaller, virtual pipes, but they’re not all created equal. Some methods allow these virtual functions to scale and use the available bandwidth of the 10GbE connection as needed, while others assign static bandwidth amounts per virtual function, limiting elasticity and leaving unused capacity in critical situations.

Enable a Solution That Works with Multiple Hypervisors

It’s likely that most cloud deployments will consist of hardware and software, including hypervisors, from multiple vendors. Different hypervisors take different approaches to I/O virtualization—and it’s important that network solutions optimize I/O performance for those various software platforms; inconsistent throughput in a heterogeneous environment could result in bottlenecks that impact the delivery of services. Intel Virtualization Technology for Connectivity, included in Intel Ethernet server adapters and controllers, includes Virtual Machine Device Queues (VMDq) and support for Single Root I/O Virtualization (SR-IOV), to improve network performance across all major hypervisors.

Utilize Quality of Service for Multi-Tenant Networking

Like a public cloud, a private cloud provides services to many different clients, ranging from internal business units or departments within the company to customers, and they all have performance expectations of the cloud.  Quality of Service (QoS) helps ensure that clients’ requirements are met.

Technologies are available that provide QoS on the network and within a physical server. QoS between devices on the network is delivered by Data Center Bridging (DCB), a set of standards that defines how bandwidth is allocated to specific traffic classes and how those policies are enforced. For traffic between virtual machines in a server, QoS can be controlled in either hardware or software, depending on the hypervisor. When choosing a network adapter for your server, support for these types of QoS should be taken into consideration.

Again, keep in mind that these are high-level looks at the recommendations. The white paper I’m summarizing goes into much greater detail on the hows and whys behind each recommendation. If you’re thinking about deploying a cloud data center, it’s highly recommended reading.

Follow @IntelEthernet for the latest updates on Intel Ethernet technologies.

There are always plenty of new terms looking to join the data center vernacular. You’re familiar with many that have taken root over the past several years: virtualization, cloud, consolidation, mission critical... the list goes on and on. A hot term you’ve probably heard recently is Big Data. There’s a lot of buzz around this one, as evidenced by the many companies introducing products designed to meet the challenges of Big Data. In this post, I’m going to take a look at Big Data and see what its implications could be for data center networks.

First things first: what is Big Data? The specifics may vary depending whom you ask, but the basic idea is consistent: Big Data means large sets of data that are difficult to manage (analyze, search, etc.) using traditional methods. Of course, there’s more to it than that but that’s a decent enough answer to make you sound semi-intelligent at a cocktail party. Unless it’s a Big Data cocktail party.

A logical follow-up question is “why is Big Data so...big?” The main cause for these massive data sets is the explosive growth in unstructured data over the past several years. Digital images, audio and video files, e-mail messages, Word or PowerPoint files – they’re all examples of unstructured data, and they’re all increasing at a dizzying rate. Need a first-hand example? Think about your home PC. How many more digital photos, MP3s, and video files are on your hard drive compared to a few years ago? Tons, right? Now imagine that growth on an Enterprise scale, where thousands of employees are each saving gigabytes worth of presentations, spreadsheets, e-mails, images, and other files. That’s a lot of data, and it’s easy to see how searching, visualizing, and otherwise analyzing it can be difficult.

[Structured data, for those of you wondering, is data organized in an identifiable structure. Examples include databases or data within a spreadsheet, where information is grouped into columns and rows.]

So what to do? There’s no shortage of solutions billed as the answer to the Big Data problem. Let’s take a look at one that’s getting a lot of attention these days: Hadoop.

Hadoop is an open source software platform used for distributed processing of vast amounts of data. A Hadoop deployment divides files and applications into smaller pieces and distributes them across compute nodes in the Hadoop cluster. This distribution of files and applications makes it easier and faster to process the data, because multiple processors are working in parallel on common tasks.

Let’s take a quick look at how it works.

Two major software components comprise Hadoop, the Hadoop Distributed File System (HDFS) and the MapReduce engine.

• HDFS runs across the cluster and facilitates the storage of portions of larger files on various nodes in the cluster. It also provides redundancy and enables faster transactions by placing a duplicate of each piece of a file elsewhere in the cluster.
• The MapReduce engine divides applications into small fragments, which are then run on nodes in the cluster. The MapReduce engine attempts to place each application fragment on the node that contains the data it needs, or at least as close to that node as possible, reducing network traffic.

So who’s using Hadoop today and why? You’ve heard of the big ones – Yahoo!, Facebook, Amazon, Netflix, eBay. The common thread? Massive amounts of data that need to be searched, grouped, presented, or otherwise analyzed. Hadoop allows organizations to handle these tasks at lower costs and on easily scalable clusters. Many of these companies have built custom applications that run on top of HDFS to meet their specific needs, and there’s a growing ecosystem of vendors selling applications, utilities, and modified file systems for Hadoop. If you’re a Hadoop fan, the future looks bright.

What are the network implications of Hadoop and other distributed systems looking to tackle Big Data? Ethernet is used in many server clusters today, and we think it will continue to grow in these types of deployments, as Ethernet's ubiquity makes it easy to connect these environments without using specialized cluster fabric devices. The same network adapters, switches, and cabling that are being used for data center servers can be used for distributed system clusters, simplifying equipment needs and management. And while Hadoop is designed to run on commodity servers, hardware components, including Ethernet adapters, can make a difference in performance. Dr. Dhabaleswar Panda and his colleagues at Ohio State University have published a research paper in which they demonstrate that 10GbE makes a big difference when combined with an SSD in an unmodified Hadoop environment. Results such as these have infrastructure equipment vendors taking notice. In its recent Data Center Fabric announcement, for example, Cisco introduced a new switch fabric extender aimed squarely at Big Data environments. You can expect to see more of this as distributed system deployments continue to grow.

Big Data isn’t going away. It’s going to keep getting bigger. We’ll see more products, both hardware and software, that will be designed to make your big data experience easier, more manageable, and more efficient. Many will use a distributed model like Hadoop, so network infrastructure will be a critical consideration.

So, here are some questions for you, dear reader: Have you deployed a Hadoop cluster or are you planning to do so soon? What network considerations did you take into account as you planned your cluster?

We’d love to hear your thoughts.

Follow us on Twitter: @IntelEthernet

Hello again. Welcome to my second post from Cisco Live! 2011. It has been a great show so far, loaded with activity at the Intel booth and a lot of exciting showcases of new technologies. With today’s post, I’d like to talk about a particular technology that seems to come up at every show: The 10GBASE-T ethernet standard.

In previous posts, I’ve mentioned that Intel’s “Twinville” 10GBASE-T controller will be the industry’s first single-chip 10GBASE-T controller and will power the 10GBASE-T LAN on motherboard (LOM) connections for mainstream servers later this year. This integration, along with 10GBASE-T’s backwards compatibility with Gigabit Ethernet and support for already deployed copper cabling, leads us to believe that 10GBASE-T will ultimately be the dominant 10GbE interface in terms of ports shipped.

On Tuesday, Cisco nudged all of us a bit closer to that reality by announcing the Nexus 2232TM Fabric Extender, its first Nexus platform that supports 10GBASE-T. Let’s take a closer look.

Cisco Nexus 2000 Fabric Extenders behave as remote line cards for Nexus parent switches. They connect to the parent switch via 10GbE fiber uplinks and are centrally managed by that parent, creating a distributed modular switch – distributed because the parent switch and fabric extenders are not physically constrained by a chassis, and modular because additional fabric extenders can be added to increase switch capacity.

The Nexus 2232TM has 32 10GBASE-T ports as well as eight SFP+ ports for connecting to its parent, in this case a Nexus 5000 series switch. With that many 10GBASE-T ports, the Nexus 2232TM can connect to every server in a typical rack. Integration of 10GBASE-T LAN on motherboard (LOM) ports on these servers will drive adoption of 10GbE and 10GBASE-T over the next few years. Since 10GBASE-T is backwards-compatible with existing Gigabit Ethernet (GbE) equipment, IT departments can upgrade to 10GBASE-T all at once or even server-by-server and use the same fabric extender for all of the servers.

Here at Cisco Live, the Nexus 2232TM and Twinville-based 10GBASE-T adapters are key ingredients in a joint demo from Intel, Cisco, and Panduit. There’s quite a bit more to the demo (iSCSI, FCoE, live migration), but I don’t really have space to cover it here. I’ll see if I can post a short video of the demo in the near future.

Earlier this week, I spoke with Aurelie Fonteny, product manager for the Nexus 2232TM, who kindly answered a handful of questions.

BY: Aurelie, how does Cisco see 10GBASE-T growing over the next few years?

The past few years have been marked with a trend towards 10 Gigabit Ethernet at the server access. Virtualization, consolidation of multiple 1Gig cables, price, and higher performance CPUs have all been drivers towards that trend. 10GBASE-T will accelerate that trend with the additional flexibility of connectivity options. Ultimately, LOM integration will drive the exponential volumes of 10GBASE-T platforms.

BY: What benefits does the Nexus 2232TM offer to your customers?

AF: Cisco is excited to introduce the Nexus 2232TM, the first 10GBASE-T product in the Nexus Family of Data Center switches. In total, 768 1/10GBASE-T ports can be managed from one single point of management. As such, the Nexus 2232TM combines the benefits of the FEX architecture together with the benefits of 10GBASE-T: 10G consolidation, 1G to 10G migration simplicity, and cabling simplicity.

How do you anticipate customers deploying this product vs. the SFP+ version of the Nexus 2232?

AF: The Nexus 2232PP (the fiber version with Direct attach copper options) and the Nexus 2232TM share the same architecture and have the same number of host interfaces and network interfaces. The choice between the two platforms will be a trade-off between power, latency, cabling type, price, and FCoE support (not supported on Nexus 2232TM at FCS).

BY: We’ve heard folks say that 10GbE is too expensive. Can you talk about pricing for the Nexus 2232TM?

AF: The Nexus 2232TM is priced at a small premium over the Nexus 2232PP (fiber version with Direct Attach copper options). Total Cost of Ownership includes not only a point product but also cabling, server adapter, and power. As such, both solutions with 10G servers attached via direct attach copper (Twinax) from server to the Fabric Extender or 10G servers attached via 10GBASE-T from server to the Fabric Extender are about the same price today. Some of the decision factors would be requirements for FCoE consolidation, distances between servers and network access, cabling preference and existing cabling structure, and mix of 1G and 10G ports required at Top of Rack.

BY: How have Cisco and Intel’s work together advanced 10GBASE-T?

AF: Our strong collaboration with Intel from the early stages of 10GBASE-T on Catalyst and Nexus platforms has been critical to the ecosystem interoperability at the server access and to the overall high level quality achieved in 10GBASE-T product integration.

We are working with Cisco and Panduit on a white paper that includes key deployment models for 10GBASE-T in the data center. I will post a short blog when the paper is available.

Follow us on Twitter for the latest updates: @IntelEthernet

Back in March 2008, my colleague Ben Hacker wrote up a blog post that compared and contrasted the 10 Gigabit Ethernet (10GbE) interface standards. It was a geeky dive into the world of fiber and copper cabling, media access controllers, physical layers, and other esoteric minutiae. It was also a tremendously popular edition of Ben’s blog and continues to get hits today.

So here we are three years later. How have things shaken out since that post? Which interfaces are most widely deployed and why? Ben has left the Ethernet arena for the world of Thunderbolt™ Technology, so I’ve penned this follow-up to bring you up to date. I’ll warn you in advance, though – this is a long read.

Still here? Let's go.

In “10 Gigabit Ethernet – Alphabet Soup Never Tasted So Good!” Ben examined six 10GbE interface standards: 10GBASE-KX4, 10GBASE-SR, 10GBASE-LR, 10GBASE-LRM, 10GBASE-CX4, and 10GBASE-T. I won’t go into the nuts and bolts of each of these standards; you can read Ben’s post if you’re looking for that info. I will, however, take a look at how widely each of these standards is deployed and how they’re being used.

10GBASE-KX4/10GBASE-KR

These standards support low-power 10GbE connectivity over very short distances, making them ideal for blade servers, where the Ethernet controller connects to another component on the blade. Early implementations of 10GbE in blade servers used 10GBASE-KX4, but most new designs use 10GBASE-KR due to its simpler design requirements.

Today, most blade servers ship with 10GbE connections, typically on “mezzanine” adapter cards. Dell’Oro Group estimates that mezzanine adapters accounted for nearly a quarter of the 10GbE adapters shipped in 2010, and projects they’ll maintain a significant share of 10GbE adapter shipments in the future.

10GBASE-CX4

10GBASE-CX4 was deployed mostly by early adopters of 10GbE in HPC environments, but shipments today are very low. The required cables are bulky and expensive, and the rise of SFP+ Direct Attach with its compact interface, less expensive cables, and compatibility with SFP+ switches (we’ll get to this later) have left 10GBASE-CX4 an evolutionary dead end. Dell’Oro Group estimates that 10GBASE-CX4 port shipments made up less than two percent of total 10GbE shipments in 2010, which is consistent with what Intel saw for our CX4 products.

The SFP+ Family: 10GBASE-SR, 10GBASE-LR, 10GBASE-LRM, SFP+ Direct Attach

This is where things get more interesting. All of these standards use the SFP+ interface, which allows network administrators to choose different media for different needs. The Intel® Ethernet Server Adapter X520 family, for example, supports “pluggable” optics modules, meaning a single adapter can be configured for 10GBASE-SR or 10GBASE-LR by simply plugging the right optics module into the adapter’s SFP+ cage. That same cage also accepts SFP+ Direct Attach Twinax copper cables. This flexibility is the reason SFP+ shipments have taken off, and Dell’Oro Group and Crehan Research agree that SFP+ adapters lead 10GbE adapter shipments today.

10GBASE-SR

“SR” stands for “short reach,” but that might seem like a bit of a misnomer; 10GBASE-SR has a maximum reach of 300 meters using OM3 multi-mode fiber, making it capable of connecting devices across most data centers. A server equipped with 10GBASE-SR ports is usually connected to a switch in a different rack or in another part of the data center. 10GBASE-SR’s low latency and relatively low power requirements make it a good solution for latency-sensitive applications, such as high-performance compute clusters. It’s also a common backbone fabric between switches.

For 2011, Dell’Oro Group projects SFP+ fiber ports will be little more than a quarter of the total 10GbE adapter ports shipped. Of those ports, the vast majority (likely more than 95 percent) will be 10GBASE-SR.

10GBASE-LR

10GBASE-LR is 10GBASE-SR’s longer-reaching sibling. “LR” stands for “long reach” or “long range.” 10GBASE-LR uses single-mode fiber and can reach distances of up to 10km, though there have been reports of much longer distances with no data loss. 10GBASE-LR is typically used to connect switches and servers across campuses and between buildings. Given their specific uses and higher costs, it’s not surprising that shipments of 10GBASE-LR adapters are much lower than shipments of 10GBASE-SR adapters. My team tells me adapters with LR optics modules account for less than one percent of Intel’s 10GbE SFP+ adapter sales. It’s an important one percent, though, as no other 10GbE interface standard provides the same reach.

10GBASE-LRM

This standard specifies support for 10GbE over older multimode fiber (up to 220m), allowing IT departments to milk older cabling. I’m not aware of any server adapters that support this standard, but there may be some out there. Some switch vendors ship 10GBASE-LRM modules, but support for this standard will likely fade away before long.

SFP+ Direct Attach

SFP+ Direct Attach uses the same SFP+ cages as 10GBASE-SR and LR but without active optical modules to drive the signal. Instead, a passive copper Twinax cable plugs into the SFP+ housing, resulting in a low-power, short-distance, and low latency 10GbE connection. Supported distances for passive cables range from five to seven meters, which is more than enough to connect a switch to any server in the same rack. SFP+ Direct Attach also supports active copper cables, which support greater distances while sacrificing a small amount of power and latency efficiency.

A common deployment model for 10GbE in the data center has a "top-of-rack" switch connecting to servers in the rack using SFP+ Direct Attach cables and 10GBASE-SR ports connecting to end-of-row switches that aggregate traffic from multiple racks.

This model has turned out to be tremendously popular thanks to the lower costs of SFP+ Direct Attach adapters and cables. In fact Dell'Oro estimates Direct Attach adapter shipments overtook SFP+ fiber adapter shipments in 2010 and will outsell them over 2.5:1 in 2011.

10GBASE-T

Last, let’s take a look at 10GBASE-T. This is 10GbE over the twisted-pair cabling that’s deployed widely in nearly every data center today. It uses the familiar RJ-45 connection that plugs into almost every server, desktop, and laptop today.

RJ-45: Look familiar?

Alternate title: Finally, a picture to break up the text

In Ben’s post, he mentioned that 10GBASE-T requires more power relative to other 10GbE interfaces. Over the last few years, however, manufacturing process improvements and more efficient designs have helped reduce power needs to the point where Intel’s upcoming 10GBASE-T controller, codename Twinville, will support two 10GbE ports at less than half the power of our current dual-port 10GBASE-T adapter.

This lower power requirement along with a steady decrease in costs over the past few years mean we’re now at a point where 10GBASE-T is ready for LOM integration on mainstream servers – mainstream servers that you’ll see in the second half of this year.

I’m planning to write about 10GBASE-T in detail next month, but in the meantime, let me give you some of its high-level benefits:

• It’s compatible with existing Gigabit Ethernet network equipment, making migration easy. SFP+ Direct Attach is not backward-compatible with GbE switches.
• It’s cost-effective. List price for a dual-port Intel Ethernet 10GBASE-T adapter is significnatly lower than the list price for an Intel Ethernet SFP+ Direct Attach adapter. Plus, copper cabling is less expensive than fiber.
• It’s flexible. Up to 100 meters of reach make it an ideal choice for wide deployment in the data center.

We at Intel believe 10GBASE-T will grow to become the dominant 10GbE interface in the future for those reasons. Crehan Research agrees, projecting that 10GBASE-T port shipments will overtake SFP+ shipments in 2013-2014.

If you’re interested in learning about what it takes to develop and test a 10GBASE-T controller, check out this Tom's Hardware photo tour of Intel’s 10 Gigabit “X-Lab." It's another long read, but at least there are lots of pictures.

In the three years that have passed since Ben’s post, a number of factors have driven folks to adopt 10GbE. More powerful processors have enabled IT to achieve greater consolidation, data center architects are looking to simplify their networks, and more powerful applications are demanding greater network bandwidth. There’s much more to the story than I can cover here, but if you are one of the many folks who read that first article and have been wondering what has happened since then, I hope you found this post useful.

Follow us on Twitter for the latest updates: @IntelEthernet

If you’re a data center or network  professional, you’ve probably heard of unified networking. If you’re not  familiar with it, the concept of unified networking is pretty simple:  combine the traffic of multiple data center networks (LAN, storage, etc.) onto  a single network or a single network fabric – in this case 10 Gigabit Ethernet.  The benefits are just as simple as the concept: simpler network  infrastructures, lower equipment and power costs, and a trusted, familiar  fabric as its base.

The idea of Ethernet incorporating  different types of network traffic isn’t new; it’s been happening for years.  VOIP, streaming video, storage traffic – Ethernet has shown that it’s flexible  and scalable enough to incorporate all of them and more. What’s been missing  until recently was the dominant data center storage fabric: Fibre Channel.

Today, nearly every Enterprise IT  department maintains separate LANs and Storage Area Networks (SANs), the latter  of which is often Fibre Channel, though iSCSI is growing rapidly. A separate  SAN infrastructure requires storage-specific server adapters, switches,  cabling, and support staff. Fibre Channel over Ethernet (FCoE) allows Fibre  Channel frames to be encapsulated in Ethernet packets and travel across a 10GbE  infrastructure. This convergence can lead to big reductions in storage network  equipment and its associated costs. There’s more to it than that, but that’s  enough Unified Networking 101 for now.

So with that out of the way, let’s take a  look at some recent developments in the world of unified networking and see  where things are headed.

In late January, Intel announced the availability of Open FCoE on the Intel Ethernet Server Adapter  X520 family and qualification by EMC and NetApp. Why was that important?  Because it means standard, trusted Intel Ethernet adapters now deliver complete  LAN and SAN unification with FCoE and iSCSI support at no additional charge. Until then, FCoE support required expensive and  complicated converged network adapters (CNAs). Open FCoE incorporates  non-proprietary FCoE support into the operating system, similar to what’s done  for iSCSI today, for performance that is optimized for multicore processors and  will scale as processors and platforms get faster. It’s also a simpler solution  that allows IT to save money and simplify management by standardizing on a  single adapter or adapter family for all LAN and SAN connectivity.

And earlier today, Cisco announced several product and  family updates that provide further evidence of Ethernet’s expanding reach in  the data center:

• FCoE  support on the Nexus 7000 series switches and MDS fabric switches is important  for a few reasons. First, it shows that FCoE and unified networking in general  are pushing deeper into the network, beyond top of rack switches to the  director-class switches that IT depends on to meet the high availability  requirements of mission-critical networks. Second, customers now have more  choices for deploying unified networking. Today, many 10GbE unified networking  deployments rely on a top of rack switch that connects to the servers in the  rack. The port density of the Nexus 7000 means they can bypass the top of rack  switch altogether, simplifying their switching infrastructure by removing a  layer of switches. And finally, these product updates bring the same  functionality and scalability of Fibre Channel SANs to FCoE, which should help  allay fears that FCoE isn’t ready for prime time.

• Cisco  has updated the Nexus 5000 switch family with fixed ports that support 10  Gigabit LAN traffic, 10GbE SAN traffic (iSCSI and FCoE), and native Fibre  Channel connections. These “unified ports” will make it easier for IT to  connect to FC SANs, as previous switches in this family required add-in modules  to connect to those networks. These ports also provide an easy upgrade path for  folks who are deploying 10GbE and plan to enable FCoE in the future.

• The  Nexus 3000 is a new family of switches aimed at environments where low-latency  performance is critical, such as financial services and high-performance  computing clusters. Infiniband networks are often used for clustering, but with  the rise of technologies such as iWARP (Internet Wide Area Remote Protocol), Ethernet  can deliver the same performance while providing a more flexible and  better-understood network fabric.

It’s pretty clear that Intel, Cisco, and  others are headed down the same “Everything Ethernet” path. And why wouldn’t we  be? Time after time, Ethernet has shown that it can expand to incorporate new  types of traffic. And with the roadmap to 40GbE and 100GbE already sketched  out, Ethernet has plenty of headroom for growth. So stay tuned. We’ll have more  to talk about in the coming months.