Have you ever asked yourself that question when you are bombarded with marketing messages from multiple different companies on why choose their products vs. a competitors product?. As a non-Engineer in an engineer centric company, I certainly have thought about this several times and asked myself a very simple question - Why should I choose one architecture type over another offering?

I suppose the best place is to start at the beginning and try and decipher the acronym soup of RISC, x86 etc. I decided to use my ‘old friend’ Wikipedia http://www.wikipedia.org/ to help with this process. What I found was another alphabet soup that I could have researched for hours, but try and simplify it below. I attach my detailed definition findings at end of this blog.

Simply put, RISC (pronounced risk) is a CPU design to use simplified instructions to execute very fast thus providing higher performance. x86 is a generic term that refers to the instruction set of another CPU architecture. So basically both RISC and x86 are types of instruction sets linked to CPU architecture.

So which one should I choose?.
Call me old fashioned, but as a business guy, it always comes down to 3 basic tenets in terms of making a decision
1) I like choice and the ability to pick and choose between multiple suppliers to get the best deal to meet my needs.(and the ability to change supplier without major obstacles)
2) Performance is really important. The higher performance means that I get my work done quicker which reduces the overall cost / improves time to revenue and ultimately improves the productivity of my business
3) System cost and total cost of ownership are key decision points in today’s era which is vastly different from the ‘dot.com’ boom. It is all about managing the bottom line through good decisions around CAPEX and OPEX spending

I applied my decision criteria and quickly found out that there is not a lot of choice from a hardware and operating system perspective with RISC architecture. In fact it looks quite the opposite of choice which always concerns me, call me pro-choice if you like, but I like the ability to move around suppliers!. On the other hand I found x86 to have lots of choice with many hardware vendors to list and a range of operating systems from windows to Linux and Solaris.

Having choice out of the way, I then moved onto performance for my business and looked at published results from many hardware vendors on different websites like http://www.spec.org. what I found was that Intel based systems had a lot of leading results against architectures like SPARC from SUN or Fujitsu and POWER from IBM.

I then looked at price (and being an ex-Accountant in my past career) nearly jumped for joy when I saw that system prices were low for x86 systems compared to the comparable RISC systems.

This analysis helped me understand it better and helped simplify my decision making.

Here is a short video with a little bit more detail. I would be interested in your thoughts and have you had any similar experiences that you would like to share.

As part of the Sun Microsystems and Intel alliance, the two companies have collaborated to bring open source Threading Building Blocks (TBB) support to the Solaris Operating System (OS) and Sun Studio software toolchain. Check out the SUN Blog for additional information. Click the video below for a short interview with Deepanker Bairagi, Principal Engineer for the Sun Studio.

Software parallelism can unleash the processing power that the newer multi-core architectures provide, including the Quad-Core Intel® Xeon® processors. For developers, multithreading offers a software parallelism model, but many existing solutions require a lot of low-level coding. Threading Building Blocks offers a rich approach to expressing parallelism in a C++ program by offering higher-level, task-based parallelism that abstracts platform details and threading mechanism for performance and scalability.

The Solaris OS is able to take advantage of multicore architectures, including the Intel Architecture, with features such as a lightweight processes (LWPs), load-balancing across cores, and processor affinities. Sun Studio software offers a complete integrated toolchain for Solaris and Linux platforms, including parallelizing compilers, performance and thread analysis tools, memory and code debuggers, NetBeans-based Integrated Development Environment, and more.

Combined with Threading Building Blocks, developers for the Solaris platform now have a fully loaded toolbox that simplifies the development of optimized multithreaded applications for multi-core Intel processors. Click here to learn more about Threading Building Blocks and optimizing performance for multi-core processors.

Would like to hear from the community on how you see this impacting the next generation of software development for Solaris running on Intel Architecture.

Technology moves at such a rapid pace - it can often be mind-boggling. Even working directly with the product teams at Intel, I sometimes have difficulty keeping pace. The good news is that there is a tremendous opportunity today to be captured thanks to this rapid innovation, as well as a steady stream of advanced technology that IT can use to better support business and gain a competitive advantage. Recently I was interviewed by Tim Phillips from the Register about the current 45nm Quad-Core Intel Xeon products and the next generation Intel platforms based on the Nehalem processor.

A few years back, Intel fundamentally changed the way we design and develop our underlying micro-processor technology. We streamlined our innovation and accelerated it's pace. Internally, we call this new model Tick-Tock. I like to call it shrink and innovate.

A "Tick" is a manufacturing process shrink that delivers smaller silicon with higher speeds, more transistors and lower power consumption (example: moving from 65nm to 45nm process technology). The 45nm quad-core xeon processors (available since Nov '07) utilize unique materials (a high-k, dielectric) that are delivering industry leading performance / watt as measured by the industry's first and only standard benchmark, SPECPower

A "Tock" represents a more extensive architectural innovation (ex. Intel Core Microarchitecture) introducing new micro-architecture features and functionality fully utilizing the higher transistor count set up by the shrink. For Intel Xeon-based servers, the next "tock" is Nehalem. In addition to the new micro-architecture based on 45nm, a system re-design will incorporate next generation memory, I/O and virtualization technology for high performance, high bandwidth solutions compatible with today's leading software solutions

Sometimes you get so deep into something that you don't realize how crazy it is until you take a step back. Like most technology companies, Intel has an inherent love for acronyms. The cacophony of standards bodies, advanced technologies, and intense rates of change in our industry necessitates the use of abbreviation just to be able to communicate clearly and briefly. However, while I am at least as much of a techno-phyliac as most of the folks in the technology jungle, even I sometimes run into an acronym wall. I thought to help myself and others it might be a good idea to decode one of the newer sets of network technologies that I work closely with and to decipher some of the associated names and acronyms that come along with it.

10 Gigabit Ethernet: It's here, it's real, and it's growing fast.

Ethernet (IEEE 802.x) has evolved over the years from a new standard linking computers together at slow rates and has moved from 10 Megabit per second (Mbps), to 100Mbps, to 1 Gigabit per second (Gbps), and a few years ago to 10GbE unidirectional throughput. Over time there have been several physical connection types for Ethernet. The most common is copper (Cat 3/4/5/6/7 cabling is used as the physical medium) but Fiber has also been prevalent as well as some other more esoteric (such as BNC Coax) physical media types. The most common 10GbE adapter (until very recently) has been Optical only due the difficulty of making 10GbE function properly over copper cabling.

But this post isn't meant to discuss the past, but more to decode the present and future as it relates to 10Gig Ethernet and the variety of flavors that are available. Below I'll cover a number of acronyms for 10GbE IEEE standards that are often lumped together as '10 Gigabit' and discuss some of the differences and usages for each. After that, I'll also try to clear up some of the confusion about ‘form factor' standards for optical modules (which are separate from IEEE) and some of terms and technologies in that realm:

10GBase-T (aka: IEEE 802.3an):

This is a 10GbE standard for copper-based networking deployments. Networking silicon and adapters that follow this specification are designed to communicate over CAT6 (or 6a/7) copper cabling up to 100 meters in length. To enable this capability, a 10GbE MAC (media access controller) and a PHY (Physical Layer) designed for copper connections work in tandem.

10GBase-T is viewed as the holy grail for 10GbE because it will work within the most prevalent Cat 6/7 based infrastructure that is already in place. For this flexibility, 10GBase-T trades off higher power, and higher latency.

10Gbase-KX4 (aka: IEEE 802.3ap):

This is a pair of standards that are targeted toward the use of 10GbE silicon in backplane applications (such as a blade design). The specifically is designed for an environment where lower power is required and shorter distances (up to only 40 inches) are sufficient.

10GBase-SR (aka: IEEE 802.3ae):

This specification is for 10GbE with optical cabling over short ranges (SR = _S_hort _R_ange) with multi-mode fiber. Depending on the kinds of fiber, SR in this instance can mean anything between 26 - 82 meters on older fiber (50-62um fiber). On the latest fiber technology, SR can reach distances of 300m. To be able to physically support a connection of the cable, any network silicon or adapter that support 10GBase-SR would need to have a 10GbE MAC connected to an Optics module designed for multi-mode fiber. (We'll discuss optics modules in more depth further down in this post.)

10GBase-SR is often the standard of choice to use inside the datacenters where fiber is already deployed and widely used.

10GBase-LR (aka: IEEE 802.3ae, Clause 49):

LR is very similar to the SR specification except that it is for _L_ong _R_ange connections over single-mode fiber. Long Range in this spec is defined as 10km, but distances above that (as much as 25km) can often be obtained.

10GBase-LR is used sparsely and really only deployed where ultra long distances are absolutely required.

10GBase-LRM (aka: IEEE 802.3aq):

LRM stands for _L_ong _R_ange over _M_ultimode and allows distances of up to 220 meters on older standard (50-62um) multi-mode fiber.

10GBase-LRM is targeted for those customers who have older fiber already in place but need extra reach for their network.

10GBase-CX4 (aka: IEEE 802.3ak):

This standard of 10GbE connection uses the CX4 connector/cabling that is used in Inifinband^TM^* networks. CX4 is a lower power standard that can be supported without a standalone PHY or optics module (the signals can be routed directly from a CX4 capable 10GbE MAC to the CX4 connector). Due to the physical specification for CX4 based 10 Gigabit, it provides a lower latency than comparable 10GBase-T copper PHY solutions. With the use of CX4 passive (copper) cables, the nominal distance you can expect between your 10GbE links is ~10-15m. There are also amplified 'active' (but still copper) cables with nominal distances up near 30m.

Below is an image of a standard CX4 based socket that would be on a 10GBase-CX4 NIC:

There are also what referred to as ‘active optical' cables are for CX4, which actually have an optics module in the termination of the cable, and the cable body is fiber. This kind of active design increases cable reach and improves flexibility (fiber is smaller than copper pairs) but also increases cost. These active cables can increase reach up to 100m.

Intel has recently released our own series of active optical CX4 cables.

For short distances (such as inside the rack in a datacenter), CX4 offers one of the lowest cost ways to deploy 10GbE from switch to server. Because of its design, CX4 also achieves very low latencies as well.

</end of IEEE standards ramble>

Ok, so we've summarized the majority of the IEEE 10GbE standards. But the immediate question arises: "Why are there so many?" Is the IEEE standards body for 10GbE just throwing out all these standards for every possible niche application? The answer is no. For any new standard IEEE phy interface standard to be approved, it must pass on several criteria including "distinct identity" and "broad market potential". While all of these standards certainly won't apply to any given institution's network, they all do all meet real market needs.

X2, XFP, SFP+... say what?

A final mystery that I've alluded to above has to do with the various optical module form factors that are available for 10GbE. XENPAK, X2, XPAK, XFP and SFP+ are standard optics module form factors that are used by both switch and NIC vendors in the industry. These modules that go along with the 10GbE networking products are an interesting beast. They are not specified by IEEE, but are standardized by a group of industry participants through what is known as a Multi-Source Agreement (MSA).

XENPAK, XPAK and X2 are the older module standards originally used for 1GbE, followed by XFP which shrunk the form factor of the actual module as well as the fiber cable pairs. SFP+ is a newer form factor that is now gaining momentum with switch and NIC vendors. An SFP+ optics module can use the same fiber pairs used with XFP (no new fiber cable needed), but the form factor of the cage in the switch or NIC as well as the optics module itself are smaller. The key advantage of using SFP+ is the new form factor can drive lower costs, lower thermals, and higher densities at the switch.

Here is an image of an older X2 optics module:

And here is a comparison of the size of XFP (right) relative to SFP+ (left):

The optics modules are driven by a low power interface from the 10GbE MAC. The interfaces are XAUI (for X2 modules), XFI (for XFP modules), and SFI (for SFP+ modules). These interfaces generally are supplied directly from the 10GbE based MAC to the module cage. One of the things the module MSA standards bodies agree on is not only a form factor for the module itself but also the electrical specifications of the driver interface that can be accepted from the MAC.

The key thing I want to hammer home here is that IEEE specification (such as 10GBase-SR) is separate from the module form factor used.

For example, you can have a Short Range optical NIC that uses X2, XFP, or SFP+. So asking for an "SFP+ NIC" isn't actually specific enough, because that could mean a lot of different things. You'd have to specify a 10GBase-SR NIC, with SFP+ optics.

SFP+Direct Attach:

Now that I've thoroughly confused everyone, I'll take it one step further. Not only can each module form factor be used with different IEEE MAC specifications, but each module doesn't even need to be used for a fiber connection at all. An interesting example of using an ‘optics' module form factor for a non-optical design is SFP+Direct Attach.

SFP+DA is similar in concept to CX4 but provides a bit more flexibility. Normally, you may have a switch or NIC that is designed to be able to support the addition of SFP+ based optics modules for a 10GbE fiber connection. Direct Attach allows for passive Twin-Axial (2 pair copper) cables to be plugged directly into the SFP+ cage (in place of an optical module) to carry the serial signal from the MAC directly over the cable to another SFP+ form factor enabled NIC or switch.

Again, the downside is that without either a standalone PHY, or optics module to send the signal over a long distance, a passive cable with SFP+DA has a reach in the ~10-15m range. The real advantage for SPF+DA over CX4 is that on the switch side the SFP+ module design allows higher density switches than CX4 can provide.

For a top of the rack switch, SFP+DA will likely provide excellent cost, power and latency characteristic and still have enough reach to be very feasible inside the rack.

10GbE - The Infrastructure is Ready!

I hope that I've lifted a little bit of the fog that surrounds the 10GbE market and the related technologies. The last thing I want to leave you with is the fact that 10GbE infrastructure is now starting to roll into the mainstream. CX4 switches are available broadly in the market today and SFP+ type designs for both optical modules as well as Direct Attach connections have been demonstrated and will be getting rolled out very soon by various vendors.

Intel is already selling a wide variety of NICs and silicon to meet the various form factors and standards based market needs I listed above along with other vendors in the market place.

After years of anticipation, 10GbE is finally hitting its stride. Next stop... 10_0_GbE...

I recently found this simple animation that breaks down the Xeon processor family into bite-sized chunks and explains which Xeon-based servers are best suited to meet common IT and business needs.

I shared it last week when traveling with customers in Taiwan and it was well received.

What do you think of this video?

Here's the 3rd follow-up post in my 10 Habits of Great Server Performance Tuners series. This one focuses on the third habit: Know Your Platform.

As we learned in my last blog, we should start our server performance tuning by looking for system-level bottlenecks. This involves understanding exactly how data flows into and out of your platform - and to do this, you need a block diagram. A block diagram shows the major components on the server's motherboard and the paths between them. From a good block diagram you can derive the maximum data transfer rate (aka bandwidth or throughput) achievable as data flows along those paths.

I usually look at my block diagram before beginning system tuning in order to identify potential bottlenecks. But some people use them in parallel: they measure the bandwidth of various parts of the system and then confirm what they see using the block diagram. You can determine if various parts of your system are heavily stressed, bottlenecked, or lightly utilized. In general you want to trace the path from where data enters your server (NIC, HBA, etc) up to the processor and back to memory or out of the server. The paths connecting one component to another are commonly known as buses. For each bus, multiply the speed by the width to determine the maximum potential bandwidth.

Let's use the block diagram for the Intel S5400SF server board as an example. It has 2 FSBs, each capable of 1333 or 1600 Mega-Transfers/second (MT/s). Each transfer on the FSB is 64 bits (8 bytes), so 8 bytes * 1,600,000,000 transfers gives a maximum theoretical bandwidth of 12.8GB/s per FSB segment. Keep in mind though that in reality a bus will not achieve its theoretical maximum bandwidth - depending on the type of bus it will probably realize 66-80% of the possible throughput.

So, where do you find these diagrams? If you are using an Intel server platform, the block diagrams can usually be found in the technical product specification for each board. If you purchase a platform from one of our OEM partners, ask your salesperson where to get it.

Look at the maximum bandwidth achievable on each link your data will travel over to gain a deeper understanding of how your workload will run on your platform.

Keep watching The Server Room for information on the other 7 habits in the coming weeks.

As a follow-up to my first post on the 10 Habits of Great Server Performance Tuners, this post focuses on the first habit: Ask the Right Question.

• You spend months of experimentation trying to match a level of performance you saw reported in a case study on the internet, only to find out later that it used un-released software you can't get yet.
• You spend months optimizing your server for raw performance. As part of your optimization you fully load it with the best available memory and adapters. Then you find out that your management/users would have been happier with a lower level of performance but a less costly system.
• Your team works hard to maximize the performance of your application server for the current number of users you have, but makes decisions that will result in bottlenecks and re-designs when the number of users increases.

• Why are you tuning your platform? (This helps you decide the amount of resources to dedicate.)
  • As part of this question, consider this one: How will the needs and usage models for this server change over the course of its life?
• What level of performance are you hoping to achieve?
• Are your expectations appropriate for the software and server system you are using?
  • In determining if your expectations are appropriate, refer to benchmarking results or case studies where appropriate and make sure any comparisons you make are apples to apples!
  • A corollary to this question is: is the server being used appropriate for the application being run?
• What qualities of your platform are you trying to optimize: raw performance, cost/performance, energy efficiency (performance/watt), or something else?
• Is performance your top priority for the system, or is scalability, extendibility, or something else a higher goal?

Every now and then a colleague, customer or acquaintance sends me a link to an article or blog that usually either features our products or those from one of our competitors. More often than not I get a lot of repeat sources (The Register, The Inquirer, CNET, etc…). The blog that comes my way most often is one from George Ou at ZDNet. One of his most recent blogs (A comparison of quad-core server CPUs) shows a bunch of our latest quad core CPUs and how they stack up against our previous versions as well as those from AMD. I won’t rehash the article here aside from saying it was positive for Intel and to say AMD’s issues with their quad core processors have been well documented.

Is Intel winning now because our products are superior? Are we winning because our competitor is struggling? Do these benchmarks mentioned in George’s blog tell the whole picture? As you can imagine we constantly ask ourselves these questions and many more internally. Our conclusions are that for processors and server platforms, as long as we provide leadership along several key vectors then our market share and overall market position will improve.

Manufacturing process, processor architecture, system architecture, cache size. These are four critical vectors that we have direct control over when we are making design and enabling decisions. At times in our past and in the present we have had leadership on all four. In those times we have won hands down. There have also been times where a competitor has chosen to focus on one or two vectors and that has led to their products being better for a specific area. The four vectors above are things that Intel focuses on but we always have to keep an eye on what end user value they deliver.

Our customers tell us they care about three main things; Price, Performance and Power. The three P’s. George’s blog shows that for one of the P’s (Performance) Intel has leadership, particularly on integer and floating point. There are similar looking examples for database, virtualization and pretty much any performance benchmark we have looked at recently. Thankfully for Intel, Performance is the “P” with the strongest correlation to success in the server market from a MSS perspective. We are also doing some amazing things with regard to Power. Some have been launched already and some will be coming soon with new products in 2008. The market is segmenting and we now make CPUs, chipsets and networking components that help OEMs build platforms targeted at high performance computing, mainstream enterprise, blades, workstations and emerging markets. Each has unique requirements with respect to the three P’s and one size no longer fits all.

I believe that overall George’s blog highlights the success that we are having today. I also think that there will be a steady stream of innovations that will be delivered in 2008 and beyond that will cause us to rethink how we deliver performance at the most efficient power level for the best possible price point. Virtualization, utility computing and charge back models for datacenter environments are all stepping up to take center stage. We all must innovate or become irrelevant…technological evolution waits for no one.

Shannon