I recently had the opportunity to sit down with Intel's Chief Virtualization Architect Rich Uhlig to discuss the new usage models and virtualization technologies in Intel new Xeon 5500 series platform. Rich and I have been friends and colleagues for several years and the video of our discussion is attached and can be viewed on Youtube. The conversation sparked some interesting questions from my colleagues, friends and children which I thought I would share with a wider audience.

First the questions from my son's (I have three boys...yes this means that my wife has the patience of a saint):

Dad, what is virtualization? Does that mean you can take people and computers and teleport them to new places, like Star Trek? Did Intel invent virtualization? Why do you think it is so cool? When I grow up, can I be virtualized?

My Answer:

Slow down.....slow down...let me try to answer the questions one at a time.

Virtualization is the ability to increase computer, network and storage utilization with multiple operating systems or logical machines, called virtual machines. This allows Dad and his friends to use more of their computers with different applications and devices. Using virtualization allows Dad and his friends to save money, save power and increase efficiency.

Response (My three son's in unison):

Boring! I thought you said your job is cool. Your such a geek......(trailing off and looking at their iPods)

My response:

Guys, hold on...let me explain. Virtualization technology IS cool. While it wasn't invented by Intel, we have worked with an industry of incredibly gifted engineers, architects and designers to create new ways for people to use their computer technology....and the best part is we are only in the beginning. By the time you are an adult you will have the opportunity to use virtualization technology in ways we are only beginning to imagine. Think of virtualization as a journey and evolution of computer technology for Dad and his friends to maximize the use of the computers that we buy/build. Hopefully, with more innovation and computer technology advances you will be able to create a virtualization layer that will allow you and your digital identity to "teleport" to new places in a virtual cloud. You won't be "virtualized" but you will be able to create your digital environment wherever there is a machine that can understand your commands. That is pretty cool. Think of it this way, you can save and play your Nintendo Wii, Sony Playstation or XBox profiles on any machine, any where in the world that can download your profile.

Response (from my 13 year old):

You mean I can play EA's Madden Football 24 hours a day with my friends, even when we are on vacation and you want me to see some historic landmark, like the Lincoln Memorial?

My response:

Well...yes but not exactly what I had in mind. (aargh!)

A recent question from my friend from a former job on Virtualization:

I hear the new Intel chip, Nehalem (formerly known as the Intel Xeon 5500 series), is the best product you guys have released in a long time, What makes the product so good, is it the virtualization technology that you work on?

My response:

     Virtualization technology provides increased instrumentation and flexibility for the Intel Xeon 5500 series platform but it is only one a host of fantastic features which make this product the best we have ever released. For Data Center managers, increased efficiency is an every day part of life. Nehalem offers increased performance, increase memory capacity, a new Quick Path Interconnect (which acts like a NUMA switch fabric on silicon, remember that cool product we launched in 1997 at Sequent Computers?) and a 2nd generation of virtualization capabilities that deliver native virtualization instruction capabilities for VMWare, Microsoft, Citrix and a host of Xen providers. It is a truly a breakthrough server product. With this new architecture and design characteristics we are able to meet the needs of a platform of new Virtualization usage models including: Rapid Application Deployment, High Availability, Virtual Desktop Infrastructures and Server Consolidation. It is a very exciting time...

My friend's response:

     Very cool. I miss working on hardware innovations...sounds like you guys at Intel are up to something special. Should I buy the stock?

My response:

     Thanks. Intel is a great place to work and we are doing some very cool product innovations. Do we always have to talk about stock price?

Finally, a recent question from a dear colleague:

     What happens if virtualization technology is deployed on every platform that Intel ships? Won't business and consumers need less devices? Won't users no longer have an insatiable demand for compute, network and storage resources?

My Answer:

    Funny you should ask that question. Rich Uhlig, Fernando Martins, Rick Olha, RK and I have debated this exact question for years. The answer is simple. Virtualization increases demand for more resources than ever before. In fact, until the recent economic downturn virtualization technology was cited by a Citigroup analyst as the key driver to Server growth in 2H 2007. For the first time in over 10 years the markets average selling price was increasing. Why? Because users could do more with every server they purchased. Virtualization actually facilitiates more usages on more application development and production environments than ever before. As we increase the performance of the instruction sets and Intel Microarchitectures we increase the capabilities that virtualization can impact for new usage models, while preserving some the legacy compatibility that users require for 32-bit application workloads. Simply stated, "we can do more with less!"

Next question (by the way this was a skeptical Intel exec.):

     Doing more with less is fine...but what about our volumes for server products? what happens when virtualization is prevalent across all of Intel CPU and Platform offerings?

My response:

    Flexibility and control are critical to all of our customers regardless of form factor. Is there anything worse then buying a new server, PC or handheld and having application compatibility errors? No. Do we really believe the world wants to become software compatibility specialists everytime Microsoft releases a new operating system? What about Dell, HP, Lenovo, IBM, Acer, Nokia, Motorola, LG, Samsung, RIM and HTC? It has taken us over 10 years of research, testing and product development to get here. Virtualization is a "Hot Topic" today and will be in the future because it makes a positive difference in our customers lives both financially and efficiently. Our job is deliver the greatest silicon products the world has ever seen, over and over and over again. Virtualization allows us to do that AND preserve the investments our customers and software partners make in developing their own operating environments. What is cooler than that? Virtualization facilitates innovation, consumption and utilization, our customers are telling us this everyday. Innovation is critical to this process, enabling our software colleagues is a must and opening up the discussion is part of the process.

Her response:

     Well, I guess you are pretty passionate about virtualization?

My response:

     I hope so...that is why you hired me.

Have a listen, enjoy the video and join the discussion of Rich and I. For us, Virtualization is a very Hot topic, that we have thought is Cool for a very long time.

I’ve spent a fair number of words in the past on the benefits of 10 Gigabit and what it means for the server market.  Through the addition of FCoE and DataCenter Ethernet as well as advanced virtualization features 10 Gigabit seems likely to have its big day in the sun here pretty soon.  But the question is still “When”?

While the proof is ultimately in the raw volumes of 10 Gigabit that ship, and the number of IT users who utilize the higher performance, there are some key reasons to think that 10 Gigabit momentum is accelerating beyond just the numbers* below:

Over the past year, there has been a raft of new 10 Gigabit switch announcements** from Cisco (Nexus 5k/7k), Arista (7100, 7124, and 7148), BNT (G8100), Extreme Networks (Summit X650) Juniper (EX8200), Voltaire (8500) and many others that have increased the choice, and the density of 10 Gigabit switches in the marketplace.   There are now many 48+ port 10 Gigabit switches available and even a few 200+ port models.  Also, the improved density and feature set of certain switches (such as Voltaire’s 280+ port 8500 series switch) provide a path for 10 Gigabit’s ascent into the clustering market by improving port density and latency for clustering applications.

Broad acceptance of SFP+ has also helped to drive a rapid improvement in price, density, and power.  SFP+ provides a smaller form factor standard for optics, as well as a standard connection methodology to connect directly from switch to NIC via a Twin-Ax copper (read: ‘low cost’) cabling solution inside the rack (up to 10m).  The widespread adoption of SFP+ form factors has dramatically reduced the entry level price points for switches, and through the ‘direct attach’ copper connection capability it has also reduced the overall cost for initial and ongoing deployments of 10 Gigabit by providing a lower cost bridge to optical or full 10GBase-T support.

There are also a few data points to suggest that the Server side cost for 10 Gigabit will also be dropping fast going forward.  As power for 10GBase-T continues to drop quickly, more and more Server vendors are looking at the options available to embedded 10 Gigabit directly into their systems.  This will not likely be a 2009 story, but it is approaching quickly.  Additionally, the acceptance of SFP+ form factors for optics/direct attach cabling has provided a path that some Server vendors may use to design 10 Gigabit down on motherboards without adding the extra cost and power of a 10GBase-T solution.  This looks like a likely near term given that the solution power and design are robust and ready for motherboard based designs today.

Finally, the continued cost reduction provides an attractive long term value of standards based 10 Gigabit Ethernet.  There is clear indication downward pressure on 10GbE prices already present today.  We will see 10 Gigabit pricing follow a similar price curve as we saw with Single Gigabit.  This is evidenced in the recent pricing announcement where Intel reduced the cost of single port 10GBASE-T adapter 40% from $999 to $599.  The competitive economics of standards based hardware will continue to drive down 10 Gigabit prices even further and we will see 10GBASE-T pricing below the $500 / port price in the near future. Once it gets on the motherboard, prices will drop even further.

Overall, the power, density, latency, and cost of 10 Gigabit are all improving at a rapid rate.  Form factor flexibility coupled with a wide array of switch and NIC vendors in the marketplace will provide choice and low cost for IT departments while virtualization and convergence in the datacenter and elsewhere continue to provide demands for ever greater I/O bandwidth and performance.

My grandfather was born in the early 1900’s.  By all accounts he was a hardworking man with a strong degree of curiosity.  He passed away in his late 80’s and before he died I remember talking to him about my pursuit of an Electrical Engineering degree.  He nodded politely, asked a few questions and when I helped to fix the electrical outlet in his garage I got the sense that he thought I was heading down the path to be an electrician.  I believe that thought pleased him.  Several years ago I was explaining to my five year old daughter in layman’s terms what I did for a living and what my company made.  I said things like “We make tiny engines that run computers” or “I work with computers that run websites like Webkinz® and Disney®”.  She seemed impressed.  Months later when she was asked by a parent of her friend what her dad did for a living I was a combination of proud and surprised to hear that she replied “They make chips…”  (proud moment) “…and salsa!” (um OK.  I still have work to do).

_

Now the other day she walked up to me and said something like “Dad, I am having trouble getting the Slingbox to work on mom’s iPod Touch.  It is connected to the Internet but the remote does not seem to be changing the channel.  Can you help me?”  Clearly she has made some progress up the technology curve, but it also struck me how far she has come.  Kids these days are surrounded by technology.  In our house alone there are at least the following electronic devices; Oven, Microwave, AppleTV, refrigerator, smoke detector (3), carbon monoxide detector, programmable thermostat, furnace, radio, garage door opener (2), wireless speakers, televisions (3), set top boxes (3), ceiling fans with remotes (3), netbook, Slingbox, Clear wireless router, remote outlet, sprinkler control box, iPod Touch, desktop computer, Wii, iPod shuffle (2), alarm clocks (3), oven timer, electronic light dimmer, cordless phones (4), AV receiver, DVD players (3), VCR, iPod docking station, security system, motion sensor, camcorder, camera (2), USB hub, music keyboard, AV switch, computer keyboard, battery chargers (4), Wii remotes (4), Wii Fit Pad, Wii drums, copier/fax/scanner, computer monitor, AC, Power supplies (4), RFID credit cards (2), washer, dryer, noise canceling headphones, answering machine, internet modem, cell phones (2), handheld GPS, auto GPS and electronic battleship.

_

I am sure I have forgotten several things and I did not count cars or anything at my children’s school.  I am also sure each of the electronic devices in our house has either a processor, microcontroller, ASIC or multiple of each.  Admittedly, the silicon content in our house is probably above average given where I work and the personalities my wife and I have.  But when I think back to my grandfather he had none of these silicon laden items.  I am sure he didn’t care since it is hard to miss something you never knew.  Of the hundreds of pieces of silicon in our house about a dozen or so are smart enough to connect to each other or to “the cloud” in some way.  I put “the cloud” in quotes because it is not only the most over-hyped word of it’s time it is also the best way to articulate what I suspect my children and many others think of the services that they get when all of this stuff gets connected.

_

I can safely say two things are fact. First, my grandchildren will have in their house many more pieces of silicon than I do. Second, they will have more pieces of silicon that can connect to each other and communicate with “the cloud”.  There are many billions of devices connected to the Internet today and that number will grow.  At Intel we are building silicon, and increasingly software assets, that facilitate the processing and movement of data both on those devices and between them. Servers are increasingly becoming an important part of that over-hyped cloud word. My cable company has a cloud delivering me my on demand video content, A social media site allows me to upload pictures into their cloud to share with my friends, someone just used a cloud architecture to develop a perpetual motion machine.  OK, one of those things was false.

_

My grandfather thought a cloud was something in the sky.  My children think it streams video to their handheld device.  What will our great-grandchildren think?

Every day in our personal lives, we’re bombarded with “opportunities” to get a better deal.  At the grocery store, we might be able to buy a single item for $2.50 or 3 for $5.00…which then forces us to go thru the mental gymnastics of figuring out how good of a deal it is, and whether or not we really need three 96 oz. bottles of salad dressing.

But there are some opportunities out there for adding a bunch of compute performance are a bit more straight-forward.

Case in point: Dell recently had Principled Technologies compare the performance for the Intel® Xeon® Processor E5520 and E5506 CPUs each running on a PowerEdge R710 server.  Both are 4 core processors, but the E5520 has many advantages over the E5506: 

• higher frequency (2.26 GHz vs. 2.13 GHz)
• faster QuickPath speeds (5.86 GT/s vs. 4.8 GT/s)
• faster memory support (1066 MHz vs. 800 MHz)
• Turbo Boost
• Hyper-Threading support.

Long story short:  Buying a slightly better processor with a server purchase can drastically increase your performance.  So if you are looking to buy a Dell PowerEdge server configured with Microsoft SQL Server 2008* and an Intel® Xeon® Processor E5506, for an additional $300 you can get up to 75% more performance by upgrading to an E5520 CPU.  More performance headroom in a similar power envelope, faster QuickPath and memory speeds, Hyper-Threading and Turbo Boost functionality – all for $300.  NOW THAT’S A GREAT VALUE!

Check out the summary document for the Dell R710 Principled Technologies performance testing, which also has comparative performance testing for the Xeon® E5540 and X5550 CPUs (also a great value for the money!), along with results for Microsoft Exchange.

NOTE:  System pricing from www.dell.com as of May 13, 2009.  Actual performance will vary based on configuration, usage and manufacturing variability. See the actual Principled Technology report in the following link for complete system configuration

The debate on how to best increase system capacity to accommodate growing applications has raged on for years; “scale up” with more CPU, memory, and I/O, or “scale out” with loosely connected systems.    Scaling out by adding networked systems to increase capacity has been a good economical solution for many IT managers because it allows them to grow by using less expensive, industry standard building blocks.  However, there are some notable exceptions to this line of thought.  One is that the class of applications that require shared memory and large database support are much better suited to run on a single, expandable system that scales up.  These are typically transaction processing, business intelligence and ERP solutions.   Until now, IT managers running applications that require scale-up systems larger than 4 or 8 CPUs have had limited platform choices and most were proprietary and expensive RISC-based servers.

The other problem with the scale out approach is the people, facilities, software and overhead costs and complexity of managing very large numbers of servers, which can grow to a point where the costs outweigh the performance and system cost benefits.  The industry solution to achieving better ROI has been to consolidate multiple scale-out servers onto single industry standard scale-up servers with virtualization solutions.  This is a good solution, but is limited by the number of application loads the IT manager feels comfortable placing on a single server, given the need to maintain peak performance and availability for each application.

Well, it looks like the scale-up, scale-out debate is about to take another turn.  In the server product update Intel gave on May 26th, they talked about new levels of system scalability and choice supported by the upcoming Nehalem-EX processor.  This processor will support systems that scale up to 8 sockets natively (shared memory, without any additional silicon), and up to 16 sockets and higher with node controllers from system manufactures that allow single systems to share memory beyond 8 sockets.   So far there are over 15 different designs from 8 OEMs that offer 8 socket or higher scalability.  But of course, for the class of application where scaling is important, socket count doesn’t tell the whole story of what’s needed for scalable performance.  Thread support, key for transaction processing and virtualization, scales at the rate of 16 threads per socket with 8 cores and Hyper Threading (2 threads per core).  That would be 128 threads for an 8-socket system, and 256 threads for 16 sockets.   And in order to keep those threads fed with data close to the CPU, each processor supports up to 24 MB of shared cache (1.5X current generation Xeon), and an impressive 16 memory slots per socket or 128 DIMMs on an 8-socket system.  In addition, the Scalable Memory Interconnect gives these systems 9 times the memory bandwidth of today’s top Xeon processor.  Finally, four QuickPath interconnect links per socket allow for high-bandwidth sharing of data across the system.

So the net of it is that the industry is going to see a broad selection of highly scalable, next-generation servers that significantly extend the economic advantage of industry standard scale-up solutions for business-critical, large database, and high-end virtualization/consolidation deployments.     I would expect these systems to give IT managers a very cost-effective alternative to the much more expensive and proprietary RISC-based servers they use today.

What are your thoughts?  Mike

Related Topics:

• NHM-EX Press Fact Sheet
• NHM-EX May 26th Press Briefing Video – condensed version
• IBM 8Socket Demo Video
• NHM-EX--A New Standard

I’ll not debate whether Cloud Computing is a passing fad, marketing hype, a revolution in computing, etc.; what I do know for a fact is that the interest in this model, from equipment vendors, service providers and end users is staying strong.  As much as Intel is reaching out into the industry to learn how people are hoping to take advantage of this phenomenon, what’s exciting to someone like me is that more and more service providers are approaching us on this is topic and seeking our input and guidance.  Service providers of various kinds are asking for Intel’s opinion and advice on how to prepare and evolve their data center architecture and practices to align with the expectations their customers have for cloud computing.  I’m not trying to brag, especially since it is obvious that there’s a ton of things “we” still need to figure out in this area; but when I see some of the giants in this community express appreciation of the contribution Intel is making, I can’t help but feel glad that we have done at least some of our homework right!

So what’s a chip company doing that could be remotely interesting to service provides?  Aren’t these the guys whose job it is to abstract all the hardware?  Absolutely!  But service providers are realizing that their solutions are better delivered and their business models are more competitive when they have a deeper understanding of what the underlying hardware is capable of.  For example, many of the customers I work with tell me that they were unaware of the technologies enabled by our platforms to intelligently manage server power consumption, not just at the individual node level, but for the whole of the data center.  My colleagues at our customers are pleasantly surprised to learn how Intel is pushing the boundaries for virtualization deployment and in collaboration with the leading vendors of virtualization software is making the use of this foundational technology more efficient for cloud computing.

There are many more topics I can add to this list, and service providers have a lot of places to go besides Intel for information.  But what I hear often from the customers I work with is that Intel’s ability to be an impartial (vendor neutral) technology advisor is most appreciated.  Of course not everyone is in a position to take advantage of the latest technology, nor does every new technology we enable serve everyone’s purpose.  But if you are a service provider interested in topics on data center optimization whether that be at: the cpu or chipset, the server, the software or the facilities, I’d encourage you to read up on our products and technologies found in this forum, and in other places on our intel.com sites.  And if there is something you need but can’t find, or need more information feel free to drop me note.

Today Intel provided a server product update for the upcoming Nehalem-EX processor and the expandable platforms based on it.  Here’s a recap of some of the interesting messages communicated to the press:

• Nehalem Architecture and Quick Path Architecture are coming to the EX (MP) segment, 4 Socket Servers and above. 
• EX Servers are ideal for server consolidation / virtualized applications, data demanding enterprise applications and technical computing environments.  Both Itanium and Xeon processors based systems represent an attractive alternative to more expensive, proprietary RISC-processor based systems.
• EX Servers are designed for the high-end.  They offer more capabilities (i.e. memory, RAS, cores/threads, sockets) than 2 Socket Servers that IT managers require for business drivers such as large scale server consolidation, high data demands, virtualization, and scalability.
• Up to eight cores / 16 threads and a whopping 24MB of cache.
• Up to 9x the memory bandwidth vs. today’s 4-Socket Xeon 7400.  The performance will be dramatic – the highest-ever jump from a previous generation processor. 
• 2x the memory capacity with up to 16 memory slots per socket (that’s 64 DIMMs on a 4 Socket Server), and four high-bandwidth QuickPath Interconnect links.
• New levels of scalability: from large memory 2 socket systems through 8 socket systems, and even more with OEM node controllers.  Matter of fact, there are over 15 8-Socket+ designs from 8 OEMs currently. 
• IBM showed their 8S Nehalem-EX server design running 128 threads (8 Sockets x 8 cores x 2 threads due to Hyper Threading)…an industry first. 
• New RAS features traditionally found on Itanium, such as Machine Check Architecture (MCA) Recovery which detects CPU, memory, and I/O errors, works with the OS to correct, and helps recover from otherwise fatal system errors. 
• Nehalem-EX is scheduled for production in the second half of 2009, with OEM systems in early 2010.

Stay tuned over the next few days – we’ll post a video from the event.  Also look for some informative blogs over the next 1-2 weeks that will offer more of an in depth view of Nehalem-EX’s 4 Socket capabilities, performance, scalability, RAS, and Virtualization.