Ok, nothing is free, but some things are a pretty good deal. I spoke last time about the capacity boost delivered through virtualization. I threw out some big numbers, so here is a bit more detail. More accurately this capacity comes from applying virtualization to a new model for data center management ( you will have to do more than install a hypervisor). I felt pretty conservative with my 5x multiplier in five years.

Even if all you ever read is the in-flight magazine, you know virtualization is a big deal. Hype aside, virtualization is the foundation for realizing the "next generation data center-NGDC". Utilization on enterprise servers is pathetic. The number I used was 15%, but I have heard many customers talk of 5% or even less. The target I used for a super efficient data center was 75% utilization - hence the 5x.

Getting to 75% average utilization will take a lot more than simple consolidation of physical servers onto a virtualized server. This is why I jump to NGDC requirement. Reality says server utilization is all over the place, with odd spikes and many differences in where the bottle neck is. Capacity limitations can be in CPU, Memory, Disk, or Network.

The key to maximizing consolidation is in achieving what I call "Dynamic Resource Management" or sometimes Dynamic Resource Pooling. DRM is what moves the NGDC beyond simple consolidation to Policy Based Balancing of data center resources. In the DRM model a server has become a virtual collection of compute, storage, and network resources. This model is beginning to emerge in commercial offerings from VMware, Microsoft, Sun, Cisco, Virtual Iron, and others.

The trick here is to couple the ability( like in vmotion from VMware) to move a VM from one set of hardware to another, with policy based moves. In my view this makes DC efficiency "just" another logistics optimization problem, not unlike airline scheduling or package delivery. "A game to maximize the utilization, minimize energy use, maximize availability, gracefully handle exceptions, and meet all my SLAs". i.e. a really hard problem. I have tried to capture this journey to NGDC in a compelling graphic, but all seem to fall short. (Thinly veiled request for better pictures of NGDC)

For now achieving the NGDC requires complex software stacks, coupled with management heroics. Intel, IMHO, has the best roadmap and view of this future as shown in the addition of virtualization features across compute, storage, and network. I would like to hear from others where you see barriers and bridges to NGDC. Who are the rabbits leading the way to this dynamic data center?

Every now and then a colleague, customer or acquaintance sends me a link to an article or blog that usually either features our products or those from one of our competitors. More often than not I get a lot of repeat sources (The Register, The Inquirer, CNET, etc…). The blog that comes my way most often is one from George Ou at ZDNet. One of his most recent blogs (A comparison of quad-core server CPUs) shows a bunch of our latest quad core CPUs and how they stack up against our previous versions as well as those from AMD. I won’t rehash the article here aside from saying it was positive for Intel and to say AMD’s issues with their quad core processors have been well documented.

Is Intel winning now because our products are superior? Are we winning because our competitor is struggling? Do these benchmarks mentioned in George’s blog tell the whole picture? As you can imagine we constantly ask ourselves these questions and many more internally. Our conclusions are that for processors and server platforms, as long as we provide leadership along several key vectors then our market share and overall market position will improve.

Manufacturing process, processor architecture, system architecture, cache size. These are four critical vectors that we have direct control over when we are making design and enabling decisions. At times in our past and in the present we have had leadership on all four. In those times we have won hands down. There have also been times where a competitor has chosen to focus on one or two vectors and that has led to their products being better for a specific area. The four vectors above are things that Intel focuses on but we always have to keep an eye on what end user value they deliver.

Our customers tell us they care about three main things; Price, Performance and Power. The three P’s. George’s blog shows that for one of the P’s (Performance) Intel has leadership, particularly on integer and floating point. There are similar looking examples for database, virtualization and pretty much any performance benchmark we have looked at recently. Thankfully for Intel, Performance is the “P” with the strongest correlation to success in the server market from a MSS perspective. We are also doing some amazing things with regard to Power. Some have been launched already and some will be coming soon with new products in 2008. The market is segmenting and we now make CPUs, chipsets and networking components that help OEMs build platforms targeted at high performance computing, mainstream enterprise, blades, workstations and emerging markets. Each has unique requirements with respect to the three P’s and one size no longer fits all.

I believe that overall George’s blog highlights the success that we are having today. I also think that there will be a steady stream of innovations that will be delivered in 2008 and beyond that will cause us to rethink how we deliver performance at the most efficient power level for the best possible price point. Virtualization, utility computing and charge back models for datacenter environments are all stepping up to take center stage. We all must innovate or become irrelevant…technological evolution waits for no one.

Shannon

Server virtualization is becoming widely accepted and vendors and customers are beginning to explore usage models beyond support for legacy applications and server consolidation. Virtual Server load-balancing, disaster recovery (server and data center), dynamic creation and migration of virtual machines, to name a few, are fast becoming widely prevalent.

One of the newest uses for server virtualization that is beginning to garner attention is application portability, packing and distribution, a concept that is becoming more concrete with the advent of virtual appliances. Like the computer/HW appliances like TiVos, firewalls, IPS/IDS and NetApp filers, virtual appliances come pre-configured with applications and just enough operating software needed to perform their tasks, and delivered to the customer as a virtual machine file(s) ready to run atop a hypervisor. Every component of the virtual appliance is pre-configured and optimized and tested by the ISV who has the deepest understanding of the application, thereby eliminating interoperability issues and resulting in a better end user experience. Unlike hardware appliances which typically need specific hardware, virtual appliances run on top of any x86 hardware that has a hypervisor.

Could this be beginning of ‘Virtual-Appliance oriented architectures'? Too early to call, but in a virtualization-enabled world, the promise of an easy application deployment, distribution and maintenance/support is surely enticing. Just like any new technology or application model, there are a lot of challenges that ISVs and customers have to overcome with virtual appliances. We will get into details of these in the next set of blogs, but here is a quick summary of some questions customers and ISVs have to comprehend as they innovate in this space. We will also look at what Intel's doing here with its broad Virtualization Technology (VT) initiative.

• - Security - Do you consider Virtual appliances as black boxes from a security perspective? Would you trust the ISV with both the app and the OS testing? Would there be any back doors? Will ISVs offload testing to third parties?

• - Heterogeneous hypervisor environments - How do you package the virtual appliances for deployment and distribution on multiple hypervisor environments? OVF is a clear direction here.

• - Performance of virtual appliances - Are there issues with virtual appliances sizes as we deploy and distribute business applications in virtual appliances? How do you deal with dependent appliances? Would there versioning issues with virtual appliances? Will there be a need for multiple versions of virtual appliances executing side-by-side?

• - Software licensing - How does software licensing work in a virtual appliance model? How do you buy Microsoft OS licenses? Ubuntu, RedHat, etc are releasing stripped down versions of Linux for Virtual appliances usage. How would the Open source model evolve?

What do you think? You buy into the Virtual Appliance model? Will it work for you? Have you done anything with it yet? Let us know.

I often get asked what type of server a customer should use when landing their virtualised infrastructure, the immediate response is an obvious one, given I work for Intel - an Intel based server ! But beyond this the answer is a little more complex and to some extent depends on the philosophical approach the data centre manager wants to take to architecting their data centre.

There are a number of choices that can be made when using standard Intel based server hardware - ignoring the obvious decision as to the hypervisor vendor - DP ( 2-way ) vs MP ( 4-way ) servers, rack mount vs blade.

Ultimately any server decision is the right one ( so long as its an Intel based solution ) but some of the factors that will influence the decision are :-

• How many virtual machines ( VMs ) are you prepared to host onto a single server - MP servers can host substantially more VMs than DP - over 2x more depending on the workload within the VMs - this is down to the better memory capacity and larger number of I/O slots that MP servers typically support compared to DP servers. Against this using a DP server may be a better solution as 2 DP servers may cost less than an MP server, and combined host as many VM's whilst not having as many VMs hosted onto a single server.

• Density & Form factor - DP servers typically have a higher density form factor than MP servers - at the expense of less I/O & memory capability. But you need to take into consideration that an MP server can host more VMs than a DP server so within a given rack space use of a lower number of MP servers may well enable hosting of more VMs than using more DP servers.

• Blades vs Rack - there is significant momentum building behind the move to bladed servers, mostly driven by the fact that the density achievable using blades is far higher than that possible use rack mount servers. Also the shared resources of a blade solution ( power supplies, cooling, network switches etc ) can lead to cost and power savings in high density configurations. The challenge with hosting a virtualised infrastructure on blade servers however is that blades tend to be limited in the amount of memory and I/O that they can support. The trade off of course is that with the increased density of a blade solution its possible with fewer VMs/Blade but more Blades/rack the overall number of VMs that can be hosted within a given data centre is greater using blades than rack mount servers.

Other factors to take into consideration are that MP servers typically have a higher level of built-in RAS ( reliability, availability & serviceability ) features than DP servers and when hosting multiple VMs on a single server the overall reliability of a servers and its ability to be serviced without shutting down all the hosted VMs becomes very important to the overall efficiency of the Data Centre.

As I said at the beginning - there is no simple answer and a lots depends on the approach you want to take in architecting your solution. Intel's own IT department has done lots of work in this area and have posted many of their results here for others to learn from their experiences.

The only thing that is for certain is that whatever decision is made on form factor the performance of the processors you specify has a direct impact on the number of VMs a server can host - the higher the CPU performance the more VMs that can be hosted and the lower the impact of the hypervisor overhead on the overall system performance, check out the latest virtualisation performance data here and here

As you read the blogs on this portal or visit most industry tradeshows, events or technology portals related to datacenter computing today, you will find it hard not to have noticed virtualization as a topic or as part of the solution for a challenge being discussed. Is it hype or are the people deploying virtualization being wiser? Are there benefits due to virtualization in datacenter? In my opinion the answer is simple: it's not hype, the benefits are real.

Virtualization has been there for decades on mainframes, but the dynamics are changing now with the availability of software and hardware assists that enhance the software and make the software implementation easy and robust for mainstream computing. The deployment of virtualization (including production environments) in mainstream servers is increasing and is projected to increase as many datacenters start to find benefits of virtualization to be real. It is one of the foremost things on the mind of IT administrators/managers, CIO's or CTO's today particularly in North America, Europe and Japan.

The primary motivator in the past few years (and most new adopters in mainstream) has been reduction in capital expenditure (CAPEX) such as consolidation of workloads running on underutilized servers and using virtualization for test and development for rapid deployment. By consolidating under utilized servers, the obvious gain is the reduction in number of servers and hence the power reduction. But that is only a portion of the real benefit. IT managers who have adopted virtualization for a while now have realized that, i.e., in the long run, they see added benefits of consolidation in terms of reduced cooling requirements, reduced physical inventory management, and better utilization of their existing facilities for scaling their services as customer demand increases. Overall a well planned and implemented consolidation can help improve the bottom line of the datacenter operation. Many utility companies also have come to realize the environmental benefits and are encouraging the datacenters in the service area to adopt virtualization. PG&E, SDG&E, and Austin Energy are among few such utilities offering incentives for adopting virtualization (read: http://www.intel.com/technology/eep/incentives ). For instance PG&E has a program where non residential customers in their service area can participate and get $158 for every server that is consolidated due to virtualization and SDG&E offer 8 cents for every KWhr reduced.

Similar to consolidation by being able to test a new environment to be deployed in an isolated manner on the real and very same system where the current workload/environment is running can speed up deployment of new environments and reduce cost due to any unforeseen downtimes.

IT managers who have already realized some the above CAPEX benefits are moving into new usages that offer better operational excellence (OPEX). That is implementing better load balancing and increasing agility by migrating workloads as required and building in operational resiliency with disaster recovery.

Given the above mentioned benefits the IT end users do not/cannot think of virtualization as a single feature or technology but most view it more as a solution. This is also the philosophy and bigger picture approach to virtualization that I can see in Intel products. After leading the introduction of Virtualization Technology hardware assists in mainstream processors in 2005, Intel has worked with a large ecosystem of software vendors to support/enable the capability for a robust solution. With Core Micro-architecture and now a year old Intel Quad-Core processing capability, IT can leverage the industries best energy efficient computing for virtualization. As consolidation and workloads on a single physical server increase, better performance per watt could deliver better results both in terms of consolidation and per VM performance and at lesser power consumption. Currently the 51xx, 53xx, 54xx, 73xx, processor families are all based on Core Micro-architecture, which means for IT focused on VM mobility and agility, this allows easy VM mobility across these different classes of servers. Introduction of Intel VT FlexMigration earlier this year acknowledges the emerging usage model of VM mobility and allows any VMM vendor to develop solutions that will allow future generation of processors to be pooled with older generation of servers (with Core Micro-architecture). This provides better invest protection for IT.

Further the holistic platform centric approach to virtualization hardware assists for greater performance and/or efficiency can also be seen in Intel's approach to virtualization. Intel VT FlexPriority capability (in the processor) most recently announced provides performance enhancing hardware assists for interrupt virtualization. Intel VT for directed I/O is a chipset centric capability that enables hardware assists for I/O virtualization that can enhance reliability and security through device isolation and I/O performance through direct assignment (read: http://www.intel.com/technology/magazine/45nm/vtd-0507.htm?iid=techmag_0507+body_vtd). And Intel VT for connectivity with technologies like VMDq at the networking device level provides throughput improvement in virtualization environment (read: http://www.intel.com/technology/platform-technology/virtualization/VMDq_whitepaper.pdf).

Overall virtualization has real end user benefits in form of capital expenditure reduction and improving operational excellence. When coupled with hardware assists that delivers platform and deployable solution centric enhancements, IT end users can stretch those benefits further.

Continuing on the theme of measuring Data Centre efficiency - power consumption of the facilities and IT load are only one element albeit a large one - that contributes to the overall efficiency of a data centre. Ultimately a DC has to deliver useful workload and the amount of workload that can be achieved within a given physical DC is an increasing challenge. Lowering server power and increasing the cooling effectiveness of a DC are one of several ways to enable more equipment to be installed into an existing facility.

General consensus seems to be that the servers in many data centres do not always run a maximum utilisation - many are in the 10-15% utilisation range. This results from many IT shops following a policy of hosting one workload ( application ) per server and sizing the server to support worse case usage of that workload - this leads to low average utilisation of the servers. There are several approaches that can be taken to increasing the server utilisation

Consolidating several applications onto the same server that have different mixes of utilisation - this is not perfect as a problem on one application could impact the others on that server causing significant business impact

Deploying virtualisation within the DC - this enables multiple OS/App instances to be run on the same server. There are multiple benefits here in that the server utilisation increases whilst the number of servers could potentially be decreased so reducing the overall electrical power consumption of the DC and consequently the utility bill. Another aspect of virtualisation is that to achieve the highest levels of consolidation it is best to deploy the latest generation high perf/low power servers, this can result in the removal of many older generation high power servers from the Data Centre and the deployment of a smaller number of newer more power efficient servers

There are circumstances where virtualisation may not be appropriate and it is necesseary to retain one workload per server - in this case an increase in the workload capacity of a DC can be achieved by replacement of older smaller servers with the latest generation high performance servers - this can enable the workload capacity of a DC to be significantly increased without building a new DC, again the side benefit here is that latest generation servers consume less power than the older servers they are replacing.

There are many different ways in which the workload capacity ( and hence utilisation ) of a DC can be increased , with care most can also result in a reduction in the electrical power consumed by the DC.

Given the right tools the utilisation of servers within a DC is 'relatively' easy to measure, so this element of DC effectiveness can be quantified. There is another major element that I believe contributes to the effectiveness of a DC - that is the processes that are in place to manage the DC and hence the way a DC can respond to the new challenges placed on it by a business unit. Gartner have an infrastructure maturity model that is useful to try and quantify how effective a DC is in responding to business needs and looks at responsiveness, Service Level Agreements, IT processes etc. Currently I do not believe many DC managers are measuring how effective their DC in terms of process and when asked to judge where they sit within a model like Gartner's many IT managers will judge themselves more efficient than they really are.

Are there other areas that contribute to the efficiency of a DC - I would be interested in your feedback.

Since this is the first time I'm blogging on this web site, let me briefly introduce myself. I'm working at Intel since 1984 and started right after University to develop software mainly for the Industrial Automation Industry (way back with good old iRMX for Multibus I/II). After a couple of years of running IT for several Intel sales Offices in EMEA, I'm now running a team of Technical PreSales people to work with End Customers in the Enterprise space.

When working with End Customers in the IT space, we often hear about the requirements of reducing costs but at the same time being more agile. Particularly in the Datacenter this is important to achieve, in order to quickly adapt to changing business requirement and thus swiftly enabling business opportunity through IT. On the way to get to real Business Agility through IT, Gartner has defined the Infrastructure Maturity Model**. This consists of 6 stages with the ultimate goal to deliver Business Agility in almost real time. Before a company can get there however, one important stage is to get to a virtualized infrastructure.

In the storage area we have seen quite some progress in this space which has been adopted already in a lot of medium and large companies. On the server side, I can see server virtualization being one of the hot topics, which almost every company is looking into or even deploying currently in order to achieve this datacenter agility at least in the infrastructure area.

In the past, people typically have used virtualization at large SMP machines to better utilize those; more recently virtualization was used to consolidate (mostly older) servers/application onto 4 way Intel Architecture based Servers to avoid a zoo of different machines and OS Revisions IT has to support. However from the cost efficiency perspective, it is also appropriate to consider using 2-way servers in virtualization too. When we discuss this with end customers, we sometimes got the concern that the ratio of Memory/CPU-Core is not good enough. While we have a great deal of Processor performance, particularly through the Quad Core Technology, which is available in Intel's Xeon^TM Processors since more than a year now, the memory capacity at the DP machines could not always live up to the desired ratio. Recently however there are some new DP Servers on the market (i.e. the Sun Microsystems x4150, http://www.sun.com), which implemented the full specification of the memory interface providing up to 64GB of Memory for Dual Processor Server hosting 8 Cores altogether. While I can hear you saying already that this would need the most expensive Memory Modules (4GB ones), I can tell you, that I was pleasantly surprised about an offer I got recently from one of our suppliers to get the full 64GB, for one of our lab servers, for less than 5900Euros (8400 US$, as you see I coming from Europe). 32GB of Memory would have been just below 2100 Euros (2940US$, 2GB Modules). Obviously prices may vary, but I just wanted to give a ball park figure what the costs are for a DP server containing 4-8GB/Core Memory. So with these types of systems you should be easily able to expand your 4-way system virtualization pool at much reduced cost.

But don't get me wrong here, I'm not promoting that the complete server virtualization pool in a DC should only consist of 2-way systems, I just wanted to point out that with the decrease of cost of the higher density Memory modules and the increase in the number of Memory slots in Dual Processor server space, you have a nice option to select that server type, that fits the best to your needs. If you have for instance applications that need a lot of aggregated CPU Performance or a lot of I/O Performance you sure would be better off using a 4-way server. But I'm sure there will be a blog soon covering the considerations of using 2-way or 4-way servers in the virtualization space.

If you agree in my train of thoughts, one thing must appear as obvious to you. Analysis of the computing resources used by your current applications and capacity planning to meet the need of your future business is the key to success for your virtualization strategy. And here we come back to the Gartner model. As IT you can only become a business value, if you understand the business needs of your company.

When speaking about agility you obviously have to have the possibility to easily migrate a Virtual Machine from a 2-way System to a 4 way system. With the recent introduction of the Intel Xeon^TM 7300 processor based 4 way servers this is possible too. Xeon 5100/5300 processors are sharing the same micro-architecture (Intel Core^TM Architecture) as the 4-way servers (Xeon 7300 Processor), which means you can live migrate VMs from DP to MP systems very easily. This live migration is offered in the various management suites from Virtualization Software vendors. In VMware's ESX (http://www.vmware.com/products/server_virtualization.html) this is called vMotion, at Virtual Iron (http://www.virtualiron.com/solutions/virtual_infrastructure_management.cfm) for instance it is called LiveMigrate.

So those of you, who carefully read Intel's announcement, might rightfully save that all the above is true but now Intel introduced the new Xeon 5200/5400 series using still the same Intel Core^TM Micro architecture, but with an extended instruction set, particularly for the SSE instructions. ...and you are right. If an application uses these new instructions you cannot do a live migrate of a VM from, say a Xeon 5400, back to a Xeon 5300 based system. But here the Intel Architecture offers some hooks (technologies) to still make this possible. For VMware for instance we have implemented a new functionality called VT Flex Migration. Since ESX has such a long experience in the Virtualization of Intel Architecture, it still uses Binary translation for 32 Bit OSs instead of Intel's VT-x (the hardware supported Virtualization). In VT-x Intel offers to mask some CPU functionality so that the OS/Application, when running in a virtualized environment, only sees a certain instruction set and thus can easily be live migrated from a Xeon 5400 to a Xeon 5300 Processor based system. So VMMs like for instance Virtual Iron or Xen (http://www.xensource.com/) may use this feature because they require VT-x. In order to enable the same functionality in ESX, Intel worked closely with VMware and implemented a hardware hook for VMware to allow even in Binary Translation (meaning outside VT-x) to mask certain capabilities (here SSE4) to be seen by the OS, hence making sure the OS uses only those instructions also available in Xeon 5100/5300/7300 Processors.

With this in mind you can setup a very powerful combination of 2-way and 4-way Intel Architecture servers being able to be shared in a virtualized Server pool and allowing live migration between them as the basis for a flexible and agile infrastructure. What you need on top of this now is the Management Software orchestrating the use of this server pool. Those are products like VMware's Infrastructure 3 or their Management and Automation tools such as Virtual Center. At Virtual Iron for instance this would be their Virtualization Manager. Those tools allow you to set rules and policies to automatically react on changes in the virtualization pool, such as a change of CPU load or memory requirements, to allow an automated move of VMs between the servers to still fulfill SLAs.

So I hope I was able to share my view of an agile infrastructure in the Datacenter, I realize that this is quite a hardware centric view of it, but after all I still work for Intel and server system oriented topics are the majority of my job.

I'm looking forward to hear your opinion or questions about it.

Best regards,

Helmut

*Other brands may be claimed as the property of others

**Source: Gartner, Inc. "Infrastructure Maturity Model," by Tom Bittman. Gartner Data Center Summit, 2006.