In our previous post, A Silent Revolution in the Information Technology Industry we observed that the cycle time to implement and deliver a business application has been steadily decreasing over the past fifty years from several years at the dawn of computing to a few weeks or faster today.

This acceleration of delivery by two or three orders of magnitude is a byproduct a rapidly evolving and maturing current IT infrastructure.

The acceleration comes from the use of pre-built components and our ability to schedule data, applications and compute engines separately, sourcing these resources to the places and methods of lowest cost. We also discovered that this phenomenon is not unique to IT.  Most mature industries have become service integrators taking advantage of pre-existing services. In the example of our previous blog entry,  it would be foolish a car insurance company wishing to build national coverage to start building a network of car repair shops.  Car insurance companies avail themselves of existing car repair shops, and it would be preposterous to think otherwise.

Yet when we think about IT for a large organization, we don't think twice about hundreds of millions of dollars spent in vertically integrated infrastructure, tens of thousands of square feet in huge data centers housing thousands of servers, many of them performing no more than file serving functions and most of the time woefully underutilized.

Under these circumstances it is no surprise that in spite of the proliferation of outsourced services in the past ten years or so, IT is still primarily a privilege for large organizations.  It could be argued that this state of affairs is a side effect of the large granularity of IT resources:  A 10-employee business may not be able to afford to purchase and maintain a collection of servers, each one dedicated to an application and the associated in house expertise.

Cloud computing is changing the dynamics of application integration and delivery very fast.  Service providers in the internet are beginning to offer fine grained services that obviate the need of a large up front capital investment by service consumers:  it is no longer necessary to purchase a complete server for data storage even if only a small fraction is used.  Storage can be rented fromthe cloud by the gigabyte per month.  Virtualization has made it possible for service providers to offer a fractional server for rent for much less than what an in house physical server would cost.

The benefit accrues not only to end user service consumers.  It is lowering the cost for new service provider entrants in the market addressing niches that were not profitable before.  The Mozy backup service and the Pi Corporation data presence services, recent acquisitions by EMC constitute examples of this new trend.

A consumer may pay just a few dollars a month for a cloud based storage service.  This is an example of an IT service scaled down to the consumer market.  Instantiating a service takes just a few mouse clicks and a credit card or a Pay Pal funds transfer.  Compare this process with the status quo of an "in-house" deployment:

The poor consumer is required to research trade publications and the Internet and identify a suitable backup product.  The consumer purchases the product from a software vendor and installs it in the target machine.  Once installed, the consumer is required to follow an onerous regime of regular backups.

Even when the backups are scheduled the user needs to be aware of a number of contingencies, such as ensuring the machines are up and running at the time of the scheduled backup, and if the backup is done to a network shared drive, to also ensure that the connection is in working order and that the target machine is up and running.

If the unthinkable happens and there is a problem with the primary drive, some consumers may not have the expertise to perform the repair and recovery and may need to hire a technician at significant cost.   Even with this hired expertise, horror of horrors, the consumer may find out in this dire moment that backups are missing or done improperly leading to partial or total data loss.

What is wrong with this picture?  First, the end user is being used as the point of integration for the IT process.  We have come to accept this situation in an IT context by sheer habit.  It would be unacceptable in any other context: would a customer hire a taxi that requires the customer to drive the vehicle?

In our next post we will use a constructive proof of how to build a consumer backup service using more primitive component services.

Using a similar approach, the lowered integration services will not only benefit the consumer end user, but also will create opportunities for service delivery in emerging markets.  The fine grained component resources that the cloud makes possible, will enable a new generation of service providers in these markets delivering services specifically tailored for these markets.  The potential economic benefit of this new paradigm is potentially enormous.

The First Annual International SOA Symposium was dubbed by its organizers as "the world's largest and most comprehensive SOA event for practitioners, providing a combination of expert speakers from around the world and a series of SOA training and certification workshops."

The conference, which took place in October 7-8 in Amsterdam, featured about 70 presentations in eleven tracks. Many of the speakers are accomplished book authors in the field representing 22 books already published or as many as 40 if we count the titles under development.

The event was a first, celebrating the coming of age of service orientation and two associated helper technologies, virtualization and distributed computing grids. It is not an overstatement to assert that service orientation has become the new established paradigm by which enterprise applications are to be delivered. Much in the same way that virtualization brought enormous operational flexibility by allowing the decoupling of application instances (virtual machines) from the hardware on which they run, service orientation allows the decoupling of an enterprise function, such as supply chain to be made up of fungible service components.

The mainstreaming of the trio of technologies is borne by their presence in all top five of the InformationWeek 500 Top Innovators in Business Technology for 2008. The top five are

• National Semiconductor with an SOA-based multi-partner supply chain system,

• Hilton Hotels with a system allowing customers to book individual rooms using grid computing,

• Highmark Health for adopting a comprehensive strategy for energy conservation in IT that includes virtualization

• Fiserv, for the development a Web 2.0 Facebook application for the company's banking customers that lets users do basic banking tasks such as paying bills, making transfers and checking balances

• Unum, a provider of health insurance packages to employers, for integrating more than 300 service operations using SOA; price quotes for new packages, which used to take as long as eight weeks, now are done in less than a week.

One dimension of flexibility is that service components may comprise existing applications that have been service-enabled through middleware, or can be service components from the ground up. Under this paradigm, enterprise applications are built from in-sourced components (i.e., corporate-owned data centers) as well as out-sourced components, such as cloud resources. Interoperability is a given. System architects have the mandate to pick the most economical alternative that meets corporate requirements based on the service components' SLAs.

At this conference, I had the privilege of delivering two presentations, Virtual Service Oriented Grids: A Prescription for Scalable SOA and Scaling the Delivery of IT Services to Consumer Space with SOA based on ideas in the book I co-authored with Jackson He, Mark Chang and Parviz Peiravi. This New Book from Intel Press is due from the printer at the end of October.

In the book we predict the emergence of a cottage industry around SOA. This process seems to be at play at the conference: the main organizer for the event is a comparatively pure play SOA consulting house in the Netherlands called Ordina. It was interesting to see the big companiess like IBM, Microsoft, HP, Oracle and my employer, Intel playing supporting role to a number of small companies. This dynamic reflects the increasing value that the role of integration has as a fraction of the value of an enterprise solution. This is not to say that the big players are soon to become irrelevant. They had prominent roles as presenters, in panels and keynote speakers.

The implications of interoperability and open participation aspects of SOA are potentially momentous. No two-billion dollar fabs are needed to as a ticket to entry. Any small team can band together with an idea and start an SOA company. The only requirements are brains and dedication. The potential for technology leapfrogging in emerging economies cannot be understated. Startups in these countries need not start from zero; re-using components already available from more advanced economies can quickly bootstrap SOA adoption.

I was having lunch the last day when the conference Chairman, Art Ligthart approached me asking if I'd be interested in participating in the panel What is the Value of Service Grids? that afternoon. Other participants would be David Chappell from Oracle and Jim Webber from Thoughtworks, hosted by Herbjörn Wilhelmsen, from Objectware. Mr. Chappell is well known I the industry as the creator of the concept of ESB or enterprise service bus, a central concept to SOA deployments. He presented grids as a software abstraction. I observed that for grids to deliver their design performance it is essential that architects pay attention to the underlying enterprise infrastructure that includes processors and considerations of memory and network latency and bandwidth as well as locality.

If you are interested in the foil sets, I'll be happy to mail you a copy. Please send me a short note to enrique.g.castro-leon at intel.com. If you can't access them for some reason, I'll be happy to mail you a copy. Some ideas in the book are featured in an article in SOA Magazine, issue XXII, Sept 2008.

The energy spent spinning up a flywheel, recharging a battery or pumping water up a reservoir can be recovered at a later time with with the appropriate infrastructure, minus a percentage loss lost to heating. This behavior is governed by the First Law of Thermodynamics.

Alas, there is no such luck in the operation of a data center. There is energy stored in the UPS batteries and the capacitors in the equipment, but this amount is minuscule compared to the total amount of electric energy fed into the data center. Hence, it is fair to say that all the electricity fed into a data center eventually gets converted into heat, warming up the air, ground or water around the facility. Again there is no way around the First Law of Thermodynamics.

In any case, the useful output for a data center is not the amount of energy that eventually gets to the UPS batteries or recharges capacitors in servers. It's the amount of computation done at the data center. However, counting CPU instructions is difficult and controversial. Hence it is common practice to settle for the next best metric as a proxy for computation, namely, the power consumed by the CPUs in all servers.

Measured as percentage of total data center power consumed, the CPU power consumption is rather small. Ainsworth, Echenique et al. from IBM (Figure 1-1, page 3) report that only 35 percent of the data center power goes to the IT equipment load. Likewise, power consumed by processors represent 30 percent of the IT equipment load. The number needs to be further derated to the CPU utilization, 20% on the average. If we do the math, the power dedicated to computation is about 2 percent of the total data center power.

John Pflueger from Dell (figure 1, page 9) reports a remarkably similar result. He estimates that 41 percent of the data center power is consumed by IT equipment, broken down into compute servers, storage and communication devices and other IT equipment. The compute server portion is 63 percent, and out of that 31 percent is consumed by the CPUs. If we apply the same 20 percent CPU utilization ratio from IBM, the end result is 1.6 percent, still within the ballpark.

Where does this analysis leave us in terms of actions we can take as part of a first order strategy? The data above is hierarchical, and hence a pyramid is a useful way to organize it:

Changes toward the top, namely in the CPU application workload will have a minuscule impact power consumption for the data center as a whole, yet they can have a dramatic impact in the data center efficiency, that is in the amount of useful computations done as defined above. These changes can take place in two ways.

First, due Moore's Law, a server refresh can potentially double the per output CPU if the servers are two years old, or more than quadruple it if the servers are four years old.

Second, a consolidation or virtualization exercise can address the low utilization numbers for CPUs from less than 20 percent to 60 to 80 percent. Higher numbers are possible, but it is desirable to reserve some headroom to make the servers more responsive to workload peaks. These benefits are attained through the deployment of software technology from VMware, Xen, or, more recently, Microsoft Hyper-V technology that comes with the newly released Microsoft Windows Server 2008, formerly code named Longhorn. A Microsoft white paper, Windows Server 2008 Power Savings reports up to 10X linear power savings in a study with Hyper-V. Results may vary. A basic assumption is that utilization factors are low to start with. Workloads that take multiple servers or workloads that need a server cluster to run, such as large database applications or mail servers might not see such large benefit if the utilization factor is initially high. However, that also means that that the CPU utilization efficiency was high to start with, so there is less room for improvement.

Near the bottom of the pyramid we are talking real megawatts. In many cases the low hanging fruit comes not from from pulling all the stops with technology, but from plain energy conservation. A homeowner intent on lowering electricity bills should not rush to install solar cells. The first step is to conduct an energy audit to identify areas of greatest impact. A data center is no different. In an engagement I was involved with, a team was investigating whether 300 servers could be landed in an aging 25,000 square foot data center without hot spots developing. The energy audit using thermal modeling tools indicated that the data center could actually support a whooping 1,800 additional servers with very minor changes, essentially plugging air leaks in the floor tiles and repositioning a few rows to define hot and cold aisles. Of course, these results must be taken with caution, becase supporting the extra servers would probably have required a power feed upgrade.

So far we have analyzed possible actions that can be taken at the top an at the bottom of the pyramid. What happens in the middle? This is a more complex question and requires the inclusion of process factors. Furthermore, a specific answer always requires a context. Below is a case study presented by Gregg Wyant and James Chen at the recent Intel Developer Forum in San Francisco. Gregg is the Intel IT CTO, Chief Architect and General Manager; James Chen is the Director for Engineering Computing. In this case study, a server refresh was conducted over 4-year old servers. The application requirements did not change, yet running the application in the newer servers allowed reducing the number of machines from 126 to 17. The potential payoff from Moore's Law is a bit over 4X, yet the actual power draw reduction was 8X. The rest comes from application optimization and IT process improvement, a tribute to the Intel IT engineers carrying the application migration.

The reduction from six cabinets to one actually understates the gain. If the cabinet is populated with 1U servers, it will be only half full. The energy density per cabinet however will have gone up. These cabinets need to be housed in a data center designed to handle higher power densities.

In a previous post, Virtual Service Oriented Grids: Scalable Enterprise Computing, I mentioned how the convergence of three old technologies is facilitating large scale computing in the enterprise. It is no coincidence that there are historical drivers for this transformation. In the IT world in the mid to late 1990s, you may recall that this was the era of eCommerce where most business activities under the sun became "Webified" and even the craziest ideas became capitalized. It as a boom which led to the inevitable bust. Some say it was triggered because work on the Millennium Bug stopped once the issue was "solved". No matter the reason, the momentum was unsustainable.

There was a lot of soul searching after the bust. Only a few survivors remain today, the most remarkable examples being Amazon.com, Google, Ebay and Yahoo. If there is one lesson coming from this period is that an essential element for sustainability is that Information Technology and Business need to be aligned.

The increasing adoption of Service Oriented Architectures or SOA represents the increasing recognition by IT organizations of the need for business and technology alignment. In fact, under SOA there is no difference between the two. The unit of delivery for SOA is a service, which is usually defined in business terms. In other words, SOA represents the up-leveling if IT, empowering IT organizations to meet the business needs of the community it serves. This up-leveling creates a gap, because for IT, eventually business requirements need to be translated into technology based solutions.

Our research indicates that this gap is being fulfilled by the resurgence of two very old technologies, namely virtualization and grid computing. To begin with, SOA allowed the de-coupling of data from applications through the magic of XML.

A lot of work that used to be done by application developers and integrators now gets done by computers. When most data centers run at 5 to 10 percent utilization, growing and deploying more data centers is not a good solution. Virtualization technology came very handy to address this situation, allowing the de-coupling of applications from the platforms in which they run. It acts as the gearbox in a car ensuring efficient transmission of power from the engine to the wheels.

The net effect of virtualization is that it allows utilization factors to go up in the 60 to 70%. The technique has been applied to mainframes for decades. Deploying virtualization to tens of thousands of servers has not been easy.

Finally, grid technology has allowed very fast, on the fly resource management, where resources are allocated not when a physical server is provisioned, but for each instance that a program is run.

Virtual service oriented grids represents the maturation of the three underlying technologies. The coming of age for a technology takes place whenever business, process and standardization become overriding considerations. Virtual service oriented grids rely heavily on standardization to attain interoperability, it is guided by governance at the corporate level, and are very much policy based and SLA driven. The underlying technologies become black boxes, their behavior defined by service level agreements (SLAs).

For any application, the management of the components is centralized, but the components ("servicelets") are assumed to be distributed. The servicelets are fungible and can be integrated in real time by design to allow applications to scale up and down, to be assembled and torn down as business conditions dictate.

In the next few entries we will go through a few examples. The subject is rich enough for a book, which indeed we have written. The book is scheduled for publication in September 2008 through Intel Press. Here is the book preface as a preview: New Book Excerpt from Intel Press: The Business Value of Service Oriented Grids.