Most data centers have a very long history with the enterprises they provide services for. Data centers grew up around the users they provide services to and are generally located within a close proximity to the user base. As LAN capabilities improved, performance of local applications became less of an issue for enterprises but new capabilities were generally landed in existing facilities. As the enterprise grew through mergers and acquisitions, additional data center capacity was generally co-located with the new users.
When availability was the key metric for measuring performance of applications, it became customary for each new application that the enterprise was rolling out (whether developed or purchased) to have its own dedicated hardware, to protect the integrity of the application and improve stability. Over time this has led to data centers around the world that are full of hardware that operates at a fraction of what it's capable of! Application owners were loathe to consider the idea of ‘stacking' applications on the same server, looking to avoid potential conflicts that arise in a shared resource environment.
We now have multiple vectors for driving efficiency into our data centers: energy efficiency, sustainability and cost are some that are moving organizations toward initiatives that will transform the way we provide services from the data center and how we support those services going forward. Driving efficiency is a painful but necessary step in the overall transformation.
Why is this abstraction of resources from users, developers and administrators necessary? A primary driver is to standardize facilities, network, compute and storage resources so the operations staff is sustaining a small number of standard offerings, which provides them with a very predictable environment that they can become expert is sustaining. By centralizing to fewer facilities and driving toward higher utilization of existing resources (i.e. network, servers and storage) the enterprise can obtain more work out of their data centers for less energy and therefore less cost.
Removing the users, developers and administrators from the data center is a process and mindset change that will take resolve and ultimately executive support. So many IT pros feel they need to ‘touch' their environment but this leads to custom configurations, unknown/undocumented changes and instability in an environment that we're trying to standardize. Building the competency within the operations group will facilitate a change in the way excursions and outages are dealt with and will form the basis of a much more predictable environment that the operations team will feel proud to own. This is one piece of the transformation that cannot be overlooked when considering the evolution of the enterprise data center.


Alan - interesting dilemma that we have been trying to address for years now. Although I've never really understood why a developer would want to have access to the data center - as a developer for 20+ years, the last thing I ever wanted to do was have access to the hardware - just give me a protected (ah... here is the real issue) environment where my applications can run with enough resources to satisfy my customers and I'm happy. I don't care where it is as long as it meets my performance and reliability concerns.
So the real issue has always been that the windows environment (web or otherwise) has never been secure/protected enough to run with "stacked" applications. Most developers have been okay with being dependent on themselves or their teams to write "well behaved" applications but reluctant (to say the least) to have to depend on others to do so. Thus the "keep your hands off of my server" mentality that is prevelant today (at many companies).
Can't say that I blame them given past history around system reliability and well behaved applications.
However now with the new virtualization technologies it is becoming a more viable option to have multiple applications (from multiple teams which is the real key) running on the same physical device - as long as they are in their own "protected" virtual machine. This is a reasonable near term alternative until we have a much better operating system that provides this type of logical partitioning within a single environment. Although I'm not sure that is really necessary. There is another thought...
Good topic by the way