So we are on the home run of deploying the new pilot cube

environment, in fact I’m on site helping supporting day one move in at our

third US site installation which has certainly been interesting. Flight over

went quickly, though at some points it was rather roller coaster (to the point

coffee was spilt on laps)

But I digress…

I wanted to discuss an item I have brought up before;

benchmarking. The project has moved on and worth asking some questions around.

Intel IT has used classic benchmarking applications to compare platforms when

going to RFP (using standard off the shelf applications) but we discovered this

testing wasn’t helping us improve the performance of our software on the client

it was simply giving us faster clients (not a bad thing) We were missing some

critical decision making criteria for evaluating newer versions of

applications, client builds or software tweaks (identifying performance improvement

or impact) As we drive towards more out of the box applications we will also be

using the tool to evaluate impact on the environment.

So we kicked off a project to begin recording certain

productivity metrics to evaluate user perception performance; not necessarily

aimed at just understanding how fast each client is; but more what impact it

has to users

Some of these timing metrics include

• Time into operating system

• Time into email application (first email)

• Time into first instant message conversation

• Time to first spreadsheet/document application

Once changes are made to the client build or application

stack an impact is recorded through the metrics. This means we can start to set

goals and performance targets (10% faster build in 3 months…etc)

We hope to publish this data with some fellow travellers to get

some indicators on quantify the overhead an ‘IT’ build compared to an off the

shelf build (we classify it as vanilla OS)

Are you recording productivity metrics to compare

applications and build generations? Any thoughts on if this data would be

useful to you?