So we are on the home run of deploying the new pilot cube
environment, in fact I’m on site helping supporting day one move in at our
third US site installation which has certainly been interesting. Flight over
went quickly, though at some points it was rather roller coaster (to the point
coffee was spilt on laps)
But I digress…
I wanted to discuss an item I have brought up before;
benchmarking. The project has moved on and worth asking some questions around.
Intel IT has used classic benchmarking applications to compare platforms when
going to RFP (using standard off the shelf applications) but we discovered this
testing wasn’t helping us improve the performance of our software on the client
it was simply giving us faster clients (not a bad thing) We were missing some
critical decision making criteria for evaluating newer versions of
applications, client builds or software tweaks (identifying performance improvement
or impact) As we drive towards more out of the box applications we will also be
using the tool to evaluate impact on the environment.
So we kicked off a project to begin recording certain
productivity metrics to evaluate user perception performance; not necessarily
aimed at just understanding how fast each client is; but more what impact it
has to users
Some of these timing metrics include
Time into operating system
Time into email application (first email)
Time into first instant message conversation
Time to first spreadsheet/document application
Once changes are made to the client build or application
stack an impact is recorded through the metrics. This means we can start to set
goals and performance targets (10% faster build in 3 months…etc)
We hope to publish this data with some fellow travellers to get
some indicators on quantify the overhead an ‘IT’ build compared to an off the
shelf build (we classify it as vanilla OS)
Are you recording productivity metrics to compare
applications and build generations? Any thoughts on if this data would be
useful to you?



