I was looking in a book store (one that tends to stock the way out their books) and came across a book of Numerology. Now for those that don’t know, which I have to admit, I did not, Numerology is about how numbers affects our lives. Supposedly using numbers you can predict everything from if a marriage will work out to if you would lend someone a toothbrush. Now before you ask, I really have no idea if this is a valid form of science, but then I guess the whole point of science is that you don’t ignore something because you can’t understand it.
I often get asked to rate the security of a device so that we can put for example all smart phones in order going from most secure to least secure. In reality this is hard to do as it’s a bit like saying can you take all the vehicles on a road and put them in order from best to worse. We can take one aspect of security and compare it but when you put it all together it’s like saying which is best a Ferrari or a 40ft articulated lorry.
So that calls into question how we do our risk assessments, should the output of a risk assessment allow for comparing of devices? Should the output be a score or a level? Well I don’t want the reputation of just reading books but it seems that risk management has changed and maybe the security world has not caught up.
Some risk management background, most organisations that I know use the standard risk assessment methodology of Threat, Vulnerability & Consequence(TVC). This seems to have come from the business definition of risk, so the finance organisations used much the same process for looking at financial and business related risks. The idea with the TVC approach is to try to quantify the ratings then apply simple maths to come out with a risk score. Well that became a standard and filtered into security. There have been some problems with the TVC model and companies that have seen it go wrong have done some studies into how it fails.
In many cases TVC fails because the risks are not well communicated or the assumptions are wrong, in my experience that’s also why different security professionals risk assessing the same thing get different results. This sort of makes sense because you’re trying to quantify something that’s not quantifiable, risk is a thought process and how you think about a risk will depend on your character and outlook on life.
Where does this all lead to? Well a good risk assessment starts with profiling the audience that will be reading it so the end communication is optimum. Organisations with standard risk assessment formats are kidding themselves this is any sort of saving. Risk assessors must not be worried about saying they don’t know rather than making an assumption and leaving it in the detail of the assessment.
The last point… trust an expert.
We live in a world that’s driven by being able to out argue or prove your point. Sometimes the best experts are those that know it’s not right but can’t say why. Sometimes this is called gut feeling but there is science to back it up. Have a look at Jonah Lehrer’s book called “how we decide” if you want to be convinced that trusting an expert is often right, even if they can’t justify themselves.