In my experience over the years, calculating security value and providing consulting to others doing the same, I have noticed the same 4 questions tend to rear their ugly heads. Requests by senior managers, finance analyst, business value analysts, project and program managers all fall into one or more of these types of inquires. And when I say they are ugly, oh they are.

In most cases the parties seeking information are in some phase of the decision cycle:

Should I spend money on security? - This is a business decision based upon compelling drivers, usually loss of some kind, including non-compliance to regulatory requirements (which could send a C-level officer to spend an extended vacation at Club Fed) or risk of a catastrophic blunder sufficient to crater the organization. The business aspects must include how many coins are in the coffers, amount of loss (both realized and unrealized) on the table, and if money could be better spent elsewhere (opportunity costs)

How much should I spend? - A value decision considering what the organization is willing to accept in losses, what can be spent on security, and the amount of loss which could be prevented. Optimally, there exists a point at any given time which management is willing to spend a certain amount on security, which prevents enough loss to bring the residual losses to an acceptable level.

What should I spend it on? - An exercise in comparative analysis of available options which drives down overall costs, while increasing the losses prevented, and maintaining the optimal level of security and residual loss.

On to the ugly questions (feel free to share your experiences):

Ugly Question #1: How do I select the security product/program with the best value?

This is typically asked by senior management or by a product/service manager seeking to identify the best solution among a pool of several competing initiatives. As an example, they might be looking to purchase an Intrusion Prevention System (IPS) and looking for the best of breed. Conversely they may be looking to establish or improve a security capability (example: data protection) and trying to determine the best product among multiple solutions (encryption, IPS, document tracking, data destruction policy, etc.) across multiple vendors.

The challenge is to be able to compare which solution will best achieve the optimal level of security. This is a function of security cost, losses prevented (effectiveness), and acceptance of residual loss. To simply go for the cheapest, most effective, or fastest to adopt is most often than not, the wrong long term answer. (..and security is a long term proposition)

Ugly Question #2: What is the value of this security product/program?

This is asked by management and project managers when a solution is in the proposal stage, by the sustaining operation folks once it has been implemented into the environment, and by management during times when the organization is looking for opportunities to cut costs. As value is a dynamic concept, it can radically change based upon business, legal, and social aspects as well as the normal fluctuations in the threat landscape. First step here is to identify what types of value was intended to be provided and the appropriate metric to measure those aspects.

As an example, management may be seeking to protect the organization's image and liability from the loss of Personal Identifiable Information (PII) through the implementation of a hard drive encryption program for company laptops. The metrics may be as simple as determining the saturation of the program and if encryption is sufficient to protect from liability in the geographies they do business in. In this manner you can estimate the amount of coverage for which liability and image concerns are abated.

You might think, wait, that is not a dollar figure! Where is the value? Well, in this case, management may be looking for the establishment of a capability. Either we are protected from this threat or we are not protected. The same stratagem could be compliance with HIPPA or other regulations. To attempt to quantify a dollar figure in this example would be overkill and may detract from what is intended. Realistically, a dollar savings cannot reasonably be calculated now matter what kind of magic hat you possess. I have seen some attempts, by people with the best intent, to do this very calculation. But not knowing if or when or to what extent a loss may occur, nor to be able to truly measure the potential losses due to the large number of unknown variables which have an astronomical range of potential damage, these assessments are pure folly (but really fun to poke holes in). Half the battle in measuring the value of security is to know what limitations exist regarding the granularity of what can realistically be measured and validated.

Ugly Question #3: How do I compare the value between security and non-security initiatives?

This one bites. Really. It is almost impossible to do, anyone can challenge the results, and if you get this wrong everybody hates you. This comes up when senior management must decide where to spend hard earned budgetary dollars. It becomes an "us versus them" battle between security and some other group. Each party wants the money to spend and the infighting can get downright dirty. So what is a manager to do? Just tap your friendly neighborhood security analyst to calculate the value (just as long as it is not me), then compare against the value of the non-security program. Easy, right?

I wish. Security programs rarely have the benefit of real dollar justification attached. Unless you are in the security products/service industry, security does not generate revenue, it is just overhead. More on that in a different blog. Non-security programs have the edge here. A marketing program may generate XX dollars, an operations efficiency program may save YY downtime or be able to cut ZZ heads from the budget. These strong arguments bark loudly to management. Security value will retort with a whimper, maybe a risk reduction of xx% or at best a loss prevented of yy dollars. Did I mention even calculating such values takes more time, with more assumptions, and can't, in most cases, ever be validated as compared to the non-security programs? Pure ugliness. Alas it is not impossible. I have seen the fight won (ie. management given accurate and comparable data to make the best decision), but be beware, the deck is stacked against security.

Ugly Question #4: How much should my organization spend on security?

This is the big-daddy of questions, posed by senior management or if the organization is large enough, by a divisional head. Although I plan on discussing this in greater detail in another blog and whitepaper, the path to take is to identify the optimal level of security.

Every organization is different with ever changing business needs and drivers. What one company desires from its security program and is willing to spend will differ from its neighbor. The willingness to accept different levels of loss also vary greatly. But there are common perspectives which are shared to a great degree by all organizations. As an example, in most instances we don't want to spend more on security than we get in return (typically in the loss prevented).

If we look at an organization individually and imagine an increasing line of spending, for each point on that line we have an amount of residual loss which will be experienced (in theory, trending down to some degree as the security spending goes up) and therefore an amount of loss prevented for each point as well. At a strategic level, these three lines give us what is needed to answer this ugly question.

How much should be spent? Optimally, an organization should spend the amount of money on security which prevents enough loss to bring the residual losses to an acceptable level. What I have found, is the target exists somewhere between the low point of a diminishing rate of return and the high crossover point where the spending exceeds the loss prevented. Only management can decide exactly where the sweet-spot exists for any given moment.

Now your turn. What ugly question has been thrown in your direction?

Practical Aspects of Measuring Security