Note, this conversation occurred in the SecurityMetrics email discussion group and is a repost of select dialogue. Thanks to all the contributors who granted me permission to post their comments.

Will the recent data breach settlement by TJX be a landmark case, setting the precedent for future lawsuits?

[http://www.boston.com/business/globe/articles/2007/09/22/tjxoffers_deal_to_end_data_breach_suit/]_

This lawsuit focused on 45.7 million credit and debit card numbers that were stolen from TJX by hackers. The company will settle the case by offering $30 store vouchers, which equates to a value of the customer's time at $10 per hour. TJX will hold a "customer appreciation" 15% sale and will also offer credit monitoring and identity theft insurance to some customers. The total costs to TJX for this incident are around $256 million.

The Math of Liability Settlements

The discussion group was alight with the paltry $30 restitution per customer.

Dan Geer shed some light on the numbers by citing a legal precedent for liability and doing the math.

Given P = the probability of loss
L = the amount of said loss
B = the cost of adequate precautions
Then Liability whenever B < PL
So, taking data from the published FTC study[2] of 2003 where they said that 4.6% of the US population had had an identity theft problem and that in solving it the affected had expended 300 million hours and 5 billion dollars, and using the then Federal minimum wage, we'd thus have:

This leads to the question of whether $30.11/yr/consumer is enough to prevent identity theft, as defined by the FTC, and if it is, then liability would ensue.
This is close enough, excluding increases in minimum wage, to the $30 figure in the press report to make me wonder if the TJX folks have been reading the same stuff I've been reading.

Impacts on Stock Price

The TJX stock has seemingly not been adversely affected.

Bill Frank noted:

I just looked up TJX stock price. It's within two points of it's all time high at $30.16. It surely dipped when the story was new. But it seems to have completely recovered.
For one of the worst security breaches of all time, it does not look like there will be any permanent damage (to TJX).

Matthew Rosenquist:

Sadly, this does not surprise me. Until the distain of such breaches becomes personally embraced by the general populace, such incidents probably will not have a significant impact. I think it will be a slow curve as society begins to alter its perspective on how data-loss events affect 'others' and begin to comprehend that it very well could and does affect them. And that they are empowered to prevent being victimized, through the simple choice of where to spend their money and whom they choose to expose their PII/PHI and financial records. Only then will it change spending habits, investing choices, and ultimately begin a cascade effect with the economy directly surrounding organizations which allow, through ignorance or indifference, such losses.
Today is a sad day, but tomorrow will be a little better as the pain will continue to grow and slowly manifest change in the herd.
After some posts recommending more governmental regulations I threw out a couple of points:
1. I believe the free market system, with its inherent checks and balances, will prevail. But the key is fixating on the real issue: Money. Follow the money.... How much will this cost the TJX consumer? How much higher prices will the need to pay for the mismanagement by TJX officers? This is the real metric (IMHO). This will determine the velocity by which the curve will occur (see my previous ranting on this thread). bq.
2. The math (disclaimer: will someone with a bigger brain check my numbers, which are ballpark anyways - just for illustration purposes):
TJX estimates total losses for the security incident: $256M
TJX estimated Sales Revenue: $18000M
TJX estimated Sales Net Profit: $738M (I chose to use Net instead of Gross, but use whatever you believe is right)
TJX estimated profit margin: ~4%

In order to recoup the $256M in Net Profit, they would need to sell an additional $6400M in product ($256M / 4%), or INCREASE prices by ~25% without selling more. For those TJX customers, are you okay in eventually paying ~25% more for the same products, due to poor management practices of the retailer? (Yes, it is the decision of the management to decide how much they want to recoup, but you get the point).
...yes these are rough numbers, for discussion purposes only. The point is somebody has to pay. It will be the customers. Let's have a bright person do the math and show the customers what they are going to have to eat, as part of the cost of doing business with TJX (substitute company name of any organization who allows a data breach).

Bill Frank:

Matthew, the only metric that really counts is the stock price.
I see your math if the point is to recoup the money lost. But too often the stock price ignores one-off events. The point is that the stock price has recovered even though they lost $250 million because the incident is seen as a one-time event that will not have any effect on earnings going forward.

Matthew Rosenquist:

Bill, you make a good point. My contentions are that due to a lack of realistic and understandable metrics both the consumer as well as investor does not have sufficient data to comprehend the future ramifications, hence the propensity of classifying these issues as one-time events. Which time will prove, they are not. Basically, the customer and investor do not know how to react. They are pensive due to a lack of understanding and experience. We are all on a path of learning. Empowering people with insights, understanding, and a strategic view is the role of metrics. In this case, I see the true power of metrics as a tool to help escalate the learning curve. I believe sometime in the future such a breach would cause significant backlash by the consumer and reflect in the stock price. We just are not there yet.

Anton Chuvakin:

I feel that there is something very wrong with this math... just not sure what exactly. My guess is that if you increase your price by 25% in this business, you'd be gone within a quarter (see narrow margins, cutthroat competition, etc) So they probably won't. Can somebody then explain, who pays?

Matthew Rosenquist:

Yes, there is something wrong, but I use it for illustrative purposes only. The missing link is the decision by management on how much loss they are willing to accept. If they choose to eat the entire $256M, then they do not need to raise prices at all. On the other end of the spectrum, if they want all $256M back, then they have to raise prices. An increase by ~25% for one year would come close, although realistically, they would spread out the pain over several years so as to be only a slight increase over a longer period of time.
The key is what management decides, either consciously or unconsciously, to be an Acceptable Loss.
Note: I grabbed the company's financial data, including the margin figures, from yahoo.com/finance

Susan Bradley:

But isn't the free market system working now? The one that has Russian/Asian hackers/Spammers/Phishers sneaking into our servers, causing breaches now working quite nicely now?
Look at the free market system of software (and I'm not talking Microsoft here). Show me an accounting application that natively has encryption surrounding the PII data in it? Granted I hang in the SMB space, but do you guys in enterprise see movement up there or am I just not looking in the right places for vendors making changes reacting to PII losses?
If the free market system was working ...then why does my Bank of America have computer terminals that look like DOS on their desktops? Of course then again why am I still banking at them and not moving to Wells Fargo where they are at least running Win2k last I looked? Aren't I guilty of not shopping for the most secure bank when BoA lost a few PII here and there? I haven't taken my business elsewhere as a result. Shouldn't I?
I myself am guilty of this "bare minimum" view as I was on a virtual committee for the 'minimum' security standards for all sized entities organized by CISecurity.org and I couldn't (wouldn't) push for two factor authentication being a defacto standard since I didn't feel that the industry was mature enough to be a standard yet for SMBs.
So while the free market industry for the spammers, phishers, etc seems to be quite robust, are the applications responding to the free market of checks and balances?

Matthew Rosenquist:

I believe the system is working, albeit not as fast as we all would like. As proof, we have dramatic changes and tension in the system. Neither side (good guys/bad guys) is completely winning but both are rapidly changing and evolving. The information security industry has skyrocketed in the past 5 years. So has cyber crime. In this dance each side is looking for advantages and continually adapting to their respective opposition. Change is afoot. Other areas of cyber security are much farther on the maturity curve than privacy and data breach security.
Security will continually seek to mitigate losses in the most cost efficient manner. In doing so, the industry will change as well as the expectations of security. In the end, we are not trying to make everything impervious to attack, instead we are seeking to achieve and maintain the optimal level of security which balances the cost of security with the loss prevented to reach an acceptable level of loss. This is a wildly gyrating target as new vulnerabilities, threats, changes to environments, etc. are constantly changing. Adaptation is in small steps. I doubt we will wake up tomorrow to have every application using encryption. The cost is just too high and we would be overshooting the optimal level of security. Eventually however, the most critical applications will use encryption.

Are you looking for that special gizmo in a box which will provide your organization a warm blanket of security? Buy it, plug it in, and viola! You are now secure. Fold up the tents and walk away, the job is done. Well, keep looking. Regardless of what some security vendors peddle to uninformed IT managers, it simply does not exist.

Security is an on-going process of diligence. The simple fact is, as long as the environment being protected changes, and the threats to that environment look for ways to take advantage, security must also adapt. No one product sufficiently spans the current and potential spectrum of attack vectors, nor does any one solution cover all aspects of technology and behaviors which may be exploited.

The booming growth of security products over the past few years can partly be attributed to organizations dumping money into the market. A common mistake of many senior IT managers was to invest bags of money under the false belief it was a one-time expenditure. As if security could be purchased in a box, installed, and the issue resolved. Especially in IT departments, people new to the realm of security apply IT thinking to the ‘problem' of security, expecting to find an engineering solution ‘fix' so life can move on. I can't blame them really, as most technology minded people deal with obstacles rather than opponents. An obstacle can be overcome. Engineers are great at going over, under, around or through obstacles. Find the right technology, gadget, toy, process, or application and the problem is solved so the diligent IT person can move on to the next obstacle.

Opponents, not obstacles

Well, security is not about obstacles, it is about opponents. Every security threat can be traced back to a person. That person, if malicious, has an agenda and an objective. Put an obstacle in their way, they will find a way to counter or go around it in the pursuit of their objective. In fact, the behavior of attackers is usually predictable, as they follow the ‘path of least resistance' to achieve their objective.

If you treat an opponent like an obstacle, you will be fighting a never-ending set of losing battles. One hole is plugged and the opponent simply adjusts to the actions and comes at you from another direction. It can degrade into a battle of attrition. The defense in this manner can only hope they ‘fix' enough things to make the attacker move on to another target. However, the cost of each ‘fix' is much greater than the cost for the attacker to adapt. For a dedicated attacker, the odds are in their favor, unless the target is willing to spend an inordinate amount of time and resources to continually fight the ‘obstacle' battle in hopes that eventually the attackers will tire or find an easier target.

I plan on going more in depth on this Attacker -> Methods - > Objective model in another blog and may go into great depth in a whitepaper, time permitting. Traditional IT thinking, when applied to security, is an endless treadmill consuming time and resources.

Feel the Pain

Be careful what you wish for. If senior management maintains a simplistic view of security, then many problems are sure to follow. Time to bring on the pain. Choosing to adopt the deceptively straightforward �obstacle� defense is an unpleasant education in futility as new issues quickly replaces ones just remedied. It is both costly and frustrating. Losses begin to tally and security spends increase as the organization is stuck in a routine of responding to each new type of attack. Management can get very aggravated at the continuing expense and interruption with such a poor strategy. From the perspective at the top, it is easy to blame the security staff and not obvious the lack of a comprehensive security strategy is the real culprit.

In this cycle, it is a safe bet management would not comprehend the strategic need to identify an optimal balance of security. Such viewpoints tend to distill the situation to a binary state, either the company is secure or it is not. Trying to argue a gradient or any other perspective may fall on deaf ears. Expect the commitment to be limited to short term security expenditures and no allowance for much in the way of sustaining costs or future additional costs necessary to mitigate new threats. Budget discussions can be frustrating; with management expecting a dramatic decrease in future security spending while those in the trenches are struggling just to maintain effectiveness against new types of attacks. The lure of an easy solution or product is very tempting, but nothing more than a mirage which distracts leaders and reinforces an overly simplistic way of thinking, leading the organization down a path of inadequate preparedness for sustaining needs of the future.

On the converse, if an organization maintains the perspective that an ‘opposition' exists, then an entirely different game is played. One which can be won or at least managed efficiently. The organization can implement a thorough defense-in-depth strategy which starts with Prediction. Predicting the opposition's objectives, capabilities, and most likely methods is the first step in applying a cost effective structure to Prevent, Detect, and Respond to attacks.

Cost of the Magic Box

If your organization is looking for the magic security box then it is suffering from the ‘obstacle' way of thinking. This will be costly. The security programs implemented under this way of thinking will most likely be rigid and have a short effective shelf-life. Many security initiatives will be in response to successful attacks and will be rushed into production. Stacking an increasing number of independent solutions weighs heavily on the computing infrastructure, complicating the very environment it is trying to protect, and sets in motion steadily increasing sustaining and support costs, with no end in sight. Bleak to say the least.

Management perception and strategy are very important aspects when evaluating the value of security programs. Security is not a snap-shot in time. Sure, buying a flashy product may fix a specific problem which cropped up, but the long term costs must be factored in. Will this product ever be End-of-Life'ed? Is their a different product which not only closes this gap in security but also provides broader protection against future issues? What are the real operating and sustaining costs? Will the product be maintained by the vendor and continually upgraded to address new threats?

The bottom line

When measuring security it is important to understand the threats, solutions, as well as the organization which everything will be applied. With all other factors equal, the value of a security product is greatly different in an organization with a comprehensive defense-in-depth strategy, versus an organization with a haphazard strategy with non-integrated solutions. No one product or service does it all. The attackers are dynamic and will adapt to an organizations defenses. Understanding the concept of ‘opposition', even embracing the idea, will thrust your organization ahead in this game.

Practical Aspects of Measuring Security

Security in a Box

The Four Dirty Questions of Measuring Information Security

Managing the Effort to Measure Security

In this audiocast, information security analyst Tim Casey talks about three tools used to help manage risks to sensitive information: risk assessments, risk modeling, and standardized threat agent characterizations. Along with other tools and methods, these three play an important part in managing Intel’s information security profile.

</embed>

I love tools. I have a whole garage full of them. Big ones, small ones, ones with wicked sharp edges, ones for removing tiny splinters from fingers, and a few really heavy ones. My wife always wants me to clean some out, but how can I handle all the things that need fixing without a full tool compliment? I especially like the power tools. Nothing says “massive amounts of impressive work” like the shouting-loud whir of a 3/4HP tool tearing through a piece of metal.

It occurred to me recently (while power-driving 3" nails into a joist for a new support) that in my work in information security at Intel, I need power tools, there, too. Information security used to mostly mean adding passwords to accounts and stamping sensitive print-outs “Secret” —essentially, we could get by with just some simple security hand tools. Now we are dealing with increasingly complex environments, and increasing sophisticated attackers, so we need better and better tools to keep our information safe. Network scanners, intrusion-detection devices and the like are essential, but we also need tools that help us understand the big picture when it comes to overall information security risk. These risk management “power tools” help anticipate problems and concentrate limited security resources where they are needed most. The three I use most often are risk assessment, risk modeling, and our new Threat Agent Taxonomy & Library.

For more in-depth information, check out my new white paper Threat Agent Library Helps Identify Information Security Risks.

These are very useful tools, but as I mentioned, there are plenty more. I’m curious as how much others use these techniques, as well as what other risk management tools or methods you are using. Are they home-grown or off-the-shelf? Are there any special adaptations you needed to make for your environment? Put on your safety goggles and let me know what infosec power tools you are using.

In my experience over the years, calculating security value and providing consulting to others doing the same, I have noticed the same 4 questions tend to rear their ugly heads. Requests by senior managers, finance analyst, business value analysts, project and program managers all fall into one or more of these types of inquires. And when I say they are ugly, oh they are.

In most cases the parties seeking information are in some phase of the decision cycle:

Should I spend money on security? - This is a business decision based upon compelling drivers, usually loss of some kind, including non-compliance to regulatory requirements (which could send a C-level officer to spend an extended vacation at Club Fed) or risk of a catastrophic blunder sufficient to crater the organization. The business aspects must include how many coins are in the coffers, amount of loss (both realized and unrealized) on the table, and if money could be better spent elsewhere (opportunity costs)

How much should I spend? - A value decision considering what the organization is willing to accept in losses, what can be spent on security, and the amount of loss which could be prevented. Optimally, there exists a point at any given time which management is willing to spend a certain amount on security, which prevents enough loss to bring the residual losses to an acceptable level.

What should I spend it on? - An exercise in comparative analysis of available options which drives down overall costs, while increasing the losses prevented, and maintaining the optimal level of security and residual loss.

On to the ugly questions (feel free to share your experiences):

Ugly Question #1: How do I select the security product/program with the best value?

This is typically asked by senior management or by a product/service manager seeking to identify the best solution among a pool of several competing initiatives. As an example, they might be looking to purchase an Intrusion Prevention System (IPS) and looking for the best of breed. Conversely they may be looking to establish or improve a security capability (example: data protection) and trying to determine the best product among multiple solutions (encryption, IPS, document tracking, data destruction policy, etc.) across multiple vendors.

The challenge is to be able to compare which solution will best achieve the optimal level of security. This is a function of security cost, losses prevented (effectiveness), and acceptance of residual loss. To simply go for the cheapest, most effective, or fastest to adopt is most often than not, the wrong long term answer. (..and security is a long term proposition)

Ugly Question #2: What is the value of this security product/program?

This is asked by management and project managers when a solution is in the proposal stage, by the sustaining operation folks once it has been implemented into the environment, and by management during times when the organization is looking for opportunities to cut costs. As value is a dynamic concept, it can radically change based upon business, legal, and social aspects as well as the normal fluctuations in the threat landscape. First step here is to identify what types of value was intended to be provided and the appropriate metric to measure those aspects.

As an example, management may be seeking to protect the organization's image and liability from the loss of Personal Identifiable Information (PII) through the implementation of a hard drive encryption program for company laptops. The metrics may be as simple as determining the saturation of the program and if encryption is sufficient to protect from liability in the geographies they do business in. In this manner you can estimate the amount of coverage for which liability and image concerns are abated.

You might think, wait, that is not a dollar figure! Where is the value? Well, in this case, management may be looking for the establishment of a capability. Either we are protected from this threat or we are not protected. The same stratagem could be compliance with HIPPA or other regulations. To attempt to quantify a dollar figure in this example would be overkill and may detract from what is intended. Realistically, a dollar savings cannot reasonably be calculated now matter what kind of magic hat you possess. I have seen some attempts, by people with the best intent, to do this very calculation. But not knowing if or when or to what extent a loss may occur, nor to be able to truly measure the potential losses due to the large number of unknown variables which have an astronomical range of potential damage, these assessments are pure folly (but really fun to poke holes in). Half the battle in measuring the value of security is to know what limitations exist regarding the granularity of what can realistically be measured and validated.

Ugly Question #3: How do I compare the value between security and non-security initiatives?

This one bites. Really. It is almost impossible to do, anyone can challenge the results, and if you get this wrong everybody hates you. This comes up when senior management must decide where to spend hard earned budgetary dollars. It becomes an "us versus them" battle between security and some other group. Each party wants the money to spend and the infighting can get downright dirty. So what is a manager to do? Just tap your friendly neighborhood security analyst to calculate the value (just as long as it is not me), then compare against the value of the non-security program. Easy, right?

I wish. Security programs rarely have the benefit of real dollar justification attached. Unless you are in the security products/service industry, security does not generate revenue, it is just overhead. More on that in a different blog. Non-security programs have the edge here. A marketing program may generate XX dollars, an operations efficiency program may save YY downtime or be able to cut ZZ heads from the budget. These strong arguments bark loudly to management. Security value will retort with a whimper, maybe a risk reduction of xx% or at best a loss prevented of yy dollars. Did I mention even calculating such values takes more time, with more assumptions, and can't, in most cases, ever be validated as compared to the non-security programs? Pure ugliness. Alas it is not impossible. I have seen the fight won (ie. management given accurate and comparable data to make the best decision), but be beware, the deck is stacked against security.

Ugly Question #4: How much should my organization spend on security?

This is the big-daddy of questions, posed by senior management or if the organization is large enough, by a divisional head. Although I plan on discussing this in greater detail in another blog and whitepaper, the path to take is to identify the optimal level of security.

Every organization is different with ever changing business needs and drivers. What one company desires from its security program and is willing to spend will differ from its neighbor. The willingness to accept different levels of loss also vary greatly. But there are common perspectives which are shared to a great degree by all organizations. As an example, in most instances we don't want to spend more on security than we get in return (typically in the loss prevented).

If we look at an organization individually and imagine an increasing line of spending, for each point on that line we have an amount of residual loss which will be experienced (in theory, trending down to some degree as the security spending goes up) and therefore an amount of loss prevented for each point as well. At a strategic level, these three lines give us what is needed to answer this ugly question.

How much should be spent? Optimally, an organization should spend the amount of money on security which prevents enough loss to bring the residual losses to an acceptable level. What I have found, is the target exists somewhere between the low point of a diminishing rate of return and the high crossover point where the spending exceeds the loss prevented. Only management can decide exactly where the sweet-spot exists for any given moment.

Now your turn. What ugly question has been thrown in your direction?

Practical Aspects of Measuring Security

Measuring information security is an exercise in total frustration. Well, maybe not total frustration but it can increase the number of wrinkles in the face, thin the hair, and turn what is left to a lighter shade of gray. Eventually, everyone taken with this passion will sport the Einstein look.

So what is the big deal anyways? How is measuring security programs any different than other IT or production programs? The heart of the problem is in trying to measure what does not occur. Security initiatives strive to prevent loss. So in effect they try and make something not happen or to lessen the outcome. And if something does not occur, how can you measure it?

The security drums. Every company should have a set:

I walked into the office to find our security operations analyst beating on drum, working hard to keep a rhythm.

I asked him what he was doing and he replied "I am beating the new security drum to ward off the computer viruses. Management just bought it from the vendor and they say it adds another level of protection."

"Is it working?" I asked.

"I'm sure it is, we have not had an infection all morning!"

Just then the security manager walked by and reported two new viruses were detected on the network and offered this advice "beat faster!"

Many falsehoods exist. In my days I have seen many wildly inaccurate, bordering on pure fictional, value assessments for security programs. Every security vendor has something to show, but none can answer the simple question: how much loss will this prevent. As the threat environment is so chaotic, is a reduction in losses due to security programs or just a simple drop in attacks? Does management understand the challenges or are they reinforcing illogical behaviors and still expecting miracles? And what should a security program achieve?

These and many more questions I intend to delve into by theorizing, discussing, tempering, and ultimately shedding light on the frustrating topic of measuring information security. Anyone want to come along for the ride?

The Four Dirty Questions of Measuring Information Security

Practical Aspects of Measuring Security

Managing the Effort to Measure Security

Security in a Box