The Bride of Useless Statistics

It's Friday, so that means it's time to pick on another Dark Reading story. I've decided to dust off an old theme and once again confront bad statistics in the press.

Tim Wilson's story, "Insider Threats Increase, But Damage Is Minimal" and the CSI/FBI Survey that he sites describe the losses from incidents involving insiders as being significantly lower than those caused by outsiders. The problem here isn't really the math as the survey itself, which , in my opinion, incorrectly categorized a number of incident types and then averaged all of their costs, skewing the data significantly.

From Tim's article, "Insider abuse of Internet access was the most frequently-cited incident among the CSI survey respondents, at 59 percent. Fifty percent cited the loss or theft of laptop or mobile devices, while 25 percent cited misuse of instant messaging services."

So they counted up all of the losses from people trying to surf porn, lost equipment, and used chat software in violation of company policy. And then they lumped them in with, "Another 25 percent said they had experienced "unauthorized access to information" in the past 12 months, and 17 percent said they have suffered loss or theft of customer/employee data."

So a smaller subset of the incidents described involves malicious intent, not just boorish behavior and bad luck. And as a result, the losses aren't very big on a per-incident basis. But comparing unauthorized chat sessions with electronic embezzlement is apples and felonies.

I also can't help but notice that lots of this stuff is of the easy-to-detect variety. Sure, by now you should be able to catch users trying to access Internet sites that aren't appropriate for a professional setting or know when a laptop goes missing. But of those that responded to the CSI survey, how many even possess the ability to detect when an administrator, accountant, or executive siphons off valuable corporate data and sells it? Data that they are authorized to access?

How will you detect the breach? How will you know unauthorized disclosure occurred? How will you calculate financial losses from it? I would guess that most of the respondents to the CSI/FBI survey can't answer all of those questions in a way that would satisfy their corporate leadership. Hell, I'll bet their vendors can't answer those questions, either.