Monday, June 23, 2008

Useless Statistics: Nate McFeters vs. Verizon

You know how much I love to tear into vendors whose studies and data analysis wouldn't pass muster for a high school statistics course. It's nice to see someone else go off for a change. Nate McFeters, Ernst & Young security dude, ZDNet blogger, con regular, and fellow Michigan native has taken issue with the results of a study on data breaches that Verizon published earlier this year.

So let's just get this out of the way:

The first thing you’re thinking is, “Wow, my consultant has been lying to me about internal threats!”, the thing is, that’s not necessarily true.

Yes it is. "Insider threat" is a red herring throughout security, but especially where data breaches are concerned. There's no breach notification law out there that defines a breach where the data ends up in the hands of someone that already works for you. Since there's no external force requiring companies to track these incidents, it's probably very safe to assume that tracking and detection of these is low, except within a handful of specific verticals.

To Nate's point about the wording of the survey and the study, I agree - it is dangerously ambiguous. However, it's probably not the cause of the improbable skew toward external attackers in the survey data.

I think I know what is. See, Nate's thinking about the Verizon study like a pen-tester, and forgetting that most data breaches and security compromises don't involve vulns and sploits, just the interesting ones. Sometimes they involve phishing, but most of the time they involve simple impersonation (the FBI calls it 'identity theft').

The thing is, if you follow basic authentication principles and practices around your self-service web apps, this stuff is hard to prevent but easy to detect and resolve. And this is how you get the disparity between the number of breaches and the amount of data breached, in the statistics. Most of those "external attacker" scenarios were someone's kid, deadbeat brother, or ex-wife impersonating them to get at their information. Not good. But not interesting.

Seriously, I don't know what it is, but it's almost always divorced/divorcing couples involved in these impersonation breaches. Nate, if you want to interview me about this some day, I've got some great stories.

Anyway, use Verizon's survey for what it's good for - getting more security funding. Because bottom line, that's a lot of breaches, no matter the circumstances.

3 comments:

Nate McFeters said...

Hey Paul,

Just saw this post. Let's chat about it sometime... always good to find another Michigan native involved in security. My long-time conference co-conspirator Rob Carter is as well.

Hit me on my email: nate.mcfeters@gmail.com

-Nate

Tim said...

Richard Bejtlich's review is more positive, and picks out different tidbits: http://taosecurity.blogspot.com/2008/06/verizon-study-continues-to-demolish.html. But I agree with Richard that the insider-threat is over-hyped. I put the hypothesis this way: "all undetectable threats are insider threats." :)

PaulM said...

@Nate:

Thanks! Yes, let's definitely talk about this stuff some time. I think there are some interesting causes for some of the statistics we see around data breaches, and if nothing else, I have a few funny stories to tell.

@Tim:

That Bejtlich link seems to be his response to a different study, also conducted by Verizon, about patching. But on that same Verizon blog, one post prior, is this. Which, like I said, points to far fewer instances of "insider threat" attacks. These numbers are similar to the CSI/FBI survey trends, I should add, which suggests that the relatively low frequency of internal attacks is repeatable across studies.