Security Metrics and Data Visualization

I've just finished compiling the security incident handler case statistics for 2009. This is the second year in a row that I've used the same set of metrics, and having two years worth of data has led to some interesting observations about security trends within my employer's environment.

One set of statistics that may be of interest to the general Internet public is the volume of malware cases that we have worked over the past two years.



There are a couple of things worth pointing out in this graph. The first, and perhaps most obvious one, is that there is a drop-off in malware related cases in 2009. Surely, that can't be right? It is, but it's due to implementing some new security technologies in December of 2008. In fact, those countermeasures reduced the number of malware cases we handled in 2009 by roughly 65% compared to 2008. I want to say two things about this. First, this demonstrates the effectiveness of the preventative countermeasures that we employed and confirms the value of those countermeasures. Notice that I'm not saying that it proves ROI. But the bottom line is that it was worth it. The second thing I want to point out about that decline, however, is that it's just a decline. It did not eliminate the problem. In fact, in 2009 we saw malware chip away at other defenses that were highly effective only two years before. And I suspect that, if we do nothing else about it, that those levels will begin to rise in 2010 and regain the same level of frequency we saw in 2008 if not higher. There's a hint of that in the graph towards the end of 2009.

The next thing I want to point out about this graph is the peak frequency. It is consistent. Every three months, there is a spike in malware incidents in our environment. I would love to see statistics from other companies or the Internet at large to see if this is an Internet-wide pattern. I suspect that it is. Despite the new countermeasures, despite the decrease in order of magnitude, the spikes occur like clockwork every third month. That leads me to believe two things. First, I believe that this pattern is driven externally since it didn't deviate, even when our environment changed significantly. Second, I believe that this is no accident. The vendors that produce malware/botnet "kits" are responsible for introducing most of the new exploits and anti-detection capabilities that we see on a regular basis. Their stuff is used more widely than custom malware as well. Therefore, this leads me to believe that there is one large group responsible for the majority of the malware in the wild, and they're on a 90-day release cycle. I've got no intelligence data to support this, but I have a hard time believing that this pattern repeats itself, without exception, for two years straight out of pure coincidence.

Bottom line, this is the kind of useful information that trend analysis can give you, and why metrics are worth gathering and analyzing.