Monday, March 26, 2007

Me vs. Mike Rothman: Could I Possibly Win?

Let me start by saying that Mike Rothman is something of an infosec sage, and has the kind of resume in this field that only dozens of other people have. When he was a VP at META Group (now Gartner), I was still camping out for jsbx tickets. (OK, I was also working on this Linux HOWTO on proxy firewalls, too. But I was then and pretty much still am a nobody.) So, it is with all due respect that I am picking on an article that Mr. Rothman wrote for Also, to answer the question posed in the title, no, no I stand zero chance.

"Plagued by expensive and integration-heavy implementations, SIM products and vendors have never lived up to their promise, taking millions of venture capital with it."

Mike doesn't seem to be impressed with the current SIM offerings. And, to be fair, it's not all good news. Some SIM products are weak, some are expensive, and some are both, and maybe some are neither. What a SIM needs to work is access to your logs. That's pretty easy to deliver in most cases either via syslog, WMI, flat file access, SNMP, and so on. Not much integration.

"Just think: How great would it be to look at one screen, or one dashboard, and be able to pinpoint problems, maybe even before they occur?"

Hellz yeah! But be real. No vendor can drop that in your lap. Or, if they can, you probably also live in a 3-bedroom house with a white picket fence, 2.6 kids, and a dog, Mr. Generic. Let's not bag on the vendors because they don't deliver the moon. SIM is a tool, and you're going to learn how to use it or you're going to hate it.

"One problem is the overactive nature of SIM; its inputs, like firewalls and IPS devices, are inherently noisy. If the inputs are rife with false positives, it has historically been difficult for SIM offerings to provide actionable information without a tremendous amount of experimentation and tuning."

This is a straw man. If the original data is flawed, then the analysis of that data will yield flawed results. I don't see how that's the fault of your SIM product. But Mike does have a good point - tune your IDS already. It's 2007.

"Also, SIM products seem to address problems after it's too late; by the time information is correlated from log files, the attack has already happened. "

Another straw man from the Gartner IDS/IPS debate. Yeah, it sucks how SIM doesn't do something it wasn't designed to do. Actually, what sucks more is that vendors are taking this idea to heart and trying to make SIM act as an IPS, too.

And in today's environments, where attacks can proliferate throughout the world in a matter of minutes, playing catch-up can be crippling."

This is the sentence that inspired me to post a response to Mike's article. SIM isn't there to stop attacks from happening. It's not a defensive tool, it's an analysis tool. This makes it ideal for incident response. Something bad has happened, and if all of your logs are in one place, you have a great tool for organizing and searching for related evidence, defining the scope of the response, and monitoring post-response activity.

First, security management is increasingly being integrated with network behavior anomaly detection (NBAD), providing pseudo real-time visibility into what's happening on your network."

I agree with Mr. Rothman that analyzing netflow is a good idea. Being able to access that data from inside your SIM is an even better idea. The problem is that logging netflow with a ratio of one event (one row in your database) to each unique connection presents serious throughput and storage issues. So this is where SIM vendors should get to work integrating with FlowCollector, NetworkVantage, and other netflow collector/analyzer products. I know that some SIM vendors are going to bundle this functionality, but a SIM interface into the third-party product data is much more elegant and practical in my book than having the SIM do the netflow collection itself. I think the horse has already left the barn here, though.

I have two more random points that I want to make on the topic of SIMs and their role in security. These aren't in direct response to Mike's article, but I think they're worth bringing up here.

First, it seems that SIM products are getting pigeonholed as consoles for IDS and other network data. This is a very narrow view of what SIM can do. In fact, I would say there's more value in operating system and application logs. I know from some of the other companies I've talked to that there are SIM installs out there that do nothing but application logs. That's very cool.

Second, there's a lot of innovation in the SIM market right now, and it comes in places security folks often don't look first. Some of the coolest GUI design work that I've seen outside of windowing environments has been in the SIM arena.


Mike Rothman said...

I don't know any Mr. Rothman. Folks just call me Mike. :-)

Thanks for the comments and the dialog. In a roundabout way, you validated the point I am making about SIM. It doesn't solve the customers problem. At least not the problem that the folks that sell the solution think it does.

And most SIM packages mess with the log data. That's a no-no when you are undertaking an investigation. To me that's the difference between a log management offering and a SIM.

Paul, it's not about winning or losing. It's about having a good dialog.


PaulM said...

"SIM can be useful for incident response, BUT ONLY IF YOU DON'T MESS WITH THE RECORDS. Any kind of normalization, data reduction or anything else is a no-no. You mess with the data, it ceases to be evidence."

If you want evidence to be "court-ready," then having access to the originals is necessary. But chomping logs - or more accurately, copies of logs - and inserting them into a database doesn't render them useless. Far from it. But maybe we're missing eachother on semantics here. For instance, from my SIM console I can open a single IDS alert and view the original payload in hex or ascii. So maybe it's all about depth of features.

My point, in case I buried it in my original rebuttal of your article, is that SIM can streamline incident response and investigation by putting lots of data from different platforms and sources in one place where it can be searched and compared. But also, your company's log data and operating environment are different enough that you can't buy a SIM that will find all the jewels and only jewels without some work on your part.

And I enjoy very much the dialog on this topic. Let's do it again some time. :-)