Let me start by saying that Mike Rothman is something of an infosec sage, and has the kind of resume in this field that only dozens of other people have. When he was a VP at META Group (now Gartner), I was still camping out for jsbx tickets. (OK, I was also working on this Linux HOWTO on proxy firewalls, too. But I was then and pretty much still am a nobody.) So, it is with all due respect that I am picking on an article that Mr. Rothman wrote for SearchSecurity.com. Also, to answer the question posed in the title, no, no I stand zero chance.
"Plagued by expensive and integration-heavy implementations, SIM products and vendors have never lived up to their promise, taking millions of venture capital with it."
Mike doesn't seem to be impressed with the current SIM offerings. And, to be fair, it's not all good news. Some SIM products are weak, some are expensive, and some are both, and maybe some are neither. What a SIM needs to work is access to your logs. That's pretty easy to deliver in most cases either via syslog, WMI, flat file access, SNMP, and so on. Not much integration.
"Just think: How great would it be to look at one screen, or one dashboard, and be able to pinpoint problems, maybe even before they occur?"
Hellz yeah! But be real. No vendor can drop that in your lap. Or, if they can, you probably also live in a 3-bedroom house with a white picket fence, 2.6 kids, and a dog, Mr. Generic. Let's not bag on the vendors because they don't deliver the moon. SIM is a tool, and you're going to learn how to use it or you're going to hate it.
"One problem is the overactive nature of SIM; its inputs, like firewalls and IPS devices, are inherently noisy. If the inputs are rife with false positives, it has historically been difficult for SIM offerings to provide actionable information without a tremendous amount of experimentation and tuning."
This is a straw man. If the original data is flawed, then the analysis of that data will yield flawed results. I don't see how that's the fault of your SIM product. But Mike does have a good point - tune your IDS already. It's 2007.
"Also, SIM products seem to address problems after it's too late; by the time information is correlated from log files, the attack has already happened. "
Another straw man from the Gartner IDS/IPS debate. Yeah, it sucks how SIM doesn't do something it wasn't designed to do. Actually, what sucks more is that vendors are taking this idea to heart and trying to make SIM act as an IPS, too.
"And in today's environments, where attacks can proliferate throughout the world in a matter of minutes, playing catch-up can be crippling."
This is the sentence that inspired me to post a response to Mike's article. SIM isn't there to stop attacks from happening. It's not a defensive tool, it's an analysis tool. This makes it ideal for incident response. Something bad has happened, and if all of your logs are in one place, you have a great tool for organizing and searching for related evidence, defining the scope of the response, and monitoring post-response activity.
" First, security management is increasingly being integrated with network behavior anomaly detection (NBAD), providing pseudo real-time visibility into what's happening on your network."
I agree with Mr. Rothman that analyzing netflow is a good idea. Being able to access that data from inside your SIM is an even better idea. The problem is that logging netflow with a ratio of one event (one row in your database) to each unique connection presents serious throughput and storage issues. So this is where SIM vendors should get to work integrating with FlowCollector, NetworkVantage, and other netflow collector/analyzer products. I know that some SIM vendors are going to bundle this functionality, but a SIM interface into the third-party product data is much more elegant and practical in my book than having the SIM do the netflow collection itself. I think the horse has already left the barn here, though.
I have two more random points that I want to make on the topic of SIMs and their role in security. These aren't in direct response to Mike's article, but I think they're worth bringing up here.
First, it seems that SIM products are getting pigeonholed as consoles for IDS and other network data. This is a very narrow view of what SIM can do. In fact, I would say there's more value in operating system and application logs. I know from some of the other companies I've talked to that there are SIM installs out there that do nothing but application logs. That's very cool.
Second, there's a lot of innovation in the SIM market right now, and it comes in places security folks often don't look first. Some of the coolest GUI design work that I've seen outside of windowing environments has been in the SIM arena.
Monday, March 26, 2007
On Dashboards and Pink Days
I love my wife, and one of the reasons I love her so much is because she and I can entertain ourselves in ways that must look inane and boring from a distance. Last night for example, we sat on the couch watching the late news cast of a local CBS affiliate and making fun of it the whole way. When the weather came on, she didn't let up, calling out their awful graphics, including a 7-day 'outlook' of colored bars without a legend that included several "blue" days, a "yellow" day, a "green" day, and a "pink" day. She had me laughing so hard I was in tears by the time the sports came on.
The point here is that there are lots of bad graphics out there. Bad graphics are those that don't mean anything to your audience. When you're in the audience, it's really easy to spot them. When it's your job to make them - especially if you are intimately familiar with the data behind them - it's not so easy. Good graphics stand alone. So here are a couple of tips on building dashboards:
The point here is that there are lots of bad graphics out there. Bad graphics are those that don't mean anything to your audience. When you're in the audience, it's really easy to spot them. When it's your job to make them - especially if you are intimately familiar with the data behind them - it's not so easy. Good graphics stand alone. So here are a couple of tips on building dashboards:
- Legends and Titles. Don't just have them, have good ones. They should be clear and explain the meaning of and differences between colors, symbols, etc.
- Data Points. Don't try and cram too much different data into a single chart or graph. Have one purpose per graph - for example, don't display moving averages and sums in the same area. But do have a high density of data.
- Scale. When placing graphs next to each other, trying to make the layout symmetrical can often skew the interpretation of the graph. For instance, one bar graph with peak values in the hundreds of thousands next to another bar graph with peak values in the tens of thousands will probably confuse your audience, even if there is a y-axis legend explaining this. Similarly, if data is important, showing it to scale next to other data that occurs at different orders of magnitude can hide that data or make it seem irrelevant.
Friday, March 23, 2007
Intro to SIM Deck
In February of 2006 I gave a preso at Grand Rapids' ISSA on SIM. This is a short deck - the first 11 slides - that has a nice overview on SIM concepts and what makes a product like ArcSight different from a product like eIQ FirewallAnalyzer (eIQ has their own SIM now).
The rest of the deck, which GR-ISSA members have access to, is lots of screen shots from ArcSight and a bit more about what we do internally with it for security ops. It's not really top secret, so if you'd like to see the whole deck, feel free to e-mail me.
More on SIM to come. Next week I call out Mike Rothman for suggesting that SIM can be "saved" by bolting crap onto it.
The rest of the deck, which GR-ISSA members have access to, is lots of screen shots from ArcSight and a bit more about what we do internally with it for security ops. It's not really top secret, so if you'd like to see the whole deck, feel free to e-mail me.
More on SIM to come. Next week I call out Mike Rothman for suggesting that SIM can be "saved" by bolting crap onto it.
Wednesday, March 21, 2007
Health & Human Services Is Teething
Lots and lots of people have declared HIPAA irrelevant and ineffective because of the lack of direct federal oversight and the perception that the penalties it could potentially level at an organization were weak in comparison to things like SOX.
But, OH SNAP!! The Health and Human Services Inspector General is auditing providers. I've got in my inbox a copy of the FAX sent to _____ Hospital in Georgia about their audit. No mention of complaint or prior incidence. Just a friendly, "Hi, we're coming to audit you," letter complete with data collection document.
It has always been my stance that if what's lacking in compliance is enforcement, then it's important to comply, because enforcement is only a budget line item away. So I guess I'm saying, "I told you so!" to everyone who has greatly exaggerated the rumors of HIPAA's death.
So, uh... think warm thoughts doc, cuz that thermometer is mighty cold.
But, OH SNAP!! The Health and Human Services Inspector General is auditing providers. I've got in my inbox a copy of the FAX sent to _____ Hospital in Georgia about their audit. No mention of complaint or prior incidence. Just a friendly, "Hi, we're coming to audit you," letter complete with data collection document.
It has always been my stance that if what's lacking in compliance is enforcement, then it's important to comply, because enforcement is only a budget line item away. So I guess I'm saying, "I told you so!" to everyone who has greatly exaggerated the rumors of HIPAA's death.
So, uh... think warm thoughts doc, cuz that thermometer is mighty cold.
Tuesday, March 20, 2007
Review: Chuvakin's Database Log Analysis Paper
Dr. Anton Chuvakin was kind enough to share with me a copy of his 2-part paper on database log analysis, which was published in CSI Alert Newsletter, Feb 2007. I enjoyed his paper(s), but I have to acknowledge that this is a topic near and dear to my current work. In Part I of his paper, Dr. Chuvakin gets right to the heart of database log analysis: You should do database logging better than your database does by default because it's probably a compliance issue for you.
Chuvakin also explores some of the pitfalls of database logging:
"Introduction to Database Log Analysis, Part II" lays out some practical considerations for any organization preparing to implement database logging. It's hard to keep this topic general, since the differences in audit logging between Oracle and Microsoft SQL Server, for example, are significant enough that the strategies for tackling log gathering, storage, and analysis are very different.
As you might expect from a guy in his line of work, Chuvakin goes on to describe why you might want to use a SIM or similar tool to aggregate and read database logs. I happen to agree with him that you need something more than LogMiner if you want to use your logs for anything more than troubleshooting. But Chuvakin gives a list of example reasons for using a SIM for compliance and operations purposes around database logging, not all of which wash with me:
Change management: modern RDBMS is a complicated piece of software with plenty of configuration files and other things to change. And being aware of all the changes constitutes one of the control objectives for IT governance as well as essential for protecting the environment.
True that COBIT and ITIL both rely on change management and that it is a common compliance requirement for things like SOX and SAS. However, I don't know that a SIM is a good candidate for this. Operationally, I use our change control documentation to explain, for instance, why a bunch of files were changed or new users created. I don't use the SIM's ability to detect these events as the impetus for documenting the change - the cart's way out ahead of the horse by the time the SIM gets to it, and if you do it right, change docs always precede the change. So, at best, this is a secondary control that the SIM is providing. Not that it's a bad idea, it's just not a business driver for log analysis.
Authentication, Authorization and Access: logging access control decision such as login failures and successes as well as access to data and various database objects is of interest for auditors as well as important for security, such as insider privilege abuse. While logging all access to data is less common, it is one of the emerging trends in logging.
Good idea. Do this with your directory and server OS, too. And RADIUS. And client VPN. And prox cards. And voicemail. And... oh, you get the idea.
Threat Detection: while both unauthorized changes and access control decisions are essential for security, database logs may be used to discover and analyze direct exploitation attempts as well, at least in some cases. Even accessing the database system at an unusual time will likely be of interest to as security team.
Log analysis is probably your best option here, unless you stick an IDS sensor between your database servers and everything else on your network. Which if your database supports a web app, you should definitely do.
Performance: DBAs are tasked with monitoring database performance and logs provide one of the avenues for doing so. This is especially important to those orgs with strict service level agreements (SLA) for database performance
While a SIM may be able to create averages and other statistics in near-real time, this is not the tool for database performance monitoring. There's lots of stuff out there that does this already and much better than your SIM can.
Business Continuity: knowing of database software starts and stops is essential since business depend on databases for their revenue streams and this a downed database directly leads to losing money
Again, lots of stuff that can do this better than a SIM. Moreover, SIM's use data analysis instead of active checks. If your database shuts down, a SIM can tell you about it once it sees a log entry for it. If it hangs, then maybe it will detect a drop in activity and determine that the event per second delta is abnormal. Of course, this can occur during normal use. IMHO, SIMs suck for tracking service up/down states.
Overall, I enjoyed reading this paper. It's clear that Dr. Chuvakin has some good ideas on this topic, and I would recommend that anyone who is considering database log auditing read this paper prior to the initial design meeting. It's not a comprehensive guide, but it will get you thinking in the right direction about the issues your organization will likely face.
Chuvakin also explores some of the pitfalls of database logging:
- Performance hits from turning on auditing
- A databases stores its different log data in multiple formats (in a table, in a redo file, in a text file, etc.)
- Finding data in database logs that is meaningful to security ops and incident response
"Introduction to Database Log Analysis, Part II" lays out some practical considerations for any organization preparing to implement database logging. It's hard to keep this topic general, since the differences in audit logging between Oracle and Microsoft SQL Server, for example, are significant enough that the strategies for tackling log gathering, storage, and analysis are very different.
As you might expect from a guy in his line of work, Chuvakin goes on to describe why you might want to use a SIM or similar tool to aggregate and read database logs. I happen to agree with him that you need something more than LogMiner if you want to use your logs for anything more than troubleshooting. But Chuvakin gives a list of example reasons for using a SIM for compliance and operations purposes around database logging, not all of which wash with me:
Change management: modern RDBMS is a complicated piece of software with plenty of configuration files and other things to change. And being aware of all the changes constitutes one of the control objectives for IT governance as well as essential for protecting the environment.
True that COBIT and ITIL both rely on change management and that it is a common compliance requirement for things like SOX and SAS. However, I don't know that a SIM is a good candidate for this. Operationally, I use our change control documentation to explain, for instance, why a bunch of files were changed or new users created. I don't use the SIM's ability to detect these events as the impetus for documenting the change - the cart's way out ahead of the horse by the time the SIM gets to it, and if you do it right, change docs always precede the change. So, at best, this is a secondary control that the SIM is providing. Not that it's a bad idea, it's just not a business driver for log analysis.
Authentication, Authorization and Access: logging access control decision such as login failures and successes as well as access to data and various database objects is of interest for auditors as well as important for security, such as insider privilege abuse. While logging all access to data is less common, it is one of the emerging trends in logging.
Good idea. Do this with your directory and server OS, too. And RADIUS. And client VPN. And prox cards. And voicemail. And... oh, you get the idea.
Threat Detection: while both unauthorized changes and access control decisions are essential for security, database logs may be used to discover and analyze direct exploitation attempts as well, at least in some cases. Even accessing the database system at an unusual time will likely be of interest to as security team.
Log analysis is probably your best option here, unless you stick an IDS sensor between your database servers and everything else on your network. Which if your database supports a web app, you should definitely do.
Performance: DBAs are tasked with monitoring database performance and logs provide one of the avenues for doing so. This is especially important to those orgs with strict service level agreements (SLA) for database performance
While a SIM may be able to create averages and other statistics in near-real time, this is not the tool for database performance monitoring. There's lots of stuff out there that does this already and much better than your SIM can.
Business Continuity: knowing of database software starts and stops is essential since business depend on databases for their revenue streams and this a downed database directly leads to losing money
Again, lots of stuff that can do this better than a SIM. Moreover, SIM's use data analysis instead of active checks. If your database shuts down, a SIM can tell you about it once it sees a log entry for it. If it hangs, then maybe it will detect a drop in activity and determine that the event per second delta is abnormal. Of course, this can occur during normal use. IMHO, SIMs suck for tracking service up/down states.
Overall, I enjoyed reading this paper. It's clear that Dr. Chuvakin has some good ideas on this topic, and I would recommend that anyone who is considering database log auditing read this paper prior to the initial design meeting. It's not a comprehensive guide, but it will get you thinking in the right direction about the issues your organization will likely face.
Monday, March 19, 2007
Insider Threats
It seems to me that over the past year or two there's been a refocus on the "insider threat" against your network within infosec trade press as well as the general infosec product space. I recall reading in the CSI/FBI Survey that in 2000 the Internet overtook internal systems as the most frequent source of attacks against networks. I can't fathom that that statistic has reversed, what with the number of SQL Slammer, SSH & VNC scans, etc. that constantly fill up my firewall logs. It begs the question, then, why refocus on insider threats? There could be a number of explanations, including the fact that everybody that is going to buy a firewall has bought one. But I think it's a mixed bag of good news and bad news.
The good news is that we've gotten better at border security. The number of remote-root exploits found drops each year. Mass-infection worms like CodeRed and Sasser are fast becoming a memory for most corporate network admins. Microsoft operating systems and most Linux distros automatically patch themselves now. But none of this addresses the problems you have by giving people passwords, so the "insider threat" still applies to you. Dang. Now you need to search everybody's e-mail and web traffic for corporate secrets being leaked. Or something. Anyway, that is the good news - we don't totally suck at security anymore.
Here's the bad news - your internal users are still not your biggest threat. It's still the Internet. It's spyware and adware and spam and bots and all of that crap that you can't block with firewall rules. Your users are still bombarded with this stuff constantly via e-mail and web sites, services that you can't turn off. And the malware authors have turned AV into an arms race - changing their software daily if need be to stay ahead of signature updates.
My point is that insider threats, while real, are a distraction. Addressing attacks from the inside of your network is now easier than addressing the new external threats effectively. And that's why people are focusing on it. It's not the biggest battle you face right now, but it is the one you have the best chance of winning.
The good news is that we've gotten better at border security. The number of remote-root exploits found drops each year. Mass-infection worms like CodeRed and Sasser are fast becoming a memory for most corporate network admins. Microsoft operating systems and most Linux distros automatically patch themselves now. But none of this addresses the problems you have by giving people passwords, so the "insider threat" still applies to you. Dang. Now you need to search everybody's e-mail and web traffic for corporate secrets being leaked. Or something. Anyway, that is the good news - we don't totally suck at security anymore.
Here's the bad news - your internal users are still not your biggest threat. It's still the Internet. It's spyware and adware and spam and bots and all of that crap that you can't block with firewall rules. Your users are still bombarded with this stuff constantly via e-mail and web sites, services that you can't turn off. And the malware authors have turned AV into an arms race - changing their software daily if need be to stay ahead of signature updates.
My point is that insider threats, while real, are a distraction. Addressing attacks from the inside of your network is now easier than addressing the new external threats effectively. And that's why people are focusing on it. It's not the biggest battle you face right now, but it is the one you have the best chance of winning.
Wednesday, March 14, 2007
On Web App Scanners
Dark Reading today covered SPI's announcement of AMP 3, their clone of AppScan Enterprise.
I haven't seen AMP, so for me to bag on it is a little unfair. But to me, the notion of automated scanning and reporting at the web-app level seems like a flawed idea. So to be clear, I'm not picking on AMP, but rather AMP and all of the other products like it.
I totally see the value of doing web-app security testing. But I worry that the click-n-fire scanning that these "enterprise" products offer is getting companies to pay more money for less depth by sacrificing quality for quantity. Regular scans are great, but it is my opinion that these products will find the low-hanging fruit and miss other vulnerabilities. Not because they can't find some of the more difficult vulns, but because the out-of-the-box scans come in two flavors; noisy, and incomplete. We already have this at the network level. Are companies ready to admit defeat at the app level already, too?
I haven't seen AMP, so for me to bag on it is a little unfair. But to me, the notion of automated scanning and reporting at the web-app level seems like a flawed idea. So to be clear, I'm not picking on AMP, but rather AMP and all of the other products like it.
I totally see the value of doing web-app security testing. But I worry that the click-n-fire scanning that these "enterprise" products offer is getting companies to pay more money for less depth by sacrificing quality for quantity. Regular scans are great, but it is my opinion that these products will find the low-hanging fruit and miss other vulnerabilities. Not because they can't find some of the more difficult vulns, but because the out-of-the-box scans come in two flavors; noisy, and incomplete. We already have this at the network level. Are companies ready to admit defeat at the app level already, too?
Subscribe to:
Posts (Atom)