Monday, September 15, 2008

Managing ArcSight ESM Tools

Last week Tim and I presented at the ArcSight user conference on using Tools in the ESM console to augment incident response and investigation. I'm hoping to have the sanitized slide deck up this week, and maybe a little bit of code to go with it.

At the end of the talk - and then a couple of times in the hallway - people asked how we manage all of these Tools. It's a great question, and the answer is, not very well. But here's the best way that I know how in ESM 4.0.

If you've ever looked at the tools editor in the ESM console, you've seen this dialog, which is pretty basic:

By default, this info is recorded in your AST file (C:\arcsight\Console\current\paul.ast), which is just a text file with a bunch of values declared. The values in the file look like this:[toolName].program=[toolName].workingdir=[toolName].iconFile=[toolName].parameters=[toolName].showInToolBar=[toolName].isExportTool=

And then there's this:


Which is pretty self-explanatory. It occurs once in the AST file and is just a CSV list of the tool names, used to populate the console's Tools menu.

I wrote a Perl program that can parse an AST file and extract tool data from it for the purposes of sharing. It's designed to help scale and distribute tools across your analysts' consoles so that they don't have to manually recreate and test them. In order for it to be useful, there are some best practices. Here's what I do.

1. Cygwin (Surprise!) If you've been reading my blog for any period of time, you knew this was coming. If your analysts use Mac OS X or Linux for their ESM console platform, not to worry. They can play, too. Beyond, Perl, bash, and Python, that's kind of the point.

2. Standardize on a source directory for scripts. Put all of your scripts in the same spot. Pathing is difficult to manage by hand, so by defining a standard (I use /usr/local/bin/arcsight), you have less to do each time you distribute a new tool.

3. Use a repository like Subversion or CVS for scripts and other tool artifacts. That way, you can make a change to your tool, check it in, and the other analysts can check it out quickly and easily. No messy manual copies. Also, when you foul something up, you have revision history to go back to. That can be a life saver if you are - like me - not a developer with good testing habits.

4. Use consoleupdates.txt on the ESM manager to distribute tool configs. Here's how you do that:

Let's say my ESM user id is 'paul' and I have developed a whole bunch of tools following the first three rules above. I can use this Perl script to create an export of the tool configs for use on the server. It looks like this in Cygwin:

$ export all /cygdrive/c/arcsight/Console/current/paul.ast > ~/consoleupdates.txt
$ scp consoleupdates.txt arcsight@esmmanager:/opt/arcsight/manager/config
$ ssh arcsight@esmmanager
arcsight@esmmanager:~ $ chown arcsight.arcsight $ARCSIGHT_HOME/config/consoleupdates.txt
arcsight@esmmanager:~ $ chmod 644 $ARCSIGHT_HOME/config/consoleupdates.txt

If you want your analysts to get the updated tool list, they need to log out of the console, move their AST file somewhere safe, and log back in to the console. No restart of the manager is necessary. Now they just need to create the same script directory you did and check out your scripts from your CVS server.

The Perl script I wrote also supports listing the tool names in an AST file as well as exporting single tool configurations, so it's more than just a one-trick pony. It's got another half a trick.

My advice at this time is not to invest a ton of time in doing this unless it's a weekly headache for your security team, but it is worth doing if you've already got the moving parts in place (like a CVS server). The reason is that ArcSight has already fixed the issue of sharing tool configs in ESM 4.5. So once that's released (later this year?), some of this will be a non-issue for you. I suspect that the first two or three best practices I list above will still be valid in ESM 4.5, so it's still a valuable exercise if you have any number of custom tools already.

Tuesday, September 9, 2008

ArcSight User Conference 2008

I'm on the floor of the ArcSight "Protect '08" conference this morning. Tim and I gave our talk on ArcSight ESM Tools yesterday, and I will post some version of those slides and some of the code after I return from the conference.

Right now I'm listening to Hugh Njemanze give his keynote on product lines. There's a lot of interesting stuff in the release pipe; Logger 3.0, ESM 4.5, a new Connector appliance, IdentityView content for ESM, and something called "McLovin."

Anyway, here's what's been good so far:
  • Customer presentations (other than mine, I mean) - I missed out last year, these are the best talks so far.
  • Location - the new hotel is within walking distance of stuff (and by stuff I mean not trees and the NSA.)
  • Networking - Always the best part of this conference. I love standing around with free beer, talking to other folks about what they're doing with their SIM, and sharing ideas. Looking forward to more tonight.

Here's what's been not-so-good:
  • Wireless - the hotel wireless has been unreliable and overloaded. Frankly, I'm surprised I've been able to stay on long enough to get this post up.
  • Vendor/sponsor floor - no offense to these guys, but the freebies this year are unimpressive. I've already got a pen, thanks.
  • No bag - Instead of a "conference bag," everyone was issued a plastic file folio thing. Not that I needed another bag, but I can't smoosh the one foam squeezy thing I did get from a vendor booth into this blue plastic thing.

And I would be remiss if I didn't drop a product scoop or two:
  • Logger 3.0 has adopted a more-ESM-like boolean filter interface. Big improvement over the chained-regex search in 2.5 and earlier.
  • Demo of Logger 3.0 shows that searches of data (no details on data set) are roughly 80x faster than a similar sized search on 2.5. (The claim is 100x faster, but I counted. Still, that's a significant improvement.)
  • Hugh has hinted that the slick, high-performance append-only storage stuff that Logger has is going to be integrated into ESM in some release beyond 4.5. That could mean the end of the Oracle / PartitionArchiver storage model. It won't be missed.

Friday, September 5, 2008

Visual Analysis of 'Ideas in Security'

Amrit Williams, former Gartner analyst and CTO at BigFix is one of the bloggers that I follow regularly. Amrit's a very smart guy and I respect what he has to say. He recently wrote a pair of blog posts (here and here) that compliment eachother.

Now, in the details of what he has to say, Amrit and I are in agreement. But I got to thinking about the second post and how it relates to the first post. And, well, I fired up Visio and mapped the relationships between Amrit's greatest and worst ideas lists.

If we look at the great ideas that didn't spawn or perpetuate the worst ideas, then we're not left with much. Just segmentation and theory of least privilege. If we drop out planning and segmentation because they're not actually security ideas - just good ideas that work lots of places - we're left with Theory of Least Privilege as the one great idea to come out of security. Oddly, that seems about right.