SIEM Market Redux

Roughly a year and a half ago, Rocky DeStefano and I had a conversation about the SIEM market in which he predicted mass acquisitions. It took longer than he originally guessed, but...

1. HP Completes Acquisition of ArcSight
2. SolarWinds Buys TriGeo for $35M
3. IBM to Acquire Q1 Labs
4. McAfee to Acquire NitroSecurity

I'll throw another one out there - after more than 2 years of steady speculation of a Splunk IPO, they hired David Conte as CFO fresh from his setting up the sale of IronKey to Imation. Expect Splunk to be acquired in 2012, or at least try really hard.

The SIEM Market Discussion Continues

Bill Roth of LogLogic commented on my Twitter exchange with Rocky DeStefano of Visible Risk where we talked about LogLogic's announcement that they were discounting their SIEM product. I then wrote a reply, and it got a little long. So I made it a blog post instead.

Rocky, Paul:
The ClueTrain Manifesto calls markets "conversations", so here goes.....

I think you're falling into a the trap of "conventional wisdom". First off, the basic assumption that the world falls neatly into the SIEM categorization is just plain false. I stand by LogLogic's model....it all starts with log management as the crucial piece, without that key use cases like network forensics are not even possible. Second, the notion that dropping the price is bad is just plain weird. Is LogLogic dropping the price to sell more? Sure we are. Are we dropping the price to take market share? Sure we are. Are we seeing a great response? Sure we are. Since when is saving people money a bad thing?

And we're always interested in a podcast. :)

Bill Roth, EVP
LogLogic

Hi Bill,

Thanks for the comment! And thanks for participating in the dialogue. I think it's awesome that LogLogic is out front and engaging on its business decisions. Very refreshing!

As to your point about log management being that crucial initial component of a SIEM implementation, I agree completely. Log management has also developed as its own market segment as well, independent of SIEM. But I don't need to tell you that. :-)

On the topic of LogLogic's decision to discount its SIEM product, I didn't mean - and I don't believe Rocky did either - that charging less for SIEM is bad, or even a bad business move.

That said, I do believe that for some significant portion of potential customers log management is a commodity technology. However, from my own experience and from everything I've seen to date, SIEM is not a commodity technology, and I'm not convinced it will be. As such, I don't see price as a strong competitive differentiator in the SIEM market.

Following the recent recession, where IT capital budgets still haven't caught up to the (hopefully sustained) economic upturn, I imagine the feedback on LogLogic's price cut has been positive, and that you'll see some SIEM sales where you wouldn't have but for the discount. But in the mid- to long-term, I have my doubts as to whether there is any meaningful gain in market share to be had for LogLogic - or any SIEM vendor for that matter - simply by competing on price with other SIEM vendors.

Let's be frank, if price were a big piece of why companies choose a particular SIEM, Cisco MARS would have the lion's share of the market and ArcSight would be folding. Instead, it's the other way around.

Twitter Killed the Blog Star

I've been really busy both in my personal and professional life for the past year or so, with no signs of slowing down soon. But I have to acknowledge that the main reason my blog posts have fallen off is Twitter. Now, all of the ideas that I have that I might have developed and expanded into a blog post are prematurely evaluated for length. If they can be abbreviated to a couple of 140-character haikus or less, they go on Twitter. Which means they never grow up to be blog posts. They're like the high school dropouts of ideas.

But every once in a while, a Twitter exchange becomes so interesting that, despite the compressed and fleeting nature of Twitter, it turns into something worthy of framing. The other night, Rocky DeStefano of Visible Risk and I had an exchange on SIEM that I thought the wider world might find interesting. The background to the conversation is this post from Rocky's blog about the recent announcement from LogLogic that they were discounting their SIEM product, and then this responding blog post from LogLogic.

rockyd
The LogLogic response ->> http://bit.ly/bAQSZO to my discounting SIEM Post ( http://bit.ly/aiW3kB )
8:47 PM May 18th via TweetDeck

rockyd
I need to noodle on the LogLogic response more. I appreciate the conversation, I think I may see the opposite end of the customer spectrum.
9:02 PM May 18th via TweetDeck

pmelson
@rockyd I think you nailed the issue. If you *NEED* SIEM, you won't compromise features/functionality for capital cost savings.
9:06 PM May 18th via TweetDeck

pmelson
@rockyd If Cisco couldn't make "Free SIEM With Purchase" work, it's not ever going to work.
9:07 PM May 18th via TweetDeck

rockyd
@pmelson let's be honest how could they possible respond any differently than they did? time for a podcast on the subject ?
9:50 PM May 18th via TweetDeck

pmelson
@rockyd They could just fess up. "We're shipping log management appliances, but SIEM isn't moving. So we put it on clearance sale." :-)
9:52 PM May 18th via TweetDeck

pmelson
@rockyd I think with Gartner's SIEM MQ being released, we're about to see another round of SIEM casualties as VC pulls out.
9:54 PM May 18th via TweetDeck

rockyd
@pmelson There has to be quickening soon, there is way too much of the same thing in the market.
9:57 PM May 18th via TweetDeck

pmelson
@rockyd Right. I've been thinking about the key SIEM differentiators and I've only got three.
10:00 PM May 18th via TweetDeck

rockyd
@pmelson which three?
10:06 PM May 18th via TweetDeck

rockyd
@pmelson Like - Sources, Scalability, Analytical Usage, Correlation / Statistical Evaluation, and getting Intelligent information out?
10:08 PM May 18th via TweetDeck

pmelson
@rockyd 1) performance/scalability 2) UI and drill-down 3) supported sources.
10:07 PM May 18th via TweetDeck

rockyd
@pmelson there are some others like context of Host, Vuln, Registry, Applications and Users that lead you towards more advanced usage
10:09 PM May 18th via TweetDeck

pmelson
@rockyd OK, so asset data model(s) makes 4, pre-defined content is 5? That's still not a lot.
10:15 PM May 18th via TweetDeck

rockyd
@pmelson each is several years of development and refinement with customers.
10:32 PM May 18th via TweetDeck

rockyd
@pmelson this comes down to a compliance check box sale versus a security team needing to integrate a tool into their process.
10:35 PM May 18th via TweetDeck

pmelson
@rockyd Agree. But a handful of differentiators == a handful of potential market leaders. Time to thin the herd. Again.
10:42 PM May 18th via TweetDeck

rockyd
@pmelson now I see where you're headed. BTW I think you'll see 3 more acqusitions by end of year.
10:45 PM May 18th via TweetDeck

rockyd
I was thinking about creating a "vegas odds" website for SIEM Quickending and donate some portion of the funds to HFC.
10:47 PM May 18th via TweetDeck

pmelson
@rockyd A SIEM futures market? Very DARPA!
10:49 PM May 18th via TweetDeck

So there, for your parsing and edification, some thoughts on the SIEM product space, the recent Gartner MQ for SIEM, and the near-term ramifications of Gartner's paper on the market.

Also, if you aren't already, you should be reading Rocky's blog, especially if you're interested in SIEM and security ops. Rocky's a guru in this space, and in addition to his blog he has already put together some great podcasts since launching his latest venture, Visible Risk.

Queries: Excel vs. ArcSight

Since ArcSight ESM 4.0, reports and trends have been based on queries. Considering that ESM runs on top of Oracle, a query in ESM is exactly what you think it is. Queries are an extremely flexible way to get at event data. But as the name implies, they go against the ARC_EVENT_DATA tablespace, and therefore you can't use them to build data monitors or rule conditions, since those engines run against data prior to insertion into the database.

Anyway, I've got a story about how cool queries are. And about how much of an Excel badass I am. And also about how queries are still better. Last month, I got a request from one of our architects who was running down an issue related to client VPN activity. Specifically, he wanted to know how many remote VPN users we had over time for a particular morning. Since we feed those logs to ESM, I was a logical person to ask for the information.

So I pulled up the relevant events in an active channel and realized that I wasn't going to be able to work this one out just sorting columns. So, without thinking, I exported the events and pulled them up in Excel. So here's the Excel badass part:



If you want to copy it, here it is:
=SUM(IF(FREQUENCY(MATCH(A2:A3653,A2:A3653,0),MATCH(A2:A3653,A2:A3653,0))>0,1))

So A is the column that usernames are in. This formula uses the MATCH function to create a list of usernames and then the FREQUENCY function to count the unique values in the match lists. You need two MATCH lists to make FREQUENCY happy because it requires two arguments, hence the redundancy. It took about an hour for me to put it together, most of that was spent finding the row numbers that corresponded to the time segment borders.

But as I finished it up and sent it off to the requesting architect, I thought, there must be an easier way. And of course there is. So here's how you do the same thing in ESM using queries:






















So, it's just EndTime with the hour function applied, and TargetUserName with the count function applied, and the Unique box (DISTINCT for the Oracle DBA's playing at home) checked. And then on the Conditions tab you create your filter to select only the events you want to query against. That's it.

Once the query is created, just run the Report Wizard and go. All told, it's about 90 seconds to the same thing with a query and report that it took an hour to do in Excel.

Inbox 3

Teguh writes,

Hi Paul,
could you give some guide to administering logger? i searched thru
google, but found nothing significant. How to(s) and tutorial would be enough i
guess. Does it have to have syslog server for the logger to be able to read data
from?
Thanks..

The documentation for Logger is available from ArcSight's download center. Only registered customers have access, but I assume that if you've got a Logger box, that generally qualifies you.

With regard to your second question, yes Logger has a syslog server. It actually has a few. In Logger nomenclature these are "receivers." Logger supports UDP and TCP syslog, FTP and SSH file pull, NFS and CIFS remote filesystem. Logger also supports some ArcSight-specific receivers including a SmartMessage receiver for events forwarded from ESM and CEF-over-syslog (OK, ArcSight wouldn't agree that this is specific to their products, but despite the C standing for Common, CEF is anything but. At least right now.)

1. Configuring Logger to act as a syslog server is pretty straightforward.
   
2. From the web interface, navigate to Configuration, Event Input/Output.
   
3. On the "Receivers" tab, click the Add button.
   
4. Name your connector and set the type as "UDP Receiver" then click Next.
   
5. The defaults for Compression Level and Encoding are fine. Select the IP address you want the listener to reside on, and set the port number. The default syslog server port is UDP/514.
   
6. Click Save.
7. On the "Receivers" tab, click the little no-smoking image next to the new receiver to enable it.
   

From The Inbox 2

lmran writes:

Hi Paul,
Do you know any reason why ArcSight ESM does not support the Cisco MARS? Right now, all my firwalls send the syslog feeds into Cisco MARS and I'm trying to set the Cisco MARS to send thoes raw feeds data to ArcSight local connector but I just found out that ArcSight does not support the Cisco MARS. Thanks in ADV for any info reading this subject.

Starting in 4.x, MARS can forward events to another remote syslog listener. ArcSight has a syslog connector. So you ought to be able to forward events from MARS to ArcSight via syslog assuming MARS doesn't change the format of the log events too much. Even if MARS does mangle the event format, ArcSight will still receive them, but then most or all of the event will be parsed into the CEF Name field and categorization and prioritization won't be accurate.

If you are unable to upgrade your MARS appliance to 4.31 or later (I think that's the rev you need), another option would be to use a syslog-ng server out front. It supports forwarding events by source to other syslog servers. You could use this to send the stuff you want in ESM to ArcSight's syslog Connector and the stuff you want in MARS to MARS.

Or, you could do the environmentally conscious thing and unplug then recycle your MARS appliance. ;-)

From The Inbox

Anonymous writes,

Hi Paul, I am one of those who, as you say, found your blog by googling ArcSight, trying to do some recon on the product for my employer. (I think I see that the most recent posts here are from 2007 so who knows if you or anybody will be seeing my question.) I'm trying to find out, can Arcsight's data be queried programmatically; i.e. is it stored in a relational database, hopefully SQL Server or Oracle, or if not, is there an API or ADO.NET provider that can allow it to be queried, preferably with SQL? Thanks for any info anyone reading can provide.

ArcSight ESM uses Oracle 10g for its back-end database. At one point, and this may still be true, DB2 was also supported. You can query the database directly, and the schema is pretty straightforward. The table ARC_EVENT_DATA is where most of the event data lives, for example. But depending on your use case, that might not be the best way to get data out of ESM.

Also, since you didn't specify, it may be worth mentioning that the same is not true of the ArcSight Logger platform, which is flat storage. Instead of querying the log store directly, Logger can be configured to forward events based on source, type, etc. to another destination, if you need them in real-time. There is a PostegreSQL database on Logger, but it's my understanding that it supports the reports engine, and doesn't store the raw or CEF events in any comprehensive way.

The interesting thing is that the storage technology behind Logger 3.0, because of its performance and relative "cheapness" may become the data store for ESM down the road. It would only make sense, since you could handle MUCH higher event rates with less disk and no Oracle license fee. If it can be done while maintaining the stability and feature set that the Oracle-based data store has, it's a walk-off home run for ArcSight.

ArcSight Tools Slide Deck

Wow, I've just been buried, both at work and at home. I promised a sanitized copy of our slides from the ArcSight User Conference and here they are. A month late. Enjoy.

Managing ArcSight ESM Tools

Last week Tim and I presented at the ArcSight user conference on using Tools in the ESM console to augment incident response and investigation. I'm hoping to have the sanitized slide deck up this week, and maybe a little bit of code to go with it.

At the end of the talk - and then a couple of times in the hallway - people asked how we manage all of these Tools. It's a great question, and the answer is, not very well. But here's the best way that I know how in ESM 4.0.

If you've ever looked at the tools editor in the ESM console, you've seen this dialog, which is pretty basic:


By default, this info is recorded in your AST file (C:\arcsight\Console\current\paul.ast), which is just a text file with a bunch of values declared. The values in the file look like this:

console.ui.tools[toolName].program=
console.ui.tools[toolName].workingdir=
console.ui.tools[toolName].iconFile=
console.ui.tools[toolName].parameters=
console.ui.tools[toolName].showInToolBar=
console.ui.tools[toolName].isExportTool=


And then there's this:

console.ui.toolsList=

Which is pretty self-explanatory. It occurs once in the AST file and is just a CSV list of the tool names, used to populate the console's Tools menu.

I wrote a Perl program that can parse an AST file and extract tool data from it for the purposes of sharing. It's designed to help scale and distribute tools across your analysts' consoles so that they don't have to manually recreate and test them. In order for it to be useful, there are some best practices. Here's what I do.

1. Cygwin (Surprise!) If you've been reading my blog for any period of time, you knew this was coming. If your analysts use Mac OS X or Linux for their ESM console platform, not to worry. They can play, too. Beyond, Perl, bash, and Python, that's kind of the point.

2. Standardize on a source directory for scripts. Put all of your scripts in the same spot. Pathing is difficult to manage by hand, so by defining a standard (I use /usr/local/bin/arcsight), you have less to do each time you distribute a new tool.

3. Use a repository like Subversion or CVS for scripts and other tool artifacts. That way, you can make a change to your tool, check it in, and the other analysts can check it out quickly and easily. No messy manual copies. Also, when you foul something up, you have revision history to go back to. That can be a life saver if you are - like me - not a developer with good testing habits.

4. Use consoleupdates.txt on the ESM manager to distribute tool configs. Here's how you do that:

Let's say my ESM user id is 'paul' and I have developed a whole bunch of tools following the first three rules above. I can use this Perl script to create an export of the tool configs for use on the server. It looks like this in Cygwin:

$ arc_tool.pl export all /cygdrive/c/arcsight/Console/current/paul.ast > ~/consoleupdates.txt
$ scp consoleupdates.txt arcsight@esmmanager:/opt/arcsight/manager/config
$ ssh arcsight@esmmanager
Password:
arcsight@esmmanager:~ $ chown arcsight.arcsight $ARCSIGHT_HOME/config/consoleupdates.txt
arcsight@esmmanager:~ $ chmod 644 $ARCSIGHT_HOME/config/consoleupdates.txt

If you want your analysts to get the updated tool list, they need to log out of the console, move their AST file somewhere safe, and log back in to the console. No restart of the manager is necessary. Now they just need to create the same script directory you did and check out your scripts from your CVS server.

The Perl script I wrote also supports listing the tool names in an AST file as well as exporting single tool configurations, so it's more than just a one-trick pony. It's got another half a trick.

My advice at this time is not to invest a ton of time in doing this unless it's a weekly headache for your security team, but it is worth doing if you've already got the moving parts in place (like a CVS server). The reason is that ArcSight has already fixed the issue of sharing tool configs in ESM 4.5. So once that's released (later this year?), some of this will be a non-issue for you. I suspect that the first two or three best practices I list above will still be valid in ESM 4.5, so it's still a valuable exercise if you have any number of custom tools already.

ArcSight User Conference 2008

I'm on the floor of the ArcSight "Protect '08" conference this morning. Tim and I gave our talk on ArcSight ESM Tools yesterday, and I will post some version of those slides and some of the code after I return from the conference.

Right now I'm listening to Hugh Njemanze give his keynote on product lines. There's a lot of interesting stuff in the release pipe; Logger 3.0, ESM 4.5, a new Connector appliance, IdentityView content for ESM, and something called "McLovin."

Anyway, here's what's been good so far:

• Customer presentations (other than mine, I mean) - I missed out last year, these are the best talks so far.
• Location - the new hotel is within walking distance of stuff (and by stuff I mean not trees and the NSA.)
• Networking - Always the best part of this conference. I love standing around with free beer, talking to other folks about what they're doing with their SIM, and sharing ideas. Looking forward to more tonight.
  

Here's what's been not-so-good:

• Wireless - the hotel wireless has been unreliable and overloaded. Frankly, I'm surprised I've been able to stay on long enough to get this post up.
• Vendor/sponsor floor - no offense to these guys, but the freebies this year are unimpressive. I've already got a pen, thanks.
• No bag - Instead of a "conference bag," everyone was issued a plastic file folio thing. Not that I needed another bag, but I can't smoosh the one foam squeezy thing I did get from a vendor booth into this blue plastic thing.

And I would be remiss if I didn't drop a product scoop or two:

• Logger 3.0 has adopted a more-ESM-like boolean filter interface. Big improvement over the chained-regex search in 2.5 and earlier.
• Demo of Logger 3.0 shows that searches of data (no details on data set) are roughly 80x faster than a similar sized search on 2.5. (The claim is 100x faster, but I counted. Still, that's a significant improvement.)
• Hugh has hinted that the slick, high-performance append-only storage stuff that Logger has is going to be integrated into ESM in some release beyond 4.5. That could mean the end of the Oracle / PartitionArchiver storage model. It won't be missed.

I'm Floored: Raffael Marty declares that SIM is dead.

No really, he said it. He would've been on the short list of people I assume would never say it. But there it is.

Here's the thing; I think that this is a lot like Gartner's IDS declaration (which he cites). IDS went through some product positioning changes (IPS, UTM, DLP, etc.) but the core idea and technology is still there, and guess what? The original IDS use case is still viable. Sure the attacks have changed, but having a sniffer that can search for known-bad and known-strange traffic on the wire is very, very useful.

So I assume that we are in the midst of a product positioning shift around SIM. Raffy's point that SIM schema are IP-centric and rules are based around correlating firewall and IDS events is true. But most of the vendors have already acknowledged this and are developing content to focus on other log sources. Either way, the use case is here to stay - the ability to search and correlate log events is highly useful, and will continue to be. You may call it "SIEM" or "IT Search" or "log management," but it's the same core concept, repurposed to address the constantly changing security environment.

One final note for vendors from the SecOps trenches: I am not open to a replace/resell on the basis that SIM is old and whatever-you-call-it-now is new and better. My SIM, like my IDS, contains custom content that our team has developed to keep on top of changing threats, including application attacks. SIM, like IDS, succeeds when you put talented security professionals in front of it and let them tune it and manage it like a tool. But it will fail miserably if you are hands-off with it.

ArcSight Logger Face-plate-lift

Not only did ArcSight deliver on the improved UI and feature set for Logger 2.5, but like any good appliance vendor, they've popped their collar. And by collar I mean front bezel.



The red with blue LED scroller is definitely a better look than the previous model. Still no backlit logo, though. :-)

ArcSight's new Logger Apps

ArcSight is releasing the Logger 2.5 software here soon, and along with it new appliances with some interesting variations. You can check out the vitals on the ArcSight website here.

Prior versions of Logger were available in small, large, and SuperSized, where the SuperSized box was the same spec as the large box with artificial limitations removed via license key. So really only two boxes, all self-contained, all CentOS, all MySQL.

Now, there's a whole new batch. It would appear by the naming designations that they are going after PCI compliance heavily with the L3K-PCI, which must have retention policies and capabilities that make it easier to comply with PCI-DSS 5.2. Another model supports SAN-attached storage and Oracle, so you can grow your Logger with SAN instead of NAS. And finally, there are two new L7100 models with 6x750MB drives. If I'm doing my math right, that works out, after compression, to about 40TB 36TB of log storage. That's a significant increase over the 15TB 12TB that the large/SuperSized L5K boxes shipped with.

Update: Talked with Ansh at ArcSight today, and aparently the 2.5 software adds columns to the CEF event view. That's a big deal for folks using CEF events in Logger, and may make CEF-only the preferred format for most Logger users. The new software also includes real-time alert views (like Active Channels in ESM), as well as a number of other enhancements to alerts and search filters and more. Current customers can download 2.5 from the software site.

From My Inbox... (ArcSight Connectors & Logger)

SC left a comment on an earlier post on ArcSight Logger and CEF vs. Raw formats...

Hi Paul,

Do you know if its possible to "insert" logs into the logger or SmartConnector if the logs are on a physical storage, e.g. DVD or external storage?

Thanks.

Kind Regards,
SC

There are probably a number of ways to do this, but I've only tested one. In earlier configurations of our syslog infrastructure, there were a couple single points of failure. In order to meet log analysis commitments, we would reload lost syslog data from file.

Start by configuring a 'Syslog Pipe' Connector. Since the connector only has to be online with the Manager when you're manually inserting logs from file, you have greater flexibility about where this Connector will live. It lived on my laptop for a while. When you set it up, point to a path that isn't already used for anything else. Then you can simply start the Connector and pipe the raw log file(s) to the named pipe:

# cat oldsyslogs.txt >> /var/spool/my_arcsight_pipe

Depending on how far the date/time stamps of the events in those files are from $Now, ArcSight will probably throw some errors. It will maintain the "End Time" from the raw log events, and apply "Manager Receipt Time" as the time the manager collects the parsed events from the Connector. This will absolutely screw up any correlation rules you wanted these events to be subject to. Sorry, no easy way around that.

ArcSight Logger: CEF vs. Raw

Here's something for potential ArcSight Logger customers to ponder. The issue is whether you should use CEF formatted logs (post-Connector) or raw logs (pre-Connector) or both in your Logger environment. In this case, a picture is worth at least a few hundred words:



If you look carefully at that image, you can see that it shows the same event in both its raw syslog format and it's Connector-ized CEF format. From my point of view, it boils down to use case. Analysis versus troubleshooting. Reporting versus response. The CEF formatted message is chock-full of metadata-and-labeling goodness. It's also overkill on the eyes. Log messages are already cryptic to the point of questionable usefulness. CEF amplifies that. The raw format, on the other hand, is easier to read due largely to the fact that it's what your UNIX admins are used to seeing. But that's where the positives end. Raw syslog is all but unformatted and trying to write a small chain of regexes that do a good job of parsing large quantities of syslog is a headache and a half.

Of course, you may have already realized that there is a right answer to this problem: Do both. Sure there's some overhead to consider, since you're going to pass syslog to a Connector that will then send raw events to Logger, CEF events to Logger, and CEF events to ESM if you have it. Or you could send raw syslog to Logger, have Logger forward it to a Connector and then configure the Connector to send CEF to Logger and ESM. There are probably many other complicated flows that you could implement as well, but you get the idea.

Abstract

I've asked to be considered for presenting at this year's ArcSight User Conference. Today I sent my abstract over and will hopefully be on the agenda this year to talk about ArcSight Tools.

The incident handlers at [my company] use ArcSight Tools in their investigations as a way to quickly and easily collect additional intelligence from existing data stores in their environment. Come see how, with very little custom code, they have harnessed existing applications and services to quickly gather in-depth information about servers, users, workstations, and external hosts during an investigation. In addition to seeing how [my company] has leveraged ArcSight Tools, learn some of the simple tricks that will help you go back to your office and do the same.

I've blogged about Tools before and this presentation aims to be an expansion of that concept. The truth is, there's lots of great data in your environment that isn't in a log flow somewhere. And while it maybe doesn't belong in your SIM, you want it at your fingertips when investigating a potential incident. It's good to have answers to questions like, "What does this server do?," or "Is this user a local admin?," or "What is this person's boss' phone number?" close at hand.

Anyway, I hope to have more to say about ArcSight Tools soon.

Anonymous writes... (ArcSight Resources)

Anonymous writes,

Hi Paul,

I would like to ask if you know of any resources I can reference for ArcSight correlation rules authoring.

In particular, I am looking for Web App and VOIP Security. Thanks in advance.

So first of all, there's an unfortunate shortage of sources on building content for ArcSight. It's part of why I blog about it, because there are only a few people putting information out there. And if SIM's in general are going to mature, then best practices and an open community are part of that maturation. Besides blogs like mine, the ArcSight forums are a good place to get questions answered and share content. Beyond that, I would highly recommend the annual User Conference that ArcSight puts on. For those that can't attend the User Conference, the slides are published to your software site, and definitely worth downloading. And of course ArcSight's own training offerings. But that is pretty much the extent of resources available at the moment.

As far as ways to monitor Web Apps and VoIP Security with ArcSight, it's going to boil down to the log sources you have available. Here are a couple of ideas I have off the top of my head.

For Web App there are tons of optiions. ArcSight works with several web security proxies, IIS and Apache, most IDS/IPS products under the sun, web app servers like Weblogic and WebSphere, and the more popular commercial databases like Oracle, MS-SQL, and DB2. Depending on what's in your web environment and which sources you're drawing from, you have lots of options here. An easy idea might be to create a filter to sift through web server logs for special characters (like < > ' or - ) or requests where the web server returned a 500 or some other obscure error (not 403 or 404).

VoIP is a trickier one to go after since there's no ArcSight connector for CallManager or whatever SIP gateway you use. You could write one with the Flex Connector SDK, but I'm not sure how great your SIP gateway logs are to begin with it comes to security. I think switches, IDS/IPS, and firewall are your best bets here. You'd want to filter firewall logs for packets sourced from your VoIP VLAN address space that might indicate a rogue device connected to your voice network. (Which reminds me, a new version of voiphopper just came out.) You might also want to filter IDS logs for traffic sourced from your VoIP VLANs as well. Hopefully you've already got "switchport port-security maximum 2" set on all of your VoIP ports (and all of your userland switch ports in general) to prevent ARP spoofing/poisoning attacks. In which case, if you send your switch syslogs to ArcSight, a rule to alert on 'NOMAC' messages could be very useful. These can be regular errors, but also occur when someone attempts ARP-based MITM attacks in a port where port-security has been configured.

Anyway, I hope that helps, Anonymous. Good luck with your projects.

ARST

Associated Press article via CNN Money. It's pretty favorable about this week's ArcSight IPO. Frankly, I don't care that much about their going public. I only like to blog about it because it's a thumb in Mike Rothman's eye. :-)

ArcSight 4.0 Part 2: Awesomeness

All you need to do to know that ArcSight has made some big improvements to it's v4.0 ESM product is read a press release. Or launch the 4.0 console even:

And while trending, improved reporting, identity correlation, and portable content are all great feature adds, you can read about them anywhere and everywhere. So I'm not going to talk about them. What I am going to talk about are the little changes to the 4.0 console that I have found to be very useful. The stuff that you might not find if someone didn't point it out.

Let's start with filters. This one you'll probably find, but I just want to say, "Hallelujah!" Seriously, this makes me want to do backflips down the cube aisles like John Belushi in The Blues Brothers. They fixed the filter editor!

If you ever created a big, complicated filter in 3.x only to open it later to find a hideous tangled nest where your neatly organized filter had been before, then this is a victory for you. Take a look:

The other obvious improvement in the filters is automatic escaping of characters. Here's a screen shot of that:

This is an interesting feature add for a number of reasons. The big advantage to you is that when you send results to external formats (think HTML or PDF reports or CSV exports), you don't have to do escaping yourself. I suspect there are a number of advantages to the ArcSight Manager and Web components as well. I don't know, so I'll leave it to you to speculate.

One of the new features that I really like is in the right-click menu of the Events tab of the Case tool.

"Retrieve correlated events" will do exactly that. If, like me, you have cases that open with correlated events only, but you want to record or examine the events that triggered the case, this saves several minutes. Very handy.

The final feature that I wanted to talk about, and my personal favorite, has to do with an enhancement to event annotation. We use event annotation as a way to create an audit trail for log review. The selected events load in an active channel and then our handler-on-duty reviews the events, opens cases where necessary, and changes the events' annotation stage to "Closed" as a way to indicate systematically that they have been reviewed. This adds up to a good 10-15 hours of work that week at least, so anything we can do to speed the process along is greatly appreciated. Starting this month, we will be migrating to using the isReviewed event annotation flag.

The main reason for doing this, though, has more to do with a menu item and shortcut that's been added to the Active Channel / Grid view.

Now instead of 5 mouse clicks, Ctrl-R marks the events as reviewed. It's a nice streamline if you use the feature.

There are lots of other enhancements to the 4.0 console, many of which I probably haven't found yet. But these were the ones that jumped out at me and will have a positive impact on the way I use the tool.

ArcSight 4.0 Part 1: Oddities

This past week, we upgraded our production ArcSight environment from 3.5 SP2 to 4.0 SP1. We've been running ArcSight 4.0 in our test environment since August 2007, but as with all things "test" there are always a few things that you won't discover in a test environment.

This is the first part of a 2-part series on the upgrade experience. I want to describe some technical challenges that we experienced that you won't find in the upgrade documentation in hopes that someone else finds this helpful.

Before I start talking about what went wrong, I should say that the process was seemingly painless aside from what I describe here. I say seemingly, because I didn't actually do it. But Tim, who did the 2-part upgrade and redesign over the course of three weeks, was still smiling on Thursday following the upgrade. He seems to have emerged from the gauntlet unscathed. I think this is because Tim's a bad-ass and also because ArcSight dramatically improved the upgrade process in 4.0.

So the first problem we ran into has to do with some changes to the Jetty code in the ArcSight manager between 3.5 and 4.0. Here's the error we got:

[2008-02-06 13:50:55,727][INFO ] [default.com.arcsight.server.Jetty311ServletContainer] [initialize] Key Store: [JKS] /opt/arcsight/manager/config/jetty/keystore com.arcsight.common.InitializationException: Exception initializing 'com.arcsight.server.Jetty311ServletContainer': The keystore may not contain more than 1 entry. Please remove excessive entries. at com.arcsight.server.Jetty311ServletContainer.initialize (Jetty311ServletContainer.java:288)

The keystore file was the same one we'd been using since 3.0. We generated our own CA key pair and certificate from OpenSSL and signed the certificate for the keystore file with it. We then added the CA certificate to the cacerts file that is used by all of the other components. While going through this process, I added the CA cert to the keystore file for posterity.



The CA cert being present in the keystore file is what caused the error, and editing the keystore with 'keytoolgui' to remove the CA cert was all it took to get it back up and running.

The other issues that we ran into occured post-upgrade and had to do with upgraded content. The first issue we saw isn't really an issue at all, rather ArcSight's improved some of its logic around Active Channels. Specifically, Active Channels that were created using one time stamp and configured to sort on another time stamp will give an error now. For example, if you created a channel and set EndTime for "Use as time stamp," and then on the Sort tab, set a sort for ManagerRecieptTime, this would cause an error in 4.0. In 3.5 it would merely hurt the channels performance, but would eventually load. This is a good change, but it may mean having to edit some of your old (and presumably really slow) Active Channels before they work again.

The second issue we saw was around filters. ArcSight has made some major improvements to Filter objects, and I'll talk more about that in the upcoming post. However, one drawback seems to be a bug in the parsing/escaping enhancements in 4.0. Filters that use a string match that contains angle brackets ( [ or ] ) will return null sets. There is no error, the only symptom is null results. In our case, all of the brackets appeared in filters matching on the Name field (gotta love undocumented syslog formats). The solution is to revise your filters to not use the brackets.