Anonymous writes... (ArcSight Resources)

Anonymous writes,

Hi Paul,

I would like to ask if you know of any resources I can reference for ArcSight correlation rules authoring.

In particular, I am looking for Web App and VOIP Security. Thanks in advance.

So first of all, there's an unfortunate shortage of sources on building content for ArcSight. It's part of why I blog about it, because there are only a few people putting information out there. And if SIM's in general are going to mature, then best practices and an open community are part of that maturation. Besides blogs like mine, the ArcSight forums are a good place to get questions answered and share content. Beyond that, I would highly recommend the annual User Conference that ArcSight puts on. For those that can't attend the User Conference, the slides are published to your software site, and definitely worth downloading. And of course ArcSight's own training offerings. But that is pretty much the extent of resources available at the moment.

As far as ways to monitor Web Apps and VoIP Security with ArcSight, it's going to boil down to the log sources you have available. Here are a couple of ideas I have off the top of my head.

For Web App there are tons of optiions. ArcSight works with several web security proxies, IIS and Apache, most IDS/IPS products under the sun, web app servers like Weblogic and WebSphere, and the more popular commercial databases like Oracle, MS-SQL, and DB2. Depending on what's in your web environment and which sources you're drawing from, you have lots of options here. An easy idea might be to create a filter to sift through web server logs for special characters (like < > ' or - ) or requests where the web server returned a 500 or some other obscure error (not 403 or 404).

VoIP is a trickier one to go after since there's no ArcSight connector for CallManager or whatever SIP gateway you use. You could write one with the Flex Connector SDK, but I'm not sure how great your SIP gateway logs are to begin with it comes to security. I think switches, IDS/IPS, and firewall are your best bets here. You'd want to filter firewall logs for packets sourced from your VoIP VLAN address space that might indicate a rogue device connected to your voice network. (Which reminds me, a new version of voiphopper just came out.) You might also want to filter IDS logs for traffic sourced from your VoIP VLANs as well. Hopefully you've already got "switchport port-security maximum 2" set on all of your VoIP ports (and all of your userland switch ports in general) to prevent ARP spoofing/poisoning attacks. In which case, if you send your switch syslogs to ArcSight, a rule to alert on 'NOMAC' messages could be very useful. These can be regular errors, but also occur when someone attempts ARP-based MITM attacks in a port where port-security has been configured.

Anyway, I hope that helps, Anonymous. Good luck with your projects.

Frank Boldewin Releases "More Advanced Unpacking - Part II"

Just last month I posted about Frank's first release in this series, along with some other recommended reading. Here's Part II via Offensive Computing. Awesome stuff. Frank is essentially giving away what he could charge thousands for as a BlackHat Training course or something similar. It's exceptionally generous of him to do so. I've been in awe of Frank's malware unpacking skills for a while now, and I hope he continues to release more tutorials.

Ranting About Insider Threat

This is another one of those posts that started out on a listserv. The original thread was about using contracting staff for security work. Along the way, someone mentioned insider threat and I went off. Enjoy.


> While he or she does not necessarily have access to
> many/all functional areas (hopefully), he/she would
> have an easier time compromising the organization's
> security. Being that internal threats are much more
> prevalent than external ones, I'd argue that this
> poses a greater risk when compared to equally-
> screened contract employees under the same NDAs, etc.

I'm not sure that I agree with your statement that internal threats are much more prevalent, or even more prevalent. Looking at the CSI/FBI Survey results over time, the Internet surpassed internal networks as the origin of attack in 2001 and has been widening ever since. I don't think the data bears this conclusion out.

If I put my tinfoil hat on here for a minute, I do think that the way that insider abuse is hyped and promoted in infosec trade press is intentionally vendor-driven. Vendors whose products address border security at layers 3-4 found their sales slumping a few years ago when everybody finally got a firewall, everybody that was going to get NIDS got NIDS, and everybody that had to comply with GLBA got DLP.

So they started telling ghost-story-anecdotes about insiders and how we need to watch our staff like they're an elite team of Chinese hackers. Of course, they never suggested how they would solve the real insider issue. Insiders getting unauthorized access to data isn't the problem. It's what they do with the data that they *are* authorized to access that's the problem. When somebody comes up with IDS signatures for bad intentions, please let me know. I'll be the first one with my checkbook out.

I guess my bottom line on insider threat is not that it's wholly imaginary, but that it's not the same as external threats. Cool appliance-based technologies don't serve you as well inside. Doing things like monitoring use of administrative accounts, auditing financial application access for proper separation of duties, and proper pre-hire screening of staff go a whole lot further than watching what files people copy to their thumb drives and guessing what they might do with them.

14.4Kbps Nostalgia

I've got some decent display real-estate on my desk, roughly 576 square inches total, so choosing background wallpaper is not a task Itypically proceed into lightly. Well, last week, my selection of background aesthetic was undertaken a bit hastily. There was fallout. In what can only be described as a Bostonian fashion, Aqua Teen Hungerforce was once again poorly received. This time by my co-workers.

So today I went to DeviantArt looking for some new wallpaper, and stumbled upon these:

They're ANSI art, all done by Thor (iCE), for BBS's I used to call back when I was in high school. Legion HQ was the place, too. No real names. Just teaching each other random bits of knowledge and (mostly) cursing each other out like the angsty, angry misfits that we were. I didn't know it then, but I made some good friends during this time. I still keep in touch with a few, 14 years later. But what ever happened to Evil Dude?

Anyway, it's cool to see these again. They remind me of a much simpler time... of Desqview and Telix, of Pascal and WordPerfect, of Kings Quest and fractint, of 2600 meetings and boot sector viruses. It was all new back then. I was all new back then.

Sometimes in life, every few years or so, you stop and look back at who you used to be. The long blue hair has been replaced with short hair that's starting to speckle grey. I still listen to the same music, but it's not cool anymore. But I think that, aside from having never gotten a tattoo, the 16yr old I used to be would've been pretty jazzed to see how his life turned out.

It was 20 years ago today... (minus 19)

This time last year, I decided to act on my idea to steal Matasano's idea for an informal meetup for infosec practitioners, proselytizers, patrons, and partygoers. And while I can take credit for something that boils down to, "Hey, who likes beer and security? Then let's go to the bar!", I can't take credit for all of the awesome conversation and fascinating stories from some of the best and brightest infosec minds in the midwest or anywhere for that matter.

But that's what GRSec has become. I've been to most of the 12 meetups, and I've had a good time with and learned from others at each one. And so I want to extend a genuine thank you to everyone who's come out to a GRSec.

If you're in the Mid- or West Michigan area and want to network and talk shop with other infosec types, I encourage you to join us for GRSec. Or if you're in another geoloc, check out CitySec.org and find a meetup in your area.

ARST

Associated Press article via CNN Money. It's pretty favorable about this week's ArcSight IPO. Frankly, I don't care that much about their going public. I only like to blog about it because it's a thumb in Mike Rothman's eye. :-)

ArcSight 4.0 Part 2: Awesomeness

All you need to do to know that ArcSight has made some big improvements to it's v4.0 ESM product is read a press release. Or launch the 4.0 console even:

And while trending, improved reporting, identity correlation, and portable content are all great feature adds, you can read about them anywhere and everywhere. So I'm not going to talk about them. What I am going to talk about are the little changes to the 4.0 console that I have found to be very useful. The stuff that you might not find if someone didn't point it out.

Let's start with filters. This one you'll probably find, but I just want to say, "Hallelujah!" Seriously, this makes me want to do backflips down the cube aisles like John Belushi in The Blues Brothers. They fixed the filter editor!

If you ever created a big, complicated filter in 3.x only to open it later to find a hideous tangled nest where your neatly organized filter had been before, then this is a victory for you. Take a look:

The other obvious improvement in the filters is automatic escaping of characters. Here's a screen shot of that:

This is an interesting feature add for a number of reasons. The big advantage to you is that when you send results to external formats (think HTML or PDF reports or CSV exports), you don't have to do escaping yourself. I suspect there are a number of advantages to the ArcSight Manager and Web components as well. I don't know, so I'll leave it to you to speculate.

One of the new features that I really like is in the right-click menu of the Events tab of the Case tool.

"Retrieve correlated events" will do exactly that. If, like me, you have cases that open with correlated events only, but you want to record or examine the events that triggered the case, this saves several minutes. Very handy.

The final feature that I wanted to talk about, and my personal favorite, has to do with an enhancement to event annotation. We use event annotation as a way to create an audit trail for log review. The selected events load in an active channel and then our handler-on-duty reviews the events, opens cases where necessary, and changes the events' annotation stage to "Closed" as a way to indicate systematically that they have been reviewed. This adds up to a good 10-15 hours of work that week at least, so anything we can do to speed the process along is greatly appreciated. Starting this month, we will be migrating to using the isReviewed event annotation flag.

The main reason for doing this, though, has more to do with a menu item and shortcut that's been added to the Active Channel / Grid view.

Now instead of 5 mouse clicks, Ctrl-R marks the events as reviewed. It's a nice streamline if you use the feature.

There are lots of other enhancements to the 4.0 console, many of which I probably haven't found yet. But these were the ones that jumped out at me and will have a positive impact on the way I use the tool.

ArcSight 4.0 Part 1: Oddities

This past week, we upgraded our production ArcSight environment from 3.5 SP2 to 4.0 SP1. We've been running ArcSight 4.0 in our test environment since August 2007, but as with all things "test" there are always a few things that you won't discover in a test environment.

This is the first part of a 2-part series on the upgrade experience. I want to describe some technical challenges that we experienced that you won't find in the upgrade documentation in hopes that someone else finds this helpful.

Before I start talking about what went wrong, I should say that the process was seemingly painless aside from what I describe here. I say seemingly, because I didn't actually do it. But Tim, who did the 2-part upgrade and redesign over the course of three weeks, was still smiling on Thursday following the upgrade. He seems to have emerged from the gauntlet unscathed. I think this is because Tim's a bad-ass and also because ArcSight dramatically improved the upgrade process in 4.0.

So the first problem we ran into has to do with some changes to the Jetty code in the ArcSight manager between 3.5 and 4.0. Here's the error we got:

[2008-02-06 13:50:55,727][INFO ] [default.com.arcsight.server.Jetty311ServletContainer] [initialize] Key Store: [JKS] /opt/arcsight/manager/config/jetty/keystore com.arcsight.common.InitializationException: Exception initializing 'com.arcsight.server.Jetty311ServletContainer': The keystore may not contain more than 1 entry. Please remove excessive entries. at com.arcsight.server.Jetty311ServletContainer.initialize (Jetty311ServletContainer.java:288)

The keystore file was the same one we'd been using since 3.0. We generated our own CA key pair and certificate from OpenSSL and signed the certificate for the keystore file with it. We then added the CA certificate to the cacerts file that is used by all of the other components. While going through this process, I added the CA cert to the keystore file for posterity.



The CA cert being present in the keystore file is what caused the error, and editing the keystore with 'keytoolgui' to remove the CA cert was all it took to get it back up and running.

The other issues that we ran into occured post-upgrade and had to do with upgraded content. The first issue we saw isn't really an issue at all, rather ArcSight's improved some of its logic around Active Channels. Specifically, Active Channels that were created using one time stamp and configured to sort on another time stamp will give an error now. For example, if you created a channel and set EndTime for "Use as time stamp," and then on the Sort tab, set a sort for ManagerRecieptTime, this would cause an error in 4.0. In 3.5 it would merely hurt the channels performance, but would eventually load. This is a good change, but it may mean having to edit some of your old (and presumably really slow) Active Channels before they work again.

The second issue we saw was around filters. ArcSight has made some major improvements to Filter objects, and I'll talk more about that in the upcoming post. However, one drawback seems to be a bug in the parsing/escaping enhancements in 4.0. Filters that use a string match that contains angle brackets ( [ or ] ) will return null sets. There is no error, the only symptom is null results. In our case, all of the brackets appeared in filters matching on the Name field (gotta love undocumented syslog formats). The solution is to revise your filters to not use the brackets.

Globalization and Online Crime

This article is an interesting read. Not so much for the story itself, but for what it means. The notion that a small agricultural town in Romania can, in three short years, overhaul its local economy via online fraud is fascinating to me. Not the first report of eBay having problems with Romanian criminals, but the idea that the region's economy is now heavily based on fraud is impressive.

But here's what's really blowing my mind: A poor local economy, unbalanced international economies (seriously, as bad as it may seem in the US, at least you're not trying to get by making your own wine and selling it to tourists on the train to Budapest), an open international marketplace (eBay) to connect the two economies, and $300K in technology training grants; this is all it takes to create a global crime syndicate whose victims exist 100% in a different country. It's the opposite of the well-organized, well-funded RBN activity we've seen so much of the past few years. Street hustling for the Internet age.

Fascinating. There's a sociology PhD thesis in there somewhere.

AB1298

A colleague of mine sent me this article, which should be of interest to pretty much everyone in the health care or human resources fields. AB1298 is an assembly bill that updates SB1386, California's much-copied breach disclosure law. The bottom line is that now an individual's health insurance ID number (which is hopefully not also their SSN) is considered PII much the same way a credit card number is. And when that data along with the corresponding name is breached, you must notify the victim.

It makes perfect sense. That number, combined with proper billing information, is enough to receive health care services from any participating medical provider. And, while I have pretty decent credit, I don't have a platinum card with a six-figure limit. But, if it were medically necessary, my insurer could be charged that kind of bill. And I would be responsible for the deductible. And, unlike my credit card's maximum personal loss, my deductible is not $50. So as an individual I stand to suffer greater financial loss if my medical identity is stolen versus my credit card.

In an America where health coverage is a problem for 47M people and the rising cost of health care is a problem for the rest, it doesn't seem at all far-fetched that trading in stolen health insurance information could become a lucrative criminal enterprise. And that would make health care data a real target.

ArcSight / CEF Patch for Snort / Barnyard

Last week Colin Grady released a patch to the Snort output tool, Barnyard 0.20, that allows you to output in ArcSight CEF format. I was going to post something here about it last week because of its sheer coolness, but then decided to hold off until I had a chance to play with it myself.

It built flawlessly, and it was easy enough to set up. It creates a new module in Barnyard named "alert_cef" that shovels CEF format messages to a syslog server (like ArcSight Logger). An example barnyard.conf might look something like this:

config daemon
config localtime
config hostname: arnold:eth1
config interface: eth1
config sid-msg-map: /opt/snort/rules/sid-msg.map
config gen-msg-map: /opt/snort/rules/gen-msg.map
config class-file: /opt/snort/rules/classification.config
output alert_cef arnold arcsight_logger.mydomain.local 16 1 514

Those arguments to the 'output alert_cef' line are, in order:
hostname
syslog server
syslog facility (integer format 16=local0)
syslog severity (integer format 1=alert)
syslog server port (UDP)

So here are my thoughts on Colin's patch. On the plus side, Barnyard is fast and lightweight, especially in comparison to the ArcSight Connector bundle which is several hundred MB on disk and in memory because it's Java. On the neutral side, its use case is pretty specific - you have ArcSight Logger collecting syslog data and forwarding to ESM. (Or you do your own thing with CEF and not ArcSight.) And on the down side, bypassing the ArcSight Connector means that you lose the categorization/prioritization stuff that ArcSight does for you with its AUP updates. And it also means no packet payload. For that you need to be running the ArcSight SnortDB Connector and logging Snort to a SQL database (preferably via Barnyard).

So that's still the architecture that I recommend for an enterprise Snort-to-ArcSight deployment. Snort doing unified logging, Barnyard shoveling logs into a SQL server on another host, ArcSight SnortDB Connector (also not on the Snort sensor host) querying that data from SQL and handing it directly to ESM. That gets you maximum scalability and functionality.

Nonetheless, Colin's patch is pretty cool, and definitely on the table for folks that have Logger and ESM. Additionally, if you use Snort and Barnyard, you should look at some of the other patches Colin has on his site. In particular, I think we will be testing and deploying his schema patch for Barnyard. Wish I had known about it 2 years ago.