Paul Melson's Blog

SIEM Market Redux

2011-10-26T19:57:00.004-05:00

Roughly a year and a half ago, Rocky DeStefano and I had a conversation about the SIEM market in which he predicted mass acquisitions. It took longer than he originally guessed, but...

I'll throw another one out there - after more than 2 years of steady speculation of a Splunk IPO, they hired David Conte as CFO fresh from his setting up the sale of IronKey to Imation. Expect Splunk to be acquired in 2012, or at least try really hard.

GrrCON: West Michigan Security Conference

2011-05-13T13:52:00.003-05:00

The Grand Rapids chapter of ISSA has announced a new event called GrrCON. It's a 1-day security conference that will be held in Grand Rapids, MI in September 2011. This will be one to keep an eye on over the next month or so as they get the speaker line-up solidified. I expect some cool talks and even a few surprises!

Website: http://www.grrcon.org/
Twitter: @GrrCON
LinkedIn Events: GrrCON 2011

Information Security for Business Majors

2010-10-09T22:11:00.003-05:00

I recently had the pleasure of guest lecturing to a group of MBA students at Grand Valley State University on the topic of Information Security. This was a fun presentation for me to put together because it challenged me to think of how to present the business value of information security in a way that's meaningful and relate-able to a wider audience not already indoctrinated with the market and regulatory constraints in which I operate. And in this case, I think I pulled it off.

So here are the slides from that presentation, minus a few that won't translate and aren't core to the presentation itself anyway. I've also included my slide notes by title below.

Information security for business majors

View more presentations from pmelson.

1. Title

What part of the car allows it to go fast? I think it’s brakes.

How fast would you drive if your car had no brakes?

Security is like brakes - it's a set of controls, only some of which are always on, that allows your company to take bigger risks with greater confidence.

Who can define what a DoS attack is?

And can anyone describe to me how the DoS handshake works? (in the book, figure 6.6, page 269)

I am sorry to inform you that you will never need to know this. Every major vendor out there fixed this bug nearly a decade ago.

That’s the nature of security – you don’t get to stop learning or adapting, because the attackers don’t stop learning and adapting.

I hope to share with you this evening things that will take a little longer to become obsolete.

2. OK, so how bad is it really?

3. Are you scared? ...or skeptical?

Gauge your response to the previous slide.

If you were scared, consider whether or not you would panic in the face of a catastrophic security event.

If you were skeptical, consider whether or not you would take a threat serious enough to be prepared.

4. The sky is always falling!

The average time for an unpatched Windows server on the Internet to be compromised is 3-6 hours.

The overwhelming majority of data breaches are caused by human error.

If you have any one of these things, hackers can monetize them.

Computers of any kind can be rented out to send spam or launch DDoS attacks.

Personal data, referred to as “dumps” are stolen and sold by the thousand on the Internet

Money in bank accounts is transferred by EFT and then wire transfer out of the country where it is laundered.

Credit card numbers are used to purchase stolen goods which are shipped overseas.

Despite all of this, consumer-based ecommerce continues to grow 15-20% annually.

If you sell to consumers, the Internet isn’t where you want to be, it’s where you HAVE to be.

5. Information Security's Business Value

Information security can be summed up as “loss avoidance”

The value proposition is that these efforts are less expensive than the consequences of not having them.

Regulation makes some parts of security the price of admission, the rest is about striking a balance between security and flexibility.

Bruce Schneier’s book, Beyond Fear.

6. How Information Security Works

Known as the CIA Triad, these are the “ilities” that security controls impact directly.

There are other “ilities”

Flexibility

Scalability

Portability

Profitability

But even at its best, security is only an enabler of these things. At either extreme, security blocks them.

7. The Goals of Security

8. Policy

9. Controls

Preventive IT controls are not infallible, and covering 100% of corner cases with your controls costs too much and hamstrings your actual business.

Auditing controls are time-consuming, and usually any damage is already done by the time an audit discovers it.

Monitoring controls are typically based on sampling, which means you might miss something. More intended as a quality or health check.

10. Tools of The Trade - Preventative

11. Tools of The Trade - Auditing

12. Tools of The Trade - Monitoring

13. Risk Management (1)

14. Risk Management (2)

15. Incident Response

I like the Richard Clarke quote from your book. “If you spend as much on information security as you spend on coffee, you will be hacked, and you’ll deserve to be hacked.”

Of course, Mr. Clarke is wrong, because having a security incident is not an issue of if, but an issue of..?

Wrong. Not “when” but “how often.”

16. (graph)

17. Awareness & Consultation

Consulting on projects or with operations teams leads to better security outcomes because security is considered earlier in the process.

Raising awareness and then inviting people to share concerns is a great way to organically scale your visibility to issues.

By being proactive and meeting colleagues where they are, you gain goodwill for your security efforts. This is a key piece of a successful security program. Strong-arm tactics are a guaranteed path to failure. Without goodwill and trust, the security practice in your company quickly becomes an obstacle for people to bypass in order to get their jobs done. This is how you lose your job.

18. How IT Security Fails

19. (image)

20. (image)

21. (image)

22. You say "potato," I say "No."

23. Communication

24. Why Buying Security Fails

Buying and integrating security technology only works some of the time, and that time is not right now.

Information security is an arms race.

Technology is both the weaponry and the battlefield.

Security is not a problem that can be solved.

Security is a practice that must be maintained with people and process.

25. Discussion

The SIEM Market Discussion Continues

2010-05-20T20:44:00.006-05:00

Bill Roth of LogLogic commented on my Twitter exchange with Rocky DeStefano of Visible Risk where we talked about LogLogic's announcement that they were discounting their SIEM product. I then wrote a reply, and it got a little long. So I made it a blog post instead.

Rocky, Paul:
The ClueTrain Manifesto calls markets "conversations", so here goes.....

I think you're falling into a the trap of "conventional wisdom". First off, the basic assumption that the world falls neatly into the SIEM categorization is just plain false. I stand by LogLogic's model....it all starts with log management as the crucial piece, without that key use cases like network forensics are not even possible. Second, the notion that dropping the price is bad is just plain weird. Is LogLogic dropping the price to sell more? Sure we are. Are we dropping the price to take market share? Sure we are. Are we seeing a great response? Sure we are. Since when is saving people money a bad thing?

And we're always interested in a podcast. :)

Bill Roth, EVP
LogLogic

Hi Bill,

Thanks for the comment! And thanks for participating in the dialogue. I think it's awesome that LogLogic is out front and engaging on its business decisions. Very refreshing!

As to your point about log management being that crucial initial component of a SIEM implementation, I agree completely. Log management has also developed as its own market segment as well, independent of SIEM. But I don't need to tell you that. :-)

On the topic of LogLogic's decision to discount its SIEM product, I didn't mean - and I don't believe Rocky did either - that charging less for SIEM is bad, or even a bad business move.

That said, I do believe that for some significant portion of potential customers log management is a commodity technology. However, from my own experience and from everything I've seen to date, SIEM is not a commodity technology, and I'm not convinced it will be. As such, I don't see price as a strong competitive differentiator in the SIEM market.

Following the recent recession, where IT capital budgets still haven't caught up to the (hopefully sustained) economic upturn, I imagine the feedback on LogLogic's price cut has been positive, and that you'll see some SIEM sales where you wouldn't have but for the discount. But in the mid- to long-term, I have my doubts as to whether there is any meaningful gain in market share to be had for LogLogic - or any SIEM vendor for that matter - simply by competing on price with other SIEM vendors.

Let's be frank, if price were a big piece of why companies choose a particular SIEM, Cisco MARS would have the lion's share of the market and ArcSight would be folding. Instead, it's the other way around.

Twitter Killed the Blog Star

2010-05-20T08:17:00.006-05:00

I've been really busy both in my personal and professional life for the past year or so, with no signs of slowing down soon. But I have to acknowledge that the main reason my blog posts have fallen off is Twitter. Now, all of the ideas that I have that I might have developed and expanded into a blog post are prematurely evaluated for length. If they can be abbreviated to a couple of 140-character haikus or less, they go on Twitter. Which means they never grow up to be blog posts. They're like the high school dropouts of ideas.

But every once in a while, a Twitter exchange becomes so interesting that, despite the compressed and fleeting nature of Twitter, it turns into something worthy of framing. The other night, Rocky DeStefano of Visible Risk and I had an exchange on SIEM that I thought the wider world might find interesting. The background to the conversation is this post from Rocky's blog about the recent announcement from LogLogic that they were discounting their SIEM product, and then this responding blog post from LogLogic.

rockyd
The LogLogic response ->> http://bit.ly/bAQSZO to my discounting SIEM Post ( http://bit.ly/aiW3kB )
8:47 PM May 18th via TweetDeck

rockyd
I need to noodle on the LogLogic response more. I appreciate the conversation, I think I may see the opposite end of the customer spectrum.
9:02 PM May 18th via TweetDeck

pmelson
@rockyd I think you nailed the issue. If you *NEED* SIEM, you won't compromise features/functionality for capital cost savings.
9:06 PM May 18th via TweetDeck

pmelson
@rockyd If Cisco couldn't make "Free SIEM With Purchase" work, it's not ever going to work.
9:07 PM May 18th via TweetDeck

rockyd
@pmelson let's be honest how could they possible respond any differently than they did? time for a podcast on the subject ?
9:50 PM May 18th via TweetDeck

pmelson
@rockyd They could just fess up. "We're shipping log management appliances, but SIEM isn't moving. So we put it on clearance sale." :-)
9:52 PM May 18th via TweetDeck

pmelson
@rockyd I think with Gartner's SIEM MQ being released, we're about to see another round of SIEM casualties as VC pulls out.
9:54 PM May 18th via TweetDeck

rockyd
@pmelson There has to be quickening soon, there is way too much of the same thing in the market.
9:57 PM May 18th via TweetDeck

pmelson
@rockyd Right. I've been thinking about the key SIEM differentiators and I've only got three.
10:00 PM May 18th via TweetDeck

rockyd
@pmelson which three?
10:06 PM May 18th via TweetDeck

rockyd
@pmelson Like - Sources, Scalability, Analytical Usage, Correlation / Statistical Evaluation, and getting Intelligent information out?
10:08 PM May 18th via TweetDeck

pmelson
@rockyd 1) performance/scalability 2) UI and drill-down 3) supported sources.
10:07 PM May 18th via TweetDeck

rockyd
@pmelson there are some others like context of Host, Vuln, Registry, Applications and Users that lead you towards more advanced usage
10:09 PM May 18th via TweetDeck

pmelson
@rockyd OK, so asset data model(s) makes 4, pre-defined content is 5? That's still not a lot.
10:15 PM May 18th via TweetDeck

rockyd
@pmelson each is several years of development and refinement with customers.
10:32 PM May 18th via TweetDeck

rockyd
@pmelson this comes down to a compliance check box sale versus a security team needing to integrate a tool into their process.
10:35 PM May 18th via TweetDeck

pmelson
@rockyd Agree. But a handful of differentiators == a handful of potential market leaders. Time to thin the herd. Again.
10:42 PM May 18th via TweetDeck

rockyd
@pmelson now I see where you're headed. BTW I think you'll see 3 more acqusitions by end of year.
10:45 PM May 18th via TweetDeck

rockyd
I was thinking about creating a "vegas odds" website for SIEM Quickending and donate some portion of the funds to HFC.
10:47 PM May 18th via TweetDeck

pmelson
@rockyd A SIEM futures market? Very DARPA!
10:49 PM May 18th via TweetDeck

So there, for your parsing and edification, some thoughts on the SIEM product space, the recent Gartner MQ for SIEM, and the near-term ramifications of Gartner's paper on the market.

Also, if you aren't already, you should be reading Rocky's blog, especially if you're interested in SIEM and security ops. Rocky's a guru in this space, and in addition to his blog he has already put together some great podcasts since launching his latest venture, Visible Risk.

Snort Signatures for New Koobface Variant

2010-04-14T22:20:00.002-05:00

The first rule is actually how we caught the first incident. The binary is served on non-standard HTTP ports via fast-flux servers. It's a signature we've had in place for years.

alert tcp $HOME_NET any -> $EXTERNAL_NET !80 (msg: "LOCAL .exe file download on port other than 80"; flow:established; content: "GET"; depth:4; content: ".exe"; nocase; classtype:misc-activity; sid:9000160; rev:1;)

And these are designed to catch the bot HTTP checkins we've seen so far. This is likely to be more of a whack-a-mole effort as we've already seen the checkin URL format change once.

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"LOCAL Koobface action=fbgen checkin"; flow:to_server,established; content:"POST"; content:"/.sys/?
action=fbgen"; nocase; classtype:trojan-activity; sid:9000220; rev:1;)

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"LOCAL Koobface go.js checkin"; flow:to_server,established; content:"POST"; content:"/go.js?"; nocase; classtype:trojan-activity; sid:9000221; rev:1;)

Security Metrics and Data Visualization

2010-01-22T12:30:00.006-05:00

I've just finished compiling the security incident handler case statistics for 2009. This is the second year in a row that I've used the same set of metrics, and having two years worth of data has led to some interesting observations about security trends within my employer's environment.

One set of statistics that may be of interest to the general Internet public is the volume of malware cases that we have worked over the past two years.

There are a couple of things worth pointing out in this graph. The first, and perhaps most obvious one, is that there is a drop-off in malware related cases in 2009. Surely, that can't be right? It is, but it's due to implementing some new security technologies in December of 2008. In fact, those countermeasures reduced the number of malware cases we handled in 2009 by roughly 65% compared to 2008. I want to say two things about this. First, this demonstrates the effectiveness of the preventative countermeasures that we employed and confirms the value of those countermeasures. Notice that I'm not saying that it proves ROI. But the bottom line is that it was worth it. The second thing I want to point out about that decline, however, is that it's just a decline. It did not eliminate the problem. In fact, in 2009 we saw malware chip away at other defenses that were highly effective only two years before. And I suspect that, if we do nothing else about it, that those levels will begin to rise in 2010 and regain the same level of frequency we saw in 2008 if not higher. There's a hint of that in the graph towards the end of 2009.

The next thing I want to point out about this graph is the peak frequency. It is consistent. Every three months, there is a spike in malware incidents in our environment. I would love to see statistics from other companies or the Internet at large to see if this is an Internet-wide pattern. I suspect that it is. Despite the new countermeasures, despite the decrease in order of magnitude, the spikes occur like clockwork every third month. That leads me to believe two things. First, I believe that this pattern is driven externally since it didn't deviate, even when our environment changed significantly. Second, I believe that this is no accident. The vendors that produce malware/botnet "kits" are responsible for introducing most of the new exploits and anti-detection capabilities that we see on a regular basis. Their stuff is used more widely than custom malware as well. Therefore, this leads me to believe that there is one large group responsible for the majority of the malware in the wild, and they're on a 90-day release cycle. I've got no intelligence data to support this, but I have a hard time believing that this pattern repeats itself, without exception, for two years straight out of pure coincidence.

Bottom line, this is the kind of useful information that trend analysis can give you, and why metrics are worth gathering and analyzing.

Malware Analysis Toolkit for 2010

2009-12-28T15:09:00.013-05:00

Back in 2008 I posted a list of the tools I use for doing malware analysis. The tools I use have changed over time, and rather than just talk about a couple of recent additions, I decided I'd put a current complete list up with links. This is by no means a comprehensive list of malware analysis tools, it's just what I like and use.

Platform

VMWare Workstation
The "vulnerable stuff:"

Windows XP
Internet Explorer 7/8
Firefox
Acrobat Reader
Flash Player

General Tools

Analysis Tools

Binary Tools

JavaScript & HTTP Tools

PDF & Flash Tools

Web Sites as Tools

ArcSight Logger VS Splunk

2009-11-18T19:14:00.005-05:00

You are here because you are searching for information on Splunk vs. ArcSight Logger. I actually wrote this post months before posting it, but sat on it for reasons that may become apparent as you read on.

If you want to hear me talk about my experience with Logger 4.0 through the beta process and beyond, you can check out the video case study I did for ArcSight. In short, Logger is good at what it does, and Logger 4.0 is fast. Ridiculously fast.

But that's not what I want to talk about. I want to talk about the question that's on everyone's mind: ArcSight Logger vs. Splunk?

Comparing features, there's not a strong advantage in either camp. Everybody's got built-in collection based on file and syslog. Everybody's got a web interface with pretty graphs. The main way Logger excels here is in its ability to natively front-end data aggregation for ArcSight's ESM SIEM product. But if you've already got ESM, you're going to buy Logger anyway. So that leaves price and performance as the remaining differentiators.

Splunk can compete on price, especially for more specialized use cases where Logger needs the ArcSight Connector software to pick up data (i.e. Windows EventLog via WMI, or database rows via JDBC). And if you don't care about performance, implying that your needs are modest, Splunk may be cheaper for you for even the straightforward use cases because of the different licensing model that scales downward. So for smaller businesses, Splunk scales down.

For larger businesses, Logger scales up. For example, if you need to add storage capacity to your existing Logger install, and you didn't buy the SAN-attached model, you just buy another Logger appliance. You then 'peer' the Logger appliances, split or migrate log flows, and continue to run search & reporting out of the same appliance you've been using, across all peer data stores. With Splunk? You buy and implement more hardware on your own. And pay for more licenses.

My thinking on performance? Logger 4.0 is a Splunk killer, plain and simple. To analogize using cars, Splunk is a Ford Taurus for log search. It gets you down the road, it's reliable, you can pick the entry model up cheap, and by now you know what you're getting. Logger 4.0, however, is a Zonda F with a Volvo price tag.

To bring the comparison to a fine point, I'd like to share a little story with you. It's kind of gossipy, but that makes it fun.

When ArcSight debuted Logger 4.0 and announced its GA release at their Protect conference last fall, they did a live shoot-out of a Logger 7200 running 4.0 with a vanilla install of Splunk 4 on comparable hardware and the same Linux distro (CentOS) that Logger is based on. They performed a simple keyword search in Splunk across 2 million events, which took just over 12 minutes to complete. That's not awful. But that same search against the same data set ran in about 3 seconds on Logger 4.

This would be an interesting end to an otherwise pretty boring story if it weren't for what happened next. Vendors other than ArcSight - partners, integrators, consultants, etc. - participate in their conference both as speakers and on the partner floor. One of these vendors, an integrator of both ArcSight and Splunk products, privately called ArcSight out for the demo. His theory was that a properly-tuned Splunk install would perform much better. Now, it's a little nuts (and perhaps a little more dangerous) to be an invited vendor at a conference and accuse the conference organizer of cooking a demo. But what happened next is even crazier. ArcSight wheeled the gear up to this guy's room and told him that if he could produce a better result during the conference that they would make an announcement to that effect.

Not one to shy away from a technical challenge, this 15-year infosec veteran skipped meals, free beer, presentations, more free beer, and a lot of sleep to tweak the Splunk box to get better performance out of it. That's dedication. There's no doubt in my mind that he wanted to win. Badly. I heard from him personally at the close of the conference that not only did he not make significant headway, but that all of his results were worse than the original 12 minute search time.

You weren't there, you're just reading about it on some dude's blog, so the impact isn't the same. But that was all the convincing I needed.

But if you need more convincing; we stuffed 6mos of raw syslog from various flavors of UNIX and Linux (3TB) into Logger 4 during the beta. I could keyword search the entire data set in 14 seconds. Regex searches were significantly worse. They took 32 seconds.

Reversing JavaScript Shellcode: A Step By Step How-To

2009-11-09T11:26:00.028-05:00

With more and more exploits being written in JavaScript, even some 0-day, there is a need to be able to reverse exploits written in JavaScript beyond de-obfuscation. I spent some time this weekend searching Google for a simple way to reverse JavaScript shellcode to assembly. I know people do it all the time. It's hardly rocket science. Yet, I didn't find any good walk-throughs on how to do this. So I thought I'd write one.

For this walk-through, I'll start with JavaScript that has already been extracted from a PDF file and de-obfuscated. So this isn't step 1 of fully reversing a PDF exploit, but for the first several steps, check out Part 2 of this slide deck.

What you'll need:

A safe place to play with exploits (I'll be using an image in VMWare Workstation.)
JavaScript debugger (I highly recommend and will be using Didier Stevens' modified SpiderMonkey.)
Perl
The crap2shellcode.pl script, which you'll find further down in this post
A C compiler and your favorite binary debugger

I'll be using one of the example Adobe Acrobat exploits from the aforementioned slides for this example. You can grab it from milw0rm.

Step 1 - Converting from UTF-encoded characters to ASCII
Most JavaScript shellcode is encoded as either UTF-8 or UTF-16 characters. It would be easy enough to write a tool to convert from any one of these formats to the typical \x-ed UTF-8 format that we're used to seeing shellcode in. But because of the diversity of encoding and obfuscation showing up in JavaScript exploits today, it's more reliable to use JavaScript to decode the shellcode.

For this task, you need a JavaScript debugger. Didier Stevens' SpiderMonkey mod is a great choice. Start by preparing the shellcode text for passing to the debugger. In this case, drop the rest of the exploit, and then wrap the unescape function in an eval function:

Now run this code through SpiderMonkey. SpiderMonkey will create two log files for the eval command, the one with our ASCII shellcode is eval.001.log.

Step 2 - crap2shellcode.pl
This is why I wrote this script, to take an ASCII dump of some shellcode and automate making it debugger-friendly.

---cut---


#!/bin/perl
#
# crap2shellcode  - 11/9/2009 Paul Melson
#
# This script takes stdin from some ascii dump of shellcode
# (i.e. unescape-ed JavaScript sploit) and converts it to
# hex and outputs it in a simple C source file for debugging.
#
# gcc -g3 -o dummy dummy.c
# gdb ./dummy
# (gdb) display /50i shellcode
# (gdb) break main
# (gdb) run
#

use strict;
use warnings;

my $crap;
while($crap=<stdin>) {
  my $hex = unpack('H*', "$crap");

  my $len = length($hex);
  my $start = 0;

  print "#include <stdio.h>\n\n";
  print "static char shellcode[] = \"";

  for (my $i = 0; $i < length $hex; $i+=4) {
    my $a = substr $hex, $i, 2;
    my $b = substr $hex, $i+2, 2;
    print "\\x$b\\x$a";
  }
  print "\";\n\n";
}

print "int main(int argc, char *argv[])\n";
print "{\n";
print "  void (*code)() = (void *)shellcode;\n";
print "  code();\n";
print "  exit(0);\n";
print "}\n";
print "\n";

--paste--

The output of passing eval.001.log through crap2shellcode.pl is a C program that makes debugging the shellcode easy.

Step 3 - View the shellcode/assembly in a debugger
First we have to build it. Since we know that this shellcode is a Linux bindshell the logical choice for where and how to build is Linux with gcc. Similarly, we can use gdb to dump the shellcode. For Win32 shellcode, we would probably pick Visual Studio Express and OllyDbg. Just about any Windows C compiler and debugger will work fine, though.

To build the C code we generated in step 2 with gcc, use the following:

gcc -g3 shellcode.c -o shellcode

The '-g3' flag builds the binary with labels for function stack tracing. This is necessary for debugging the binary. Or at least it makes it a whole lot easier.

Now open the binary in gdb, print *shellcode in x/50i format, set a breakpoint at main(), and run it.

$ gdb ./shellcode
(gdb) display /50i shellcode
(gdb) break main
(gdb) run

Two-For-One Talk: Malware Analysis for Everyone

2009-10-18T15:10:00.002-05:00

These two mini-talks were originally going to be blog posts, but I needed a speaker for this month's ISSA meeting. So I volunteered myself. Here are the slides.

Two-For-One Talk: Malware Analysis for Everyone

View more presentations from pmelson.

Queries: Excel vs. ArcSight

2009-09-23T22:07:00.020-05:00

Since ArcSight ESM 4.0, reports and trends have been based on queries. Considering that ESM runs on top of Oracle, a query in ESM is exactly what you think it is. Queries are an extremely flexible way to get at event data. But as the name implies, they go against the ARC_EVENT_DATA tablespace, and therefore you can't use them to build data monitors or rule conditions, since those engines run against data prior to insertion into the database.

Anyway, I've got a story about how cool queries are. And about how much of an Excel badass I am. And also about how queries are still better. Last month, I got a request from one of our architects who was running down an issue related to client VPN activity. Specifically, he wanted to know how many remote VPN users we had over time for a particular morning. Since we feed those logs to ESM, I was a logical person to ask for the information.

So I pulled up the relevant events in an active channel and realized that I wasn't going to be able to work this one out just sorting columns. So, without thinking, I exported the events and pulled them up in Excel. So here's the Excel badass part:

If you want to copy it, here it is:
=SUM(IF(FREQUENCY(MATCH(A2:A3653,A2:A3653,0),MATCH(A2:A3653,A2:A3653,0))>0,1))

So A is the column that usernames are in. This formula uses the MATCH function to create a list of usernames and then the FREQUENCY function to count the unique values in the match lists. You need two MATCH lists to make FREQUENCY happy because it requires two arguments, hence the redundancy. It took about an hour for me to put it together, most of that was spent finding the row numbers that corresponded to the time segment borders.

But as I finished it up and sent it off to the requesting architect, I thought, there must be an easier way. And of course there is. So here's how you do the same thing in ESM using queries:

So, it's just EndTime with the hour function applied, and TargetUserName with the count function applied, and the Unique box (DISTINCT for the Oracle DBA's playing at home) checked. And then on the Conditions tab you create your filter to select only the events you want to query against. That's it.

Once the query is created, just run the Report Wizard and go. All told, it's about 90 seconds to the same thing with a query and report that it took an hour to do in Excel.

The 'Cyberwarfare' Problem

2009-09-20T23:24:00.003-05:00

Last week I attended ArcSight's annual user conference in Washinton DC. More about that in a later post. During the conference, ArcSight hosted a panel discussion on cyberwarfare. In DC, where many of ArcSight's biggest customer are based, this is a hot topic, and there will be a lot of time spent discussing it and a lot of money spent on defending against it, maybe.

What struck me about the panel discussion were two comments, both made by James Lewis, one of the panelists, and a director at the Center for International and Strategic Studies. At one point, Mr. Lewis invoked Estonia as an example of state-sponsored cyberwarfare, and made the comment that, "the Russians are tickled that they got away with it." Not ten minutes later, an audience member asked a question about retaliation against cyber-attacks. Mr. Lewis responded to the question by pointing out the problem of attribution. That is, from the logs that the victim systems generated, the IP address(es) recorded can't reliably be used to identify the actual individual(s) responsible for the attack.

Now, I don't intend to pick on James Lewis. It just so happened that one person on the panel expressed the paradox of cyberwarfare. The attribution problem is a big problem for all outsider attacks, not just cyberwarfare. A decade ago, security analysts were calling it "the legal firewall" because US-based hackers would first hack computers in China, Indonesia, Venezuela, or another country that doesn't openly cooperate with US law enforcement, and then hack back into the US from there, causing an investigative barrier that would hinder or prevent an investigation being able to get back to the attacker's actual location.

So knowing that there's a very real problem with being able to identify the source country for Internet-based attacks, it stands to reason that using the same limited forensic data to not only identify the actual source of an attack, but to determine that it is in fact state-sponsored, and not, say, a grassroots attack armed by a teenager, is a stretch. And for that reason, the question of cyberwarfare is an open one. Until a government actually comes forward and claims responsiblity for an attack, it's unprovable.

So as the government spends $100M on cyberdefense over the next six months, it's important to try and answer the question, "What is the military actually defending against?" At the very least, it's fair to say nobody knows for certain.

Inbox 3

2009-08-12T15:57:00.004-05:00

Teguh writes,

Hi Paul,
could you give some guide to administering logger? i searched thru
google, but found nothing significant. How to(s) and tutorial would be enough i
guess. Does it have to have syslog server for the logger to be able to read data
from?
Thanks..

The documentation for Logger is available from ArcSight's download center. Only registered customers have access, but I assume that if you've got a Logger box, that generally qualifies you.

With regard to your second question, yes Logger has a syslog server. It actually has a few. In Logger nomenclature these are "receivers." Logger supports UDP and TCP syslog, FTP and SSH file pull, NFS and CIFS remote filesystem. Logger also supports some ArcSight-specific receivers including a SmartMessage receiver for events forwarded from ESM and CEF-over-syslog (OK, ArcSight wouldn't agree that this is specific to their products, but despite the C standing for Common, CEF is anything but. At least right now.)

Configuring Logger to act as a syslog server is pretty straightforward.
From the web interface, navigate to Configuration, Event Input/Output.
On the "Receivers" tab, click the Add button.
Name your connector and set the type as "UDP Receiver" then click Next.
The defaults for Compression Level and Encoding are fine. Select the IP address you want the listener to reside on, and set the port number. The default syslog server port is UDP/514.
Click Save.
On the "Receivers" tab, click the little no-smoking image next to the new receiver to enable it.

Nobody Sells Laptops for The Price of Silver

2009-06-23T12:47:00.005-05:00

If you haven't already, I recommend that you take 20 minutes and read "Nobody Sells Gold for the Price of Silver" by Cormac Herley and Dinei Florencio. (PDF Link) This is an excellent analysis of the research into and press coverage of the underground economy. It's a fascinating read, and they make a cogent argument that the underground economy is more myth than reality. I don't want to say more because it will ruin it for you.

Now I have an excercise for you. First, read the Herley/Florencio article. Then, read Bruce Schneier's experiences with trying to sell a laptop on eBay. Now think about the implications of the "Ripper Tax" on eBay. Now ask yourself why you haven't already sold any stock you own in eBay.

PCI-DSS and Encrypting Card Numbers

2009-06-18T22:22:00.009-05:00

OK, I'm about to do something dumb and talk about cryptography and cryptanalysis. I'm an expert in neither of these things. But despite the fact that somebody smarter than me should be telling you this, you're stuck with me, and I think I have a point. So here goes.

I had a bit of an "A-ha!" moment earlier today around PCI-DSS, specifically requirement 3.4 from v1.2 of the standard. Here's the relevant language from that requirement:

3.4 Render PAN, at minimum, unreadable anywhere it is stored (including on portable digital media, backup media, in logs) by using any of the following approaches:
One-way hashes based on strong cryptography
Truncation
Index tokens and pads (pads must be securely stored)
Strong cryptography with associated key-management processes and procedures

The bottom line is that this requirement fails to provide adequate protection to card numbers. Here's why.

Truncation and tokenized strings with pads have limited use cases. In the case of truncating card numbers, PCI-DSS recommends only storing the last 4 digits of the card number. You wouldn't choose truncation for a program that validates a card number because there would be too great a potential for false matches. It would only be helpful for including in receipts, billing statements, and for use in validating a customer identity in conjunction with other demographic information. Database tokens only provide adequate protection in environments where there is a multi-user or multi-app security model, and if there are flaws in the applications that have access to the pads, then your data is pwned.

So for the sake of maximum versatility and security, you're likely (or your software vendor is likely) to opt for hashing or encryption. But you still have a serious problem. While one-way hashes like SHA and block ciphers like AES can provide good protection to many forms of plaintext, credit cards aren't one of them. That's right, the problem isn't actually in the way you encrypt credit card numbers, it's that credit card numbers make for lousy plaintext to begin with.

Take for example the following row of data from my hypothetical e-commerce application's cardholder table:

LNAME,FNAME,CTYPE,EXP,HASH,LASTFOUR
Melson,Paul,DISCOVER,06/2009,e4b769607856a2f30b57fd26079dfefb,1111

In this case, we have what we need to use the card, except the card number is hashed with MD5. (Ignore what you know about MD5 collisions for a moment, since this problem also exists for SHA or any other method of encrypting the card number.) If we calculate the possible number of values that could be on the other side of that hash, it would be 10^16, or about 10,000 trillion for the 16-digit card number. That's roughly twice as many possibilities as an 8-character complex password (96^8), which is an acceptable keyspace size, but also completely doable for a tool like John The Ripper.

But if you know credit card numbers, then you've already realized that it's even worse than that. The first 4-6 digits of the card number are a misnomer in calculating keyspace. There aren't 1 million actual possible values. Since that row from my e-commerce app's database told me the card issuer, I know within 4-5 guesses the first two to four digits of the card number, and the last four are right there as well for inclusion on statements, etc. In this case, since it's a Discover card, we already know that the card number is 6011XXXXXXXX1111. Now we've cut the possible values we must guess in half, from 10^16 down to 10^8, which is a mere 100 million possibilities. There are other clever things we can do if it's encrypted with a stream cipher like RC4 or FISH, because we know the beginning and end values of the plaintext. But guess what? It's cheaper and easier to brute-force it even if lousy crypto is used. Even on the scale of millions of records. Even with salting, it's still worth it to brute-force the middle digits.

But wait, there's more! As if publicly known prefix values weren't enough, credit card numbers are also designed to be self-checking. That is to say, the numbers contain something like a checksum that, when a known algorithm is applied to the 7-digit account number, 3 digits of which we know from our last-four field, can be used to validate the card number. This was designed as an anti-fraud mechanism that would allow cards to be checked without a need to communicate with a clearinghouse. But this algorithm allows us to only generate valid account numbers, combined with partially-known prefixes, to reduce the keyspace significantly. And since this is a known algorithm I can (and someone already has) very easily write a tool that combines a brute-force password cracker with a credit card generator.

The bottom line is that, because of the already-partially-known nature of credit card numbers, simply encrypting card numbers inside a database or extract file is insufficient protection. The PCI Security Standards Council should revisit this requirement and modify it to, at the very least, require symmetric-key block ciphers and disallow stream ciphers and one-way hashes. But even then, I suspect, encrypted card numbers will be at risk. Certainly row-level encryption of card numbers should not qualify for "safe harbor" when it comes to breach notification laws.

PS - Extra credit if you crack the full card number from the hash above and post it below.

From The Inbox 2

2009-06-11T18:56:00.003-05:00

lmran writes:

Hi Paul,
Do you know any reason why ArcSight ESM does not support the Cisco MARS? Right now, all my firwalls send the syslog feeds into Cisco MARS and I'm trying to set the Cisco MARS to send thoes raw feeds data to ArcSight local connector but I just found out that ArcSight does not support the Cisco MARS. Thanks in ADV for any info reading this subject.

Starting in 4.x, MARS can forward events to another remote syslog listener. ArcSight has a syslog connector. So you ought to be able to forward events from MARS to ArcSight via syslog assuming MARS doesn't change the format of the log events too much. Even if MARS does mangle the event format, ArcSight will still receive them, but then most or all of the event will be parsed into the CEF Name field and categorization and prioritization won't be accurate.

If you are unable to upgrade your MARS appliance to 4.31 or later (I think that's the rev you need), another option would be to use a syslog-ng server out front. It supports forwarding events by source to other syslog servers. You could use this to send the stuff you want in ESM to ArcSight's syslog Connector and the stuff you want in MARS to MARS.

Or, you could do the environmentally conscious thing and unplug then recycle your MARS appliance. ;-)

From The Inbox

2009-06-09T12:32:00.004-05:00

Anonymous writes,

Hi Paul, I am one of those who, as you say, found your blog by googling ArcSight, trying to do some recon on the product for my employer. (I think I see that the most recent posts here are from 2007 so who knows if you or anybody will be seeing my question.) I'm trying to find out, can Arcsight's data be queried programmatically; i.e. is it stored in a relational database, hopefully SQL Server or Oracle, or if not, is there an API or ADO.NET provider that can allow it to be queried, preferably with SQL? Thanks for any info anyone reading can provide.

ArcSight ESM uses Oracle 10g for its back-end database. At one point, and this may still be true, DB2 was also supported. You can query the database directly, and the schema is pretty straightforward. The table ARC_EVENT_DATA is where most of the event data lives, for example. But depending on your use case, that might not be the best way to get data out of ESM.

Also, since you didn't specify, it may be worth mentioning that the same is not true of the ArcSight Logger platform, which is flat storage. Instead of querying the log store directly, Logger can be configured to forward events based on source, type, etc. to another destination, if you need them in real-time. There is a PostegreSQL database on Logger, but it's my understanding that it supports the reports engine, and doesn't store the raw or CEF events in any comprehensive way.

The interesting thing is that the storage technology behind Logger 3.0, because of its performance and relative "cheapness" may become the data store for ESM down the road. It would only make sense, since you could handle MUCH higher event rates with less disk and no Oracle license fee. If it can be done while maintaining the stability and feature set that the Oracle-based data store has, it's a walk-off home run for ArcSight.

New Rules

2009-06-01T22:02:00.004-05:00

After many months off, I'm jumping back in to the blog with both feet. Mostly in a Howard Beale sort of way. Didja miss me? Anyway, stealing a meme from Bill Maher, I've got something to say to security vendors. Without further ado, New Rules.

If you are a vendor, especially a vendor of security products or services, these are the rules I expect your product to follow. These are common sense, and I feel a little condescending telling them to you. But if recent experience is any indicator, you need to hear them. And you deserve the condescension.

Do not store credentials in clear text! Seriously, you can get free libraries to hash credentials or store them in a secure container file that requires a secret key. There's no reason for a password to be in a text file or HKLM Registry key. None.
Do not hardcode passwords! If I can't change every single password associated with your product simply and easily, then there should be a law that strips all of your developers of any degree they hold and forces them to go back to college and learn file IO methods.
Do not use HTTP/Telnet/FTP/LDAP for authentication! Seriously, more than enough free libraries for SSH, TLS, IPSec exist. Use one. Or buy the one you really like. It beats having to issue a "patch" to sell to government and regulated industry.
Don't run as root/SYSTEM/sa/DBA! Your product is not so special that it actually needs administrative privileges to run on the server or database that hosts it. Unless by "special" you mean "coded by lazy fools that don't want to define even the most basic security model." OK, then it is special.
Don't use broken crypto algorithms! Sorry, but if you are shipping new product that uses 56-bit DES, RC4, or ROT13, please see rule #3.
Don't send passwords in e-mail! Remote password reset is easy enough to do properly, there's no reason to be lazy and just send me my password if I forget it. Also, it means you're breaking rule #1. Busted.

There are no excuses for any product to not follow these rules, but especially security/compliance products. Gee, thanks. I just spent six figures on a product to help me manage or achieve compliance, and the product itself can't comply with the regulation I'm trying to address.

The Next Phase

2009-01-19T23:37:00.006-05:00

For those of you who haven't given up on my blog (or forgot it was still in your feed list), I want to let you know that I will be back to it later this year. More punditry, more metrics, more SIM, more cool random technical stuff. I'll try anyway. I've been missing it, but I had too much going on, had to prioritize, and this blog has rusted as a result.

A lot has changed since my last blog post in November - a new position at work, a new baby daughter - and the one thing that I've come to realize is that changing is hard work, but if you want it, it's worth it. There's been an excessive amount of talk about change this past year, and on the eve of President Obama's inauguration, I've decided to share with you this story of a moment I had recently.

On November 5th, the day after Election Day 2008, I spoke at the SecureWorld Expo conference in Detroit. I've been in West Michigan for the past several years, but I used to live and work on the East side of the state. It was a gorgeous Wednesday, clear and unseasonably warm for November. And as I was driving westbound on I-96, into the dusk between me and the sunset, I looked up and found myself in familiar territory - Webberville.

You've probably never heard of Webberville, Michigan. That's OK. It's a rural town on the automotive corridor where in the 1990's, companies got huge tax breaks to buy up farmland and build factories. And in 2001, I had an office in one of those factories. That company (a "Tier One" in industry lingo because we sold directly to car makers), like many automotive suppliers, has since gone out of business. And despite working there only a year, I have some very fond and vivid memories of that job. Perhaps the most vivid, however, is driving that stretch of I-96 between Webberville and Wixom and hearing the radio newscaster describe the second plane hitting the World Trade Center on 9/11.

That day changed everything for Americans. I was living in the Midwest, working in a one-story office that had highway on one side and cows on the other, but for the weeks that followed the attacks, I was afraid. We all were. I recall making that drive to Webberville again a week later while all of the planes were still grounded and thinking to myself, "How long until we recover? Can we recover? What will it take for us to move forward?"

Not get over it. Not forget. But move forward - take the next step as a society, as a culture, as a country.

So back to 11/5/2008, and my drive home from SecureWorld, less than 24 hours after learning that Barack Obama - a young, African-American man - would be our next president. And it was there, on that piece of highway in rural Michigan that I answered my own question. Seven years and two months later, I knew America was moving forward. We were moving forward.

SecureWorld Expo Slides

2008-11-21T16:02:00.003-05:00

Earlier this month I gave a talk at SecureWorld Expo Detroit on malware analysis. The goal of the talk was to discuss the state of malware and tools for people who aren't ready to go to town with a debugger. Unfortunately, to put it on SlideShare, I had to replace the cool Camtasia videos with lame screen shots.

Malware Analysis Made Simple

View SlideShare presentation or Upload your own.

SecureWorld Expo Detroit

2008-10-10T13:59:00.004-05:00

SecureWorld Expo Detroit is coming up at the beginning of next month. I will be presenting on operationalized malware analysis and response. In this case "operationalized" means, "without a debugger.

Cathy Luders, a friend and colleague that I met through the local ISSA chapter, is also presenting at SecureWorld. On the same day. At the exact same time. Which has me bummed out more than a little because I've not gotten to see her present before. But now that I know she's got a talk in her back pocket, I'll probably ask her to present at an upcoming ISSA meeting. :-)

ArcSight Tools Slide Deck

2008-10-10T13:56:00.002-05:00

Wow, I've just been buried, both at work and at home. I promised a sanitized copy of our slides from the ArcSight User Conference and here they are. A month late. Enjoy.

Managing ArcSight ESM Tools

2008-09-15T09:11:00.009-05:00

Last week Tim and I presented at the ArcSight user conference on using Tools in the ESM console to augment incident response and investigation. I'm hoping to have the sanitized slide deck up this week, and maybe a little bit of code to go with it.

At the end of the talk - and then a couple of times in the hallway - people asked how we manage all of these Tools. It's a great question, and the answer is, not very well. But here's the best way that I know how in ESM 4.0.

If you've ever looked at the tools editor in the ESM console, you've seen this dialog, which is pretty basic:

By default, this info is recorded in your AST file (C:\arcsight\Console\current\paul.ast), which is just a text file with a bunch of values declared. The values in the file look like this:

console.ui.tools[toolName].program=
console.ui.tools[toolName].workingdir=
console.ui.tools[toolName].iconFile=
console.ui.tools[toolName].parameters=
console.ui.tools[toolName].showInToolBar=
console.ui.tools[toolName].isExportTool=

And then there's this:

console.ui.toolsList=

Which is pretty self-explanatory. It occurs once in the AST file and is just a CSV list of the tool names, used to populate the console's Tools menu.

I wrote a Perl program that can parse an AST file and extract tool data from it for the purposes of sharing. It's designed to help scale and distribute tools across your analysts' consoles so that they don't have to manually recreate and test them. In order for it to be useful, there are some best practices. Here's what I do.

1. Cygwin (Surprise!) If you've been reading my blog for any period of time, you knew this was coming. If your analysts use Mac OS X or Linux for their ESM console platform, not to worry. They can play, too. Beyond, Perl, bash, and Python, that's kind of the point.

2. Standardize on a source directory for scripts. Put all of your scripts in the same spot. Pathing is difficult to manage by hand, so by defining a standard (I use /usr/local/bin/arcsight), you have less to do each time you distribute a new tool.

3. Use a repository like Subversion or CVS for scripts and other tool artifacts. That way, you can make a change to your tool, check it in, and the other analysts can check it out quickly and easily. No messy manual copies. Also, when you foul something up, you have revision history to go back to. That can be a life saver if you are - like me - not a developer with good testing habits.

4. Use consoleupdates.txt on the ESM manager to distribute tool configs. Here's how you do that:

Let's say my ESM user id is 'paul' and I have developed a whole bunch of tools following the first three rules above. I can use this Perl script to create an export of the tool configs for use on the server. It looks like this in Cygwin:

$ arc_tool.pl export all /cygdrive/c/arcsight/Console/current/paul.ast > ~/consoleupdates.txt
$ scp consoleupdates.txt arcsight@esmmanager:/opt/arcsight/manager/config
$ ssh arcsight@esmmanager
Password:
arcsight@esmmanager:~ $ chown arcsight.arcsight $ARCSIGHT_HOME/config/consoleupdates.txt
arcsight@esmmanager:~ $ chmod 644 $ARCSIGHT_HOME/config/consoleupdates.txt

If you want your analysts to get the updated tool list, they need to log out of the console, move their AST file somewhere safe, and log back in to the console. No restart of the manager is necessary. Now they just need to create the same script directory you did and check out your scripts from your CVS server.

The Perl script I wrote also supports listing the tool names in an AST file as well as exporting single tool configurations, so it's more than just a one-trick pony. It's got another half a trick.

My advice at this time is not to invest a ton of time in doing this unless it's a weekly headache for your security team, but it is worth doing if you've already got the moving parts in place (like a CVS server). The reason is that ArcSight has already fixed the issue of sharing tool configs in ESM 4.5. So once that's released (later this year?), some of this will be a non-issue for you. I suspect that the first two or three best practices I list above will still be valid in ESM 4.5, so it's still a valuable exercise if you have any number of custom tools already.

ArcSight User Conference 2008

2008-09-09T08:59:00.005-05:00

I'm on the floor of the ArcSight "Protect '08" conference this morning. Tim and I gave our talk on ArcSight ESM Tools yesterday, and I will post some version of those slides and some of the code after I return from the conference.

Right now I'm listening to Hugh Njemanze give his keynote on product lines. There's a lot of interesting stuff in the release pipe; Logger 3.0, ESM 4.5, a new Connector appliance, IdentityView content for ESM, and something called "McLovin."

Anyway, here's what's been good so far:

Customer presentations (other than mine, I mean) - I missed out last year, these are the best talks so far.
Location - the new hotel is within walking distance of stuff (and by stuff I mean not trees and the NSA.)
Networking - Always the best part of this conference. I love standing around with free beer, talking to other folks about what they're doing with their SIM, and sharing ideas. Looking forward to more tonight.

Here's what's been not-so-good:

Wireless - the hotel wireless has been unreliable and overloaded. Frankly, I'm surprised I've been able to stay on long enough to get this post up.
Vendor/sponsor floor - no offense to these guys, but the freebies this year are unimpressive. I've already got a pen, thanks.
No bag - Instead of a "conference bag," everyone was issued a plastic file folio thing. Not that I needed another bag, but I can't smoosh the one foam squeezy thing I did get from a vendor booth into this blue plastic thing.

And I would be remiss if I didn't drop a product scoop or two:

Logger 3.0 has adopted a more-ESM-like boolean filter interface. Big improvement over the chained-regex search in 2.5 and earlier.
Demo of Logger 3.0 shows that searches of data (no details on data set) are roughly 80x faster than a similar sized search on 2.5. (The claim is 100x faster, but I counted. Still, that's a significant improvement.)
Hugh has hinted that the slick, high-performance append-only storage stuff that Logger has is going to be integrated into ESM in some release beyond 4.5. That could mean the end of the Oracle / PartitionArchiver storage model. It won't be missed.

Visual Analysis of 'Ideas in Security'

2008-09-05T15:26:00.006-05:00

Amrit Williams, former Gartner analyst and CTO at BigFix is one of the bloggers that I follow regularly. Amrit's a very smart guy and I respect what he has to say. He recently wrote a pair of blog posts (here and here) that compliment eachother.

Now, in the details of what he has to say, Amrit and I are in agreement. But I got to thinking about the second post and how it relates to the first post. And, well, I fired up Visio and mapped the relationships between Amrit's greatest and worst ideas lists.

If we look at the great ideas that didn't spawn or perpetuate the worst ideas, then we're not left with much. Just segmentation and theory of least privilege. If we drop out planning and segmentation because they're not actually security ideas - just good ideas that work lots of places - we're left with Theory of Least Privilege as the one great idea to come out of security. Oddly, that seems about right.

Evidence FAIL

2008-08-19T20:13:00.009-05:00

So, first read this .

John Dozier, self-described "SuperLawyer" of the Internet, thinks you kids and your DefCon are a bunch of punks. Stay off his lawn.

Of course, I disagree. DefCon used to be a hacker conference by hackers for hackers. Now it's the BlackHat afterparty-slash-olympics. But what it isn't is a bunch of criminals. Sure, there's some mischief, and a few folks even break the rules. But everyone I know who attended DefCon this year (and that number is solidly in the double-digits), works in InfoSec, and uses what they learn at DefCon in their professional lives.

Compelling as my argument may fail to be to people like Mr. Dozier, his argument is weaker than mine. Let's dissect, shall we:

Defcon ... began August 8 and it looks like the hackers sitting in the audience and participating in the hacking competitions spent two days trying to hack into the Dozier Internet Law website using SQL Injection Attacks, Mambo Exploits, encoded cross site scripting attempts, shared ciphers overflow attempts, and the like.

The favorite and most common ISP access was from Vietnam and China, with Beijing the host and doorway of the Olympic Games as well as many, many hackers.

OK, so what we have here is a number of known, old, web attacks from China against his web server that coincide with the timing of DefCon. And aside from the timing, there's nothing to implicate anybody having anything to do with DefCon. My guess is that this wasn't even an actual human being at all, but rather an ASPROX scan that Dozier's IDS detected.

The graph above shows what these hackers do. They come to Vegas to learn how to hack into systems and create havoc.

The funny thing about this is that, with the notable exception of Dan Kaminsky's DNS attacks, there aren't IDS signatures for the research presented at DefCon. So any attacks that did come as a result of learning done at DefCon wouldn't be on that graph.

The frustrated perpetrators (they never got access) were sitting in the Riviera Hotel ballrooms, I suspect...

First, the key word there is suspect. Mr. Dozier has zero evidence that these IDS alerts had anything to do with DefCon. None. Not a shred. Second, they would've gotten in.

Going after law firm websites and administration areas that contain attorney/client protected communications and documentation, and even court ordered "sealed" files, is a direct attack on the integrity of the judicial process and the judiciary

If you have documents that are sealed by a court order stored on your company website, then you have problems. Most federal district courts won't allow you to electronically file with the court to have a document "sealed" if that document must be or otherwise is included in the filing. Those general orders aren't accidents. It's a recognition on the part of the judiciary that electronic documents are inherently less secure. But I digress.

Many attendees commit criminal acts while in attendance in organized war games.

This is simply untrue. There are organized wargames, conducted on an air-gapped network off the Internet or any other network. This is perfectly legal. The US Air Force has staffed a team in the past. By the way, congratulations to Chris Eagle and sk3wl0fr00t on their CTF win. They bested two-time champs 1@stplace, who are some of the smartest people I know, and who are all highly ethical InfoSec professionals.

Others commit criminal acts as they learn the tools of the trade in the very ballroom during speaker presentations. They hack into banks, into personal computers, into businesses, into government agencies, and steal private information, cost businesses billions of dollars annually, and ruin the financial well-being and impair the emotional stability of individuals all across our country.

This is sensational and unsubstantiated. Or as a judge would describe it, hearsay.

This is the mob of the 21st century;

No, John, this is the mob of the 21st century.

The only "security researchers" in attendance, I suspect, are the good guys.

Yes, the security researchers at DefCon are the good guys. And I promise you that the DoD and DoJ agree, as many of the speakers, attendees, volunteers, and contestants at DefCon are paid consultants to these organizations.

UPDATE: John Sawyer has an excellent write-up on this issue and on this year's DefCon (unlike John Dozier, he was actually there) on his blog, Evil Bits, over at Dark Reading. Go read.

On Blended Threats

2008-08-13T05:30:00.003-05:00

Dave Hull over at Trusted Signal has an interesting post on his blog right now about blended threats. (Unfortunately, I can't find a permalink for it, so I don't know how long you'll be able to read it.)

If it's not still there for you to read, let me give you the gist of it. There's been some recent research into and discussion of blended threat scenarios by some very smart people.

So what is a blended threat? It's where two or more lesser-severity vulnerabilities are exploited in conjunction with each other to lead to a greater compromise. An example would be a pen-test I did some years back where we found a SQL injection vulnerability in a low-value web app with no insert/delete grant to an older, unpatched version of Oracle. Individually, you wouldn't rank either vuln especially high. You could break the web app, but there wasn't sensitive data in there, and you couldn't tamper with the data itself. The Oracle database wasn't exposed to the Internet directly. But by using SQL injection to attack Oracle, I broke out into the server OS, reverse tunneled a command shell, and had the Administrator password in very short order. Which was also the Administrator password of the other servers I could talk to.

Myself and others have been predicting the emergence of wide scale blended threat attacks since at least about 2002/2003. And so far we've been wrong, which is good. For now, blended attacks are, as Dave points out, the stuff of professional pen-testers and other intelligent intruders. But frankly, I don't know why.

The problem with blended threats is that they're harder to identify and calculate risk for. CVSS doesn't provide a way for scoring vuln A when also in the presence of vuln B. And this has lead to vendors delaying patches or downplaying the severity of vulnerabilities based on the assumption that any vulnerability the only vulnerability present.

This creates an opening in the patching cycle for malware/botnet folks to capitalize on if the right blended threat comes along. Maybe we haven't seen it becauuse, to date, these folks simply haven't needed to go there in order to be successful.

What Role Will Security Researchers Play a Decade From Now?

2008-08-02T17:09:00.011-05:00

The whole Dan Kaminsky DNS Thing has gotten me thinking about disclosure. I intentionally haven't blogged about it because, well, the speculation around Dan's finding has turned into something of a spectacle. And you didn't need to read yet another blog post about the sky falling.

But on the eve of Black Hat, Dan's talk is less than a week away, and I can't help feeling like we've gotten no closer to understanding the issue of disclosure than we were a year ago. So, all I'm going to say about Dan's recent "situation" is that I, for one, am impressed by the level of care and coordination that went into working with vendors to get patches. This is hard. Researchers hate it because vendors can be uncooperative, incompetent, and downright vindictive. So, thank you, Dan, for spending what must have been countless hours on conference calls and e-mail getting vendors onboard.

Now that that's out of the way, let's talk about research, disclosure, and the future. Dino Dai Zovi noted in a recent blog post that the 90's were the era of full disclosure, and that that is now over. (It's an excellent post. Go read the whole thing.) And this is evident in a number of ways. For one, ZDI and other pay-per-sploit buyers. For another, in-the-wild 0days showing up for sale from malware vendors like the MPack team.

And then there's the ongoing "debate" (read: stalemate) between researchers and vendors about protocol, grace periods, and credit.

So disclosure is a mess. But I don't think it has to stay this way, at least not in the USA. Researchers who publish - as opposed to sell - have the opportunity to become consumer advocates. By cooperating with vendors in a way that still holds them accountable, researchers can demonstrate value to the consumer public. When that becomes the prevalent sentiment, then other interesting things like grants and nonprofits make it possible for researchers to earn a living without having to also do consulting or sell their exploits to a third party.

And that's the dead horse I'm beating in the disclosure race - the consumers of IT products don't have a voice in the disclosure dialogue and desperately need one. Researchers can, if they're able to forego infighting and ego theatre, be that voice.

Coffee Shop Warfare

2008-07-16T20:37:00.006-05:00

It seems like I can't go to a coffee shop, conference center, or bar these days without some jackass on the network abusing the bandwidth. Running MMO games, BitTorrent, gnutella, or even just a large FTP/HTTP download will saturate the wireless access point, let alone the modest DSL line it's connected to, rendering it unusable for the other patrons there. This is just plain rude. And since the barrista can make a mean caramel cappucino, but doesn't have the ability to blacklist your MAC on the AP (which I realize isn't a very effective control, but hey - maybe you'd get the message then?), we're all stuck to suffer.

And I wouldn't do anything hostile on a public network. But in the name of network self-defense, there are a couple of tools you might want to take with you to the coffee shop next time.

Wireshark - The quickest, easiest way to identify the abuser's MAC/IP is with a sniffer like Wireshark, tcpdump, or iptraf.

Snort - Snort with flexresp2 enabled, bound to your wireless interface, and the p2p.rules set enabled and modified with "resp:reset_both,icmp_host" is an effective deterrent for people using P2P file-sharing software.

Ettercap - More severe than Snort, you can use Ettercap to perform ARP poisoning and essentially blackhole the client(s) of your choice by MAC address. You could also use this tool to sniff unencrypted traffic between clients and the AP (and points beyond). But you wouldn't do this. It would be uncivilized, and possibly illegal.

There are lots of other wireless tools out there that have some application here, but many of them either go to far to be civil (Void11) or legal (Hotspotter), so I don't recommend them. For that matter, what I do recommend is getting your own EVDO card. Then you don't have to put up with rude WiFi users in the first place.

A Conversation With My Wife

2008-07-15T21:34:00.005-05:00

My wife was at her mother's tonight when she caught me on GMail chat. This is the log of that chat, unedited:

Jessica: boo!

me: hey there

Jessica: hey baby!
Just looking at my moms task mamanger, she has a ton of stuff running
inlcuiding a bunch of exe file

me: that's all you should see in task manager - exe files

Sent at 10:28 PM on Tuesday

Jessica: how amobile deviceservice.exe, alg.exe, msmsgs.exe, searchprotection.exe, jusched.exe, E-S10IC1.exe
all of these are listed under "Administrator"

me: some of those are fine
type them into google
liutilities.com
searchprotection.exe sounds suspicious
don't log into the bank or anything

Jessica: why would there be 4 svchost.exe's?

me: that's typical

Jessica: or services.exe
winlogon.exe

me: both fine

Jessica: csrss.exe

me: also fine

Jessica: smss

me: seriously
google

Jessica: mDNSR

me: that sounds suspicious

Jessica: I don't need no stinkin google, I have you
:)

me: meh
Sent at 10:33 PM on Tuesday

When is a Security Event Not a Security Event?

2008-07-14T13:19:00.003-05:00

When it's also a beer event, of course!

July's GRSec meetup will be Wednesday, 7/23/08. The reason for the Wednesday date is two-fold. First, Tuesdays don't work for everybody, so we're switching it up over the summer to see if we can get some fresh faces out to GRSec. Second, this month we're at the new Graydon's Derby Station, and that particular evening, they will be tapping a cask of Victory Hop-Devil IPA.

If that's not enough reason for you to be there, then I don't know who you are anymore, man! I don't know you at all...

Details & Map

Monkey-Spider

2008-07-08T21:35:00.004-05:00

It's been awhile since I've covered anything to do with honeypots or honeyclients. But it's also been awhile since anything new came along.

Via Thorsten Holz at honeyblog: Sicherheit'08: "Monkey-Spider: Detecting Malicious Web Sites with Low-Interaction Honeyclients"

Monkey-Spider, not to be confused with SpiderMonkey, is a new honeyclient from Thorsten, Ali Ikinci, and Felix Freiling. Like HoneyC, it's a crawler-based client that detects web-based, client-side attacks. It was presented at Sicherheit in Germany in April. Fortunately, the whitepaper and documentation are in English.

After reading the whitepaper and playing with the code a little, the thing that occurs to me is that, while this is very cool, and still somewhat useful, what I really want for operationalizing a honeyclient in my enterprise is the ability to seed the honeyclient from firewall/proxy logs. That way the honeyclient is analyzing my web traffic, not off looking for random malicious sites to add to already big blacklists.

MiniMetriCon 2.5 Slide Decks

2008-07-07T21:01:00.002-05:00

MiniMetricon 2.5 was a one-day security metrics event held in San Francisco back in April. Some of the slides decks were published to securitymetrics.org earlier today. I'm only about half way through them, but there's some good stuff in there, and if you're doing anything around security metrics, I recommend you check them out.

So far, the standouts for me are Pete Lindstrom's slides on Enterprise Security Metrics, and Wade Baker's deck on Incident Reponse Trends. And speaking of Wade Baker, he and a few of the other rockstars at Verizon Business have a blog that you should add to your feeds list.

I'm Floored: Raffael Marty declares that SIM is dead.

2008-06-27T11:38:00.003-05:00

No really, he said it. He would've been on the short list of people I assume would never say it. But there it is.

Here's the thing; I think that this is a lot like Gartner's IDS declaration (which he cites). IDS went through some product positioning changes (IPS, UTM, DLP, etc.) but the core idea and technology is still there, and guess what? The original IDS use case is still viable. Sure the attacks have changed, but having a sniffer that can search for known-bad and known-strange traffic on the wire is very, very useful.

So I assume that we are in the midst of a product positioning shift around SIM. Raffy's point that SIM schema are IP-centric and rules are based around correlating firewall and IDS events is true. But most of the vendors have already acknowledged this and are developing content to focus on other log sources. Either way, the use case is here to stay - the ability to search and correlate log events is highly useful, and will continue to be. You may call it "SIEM" or "IT Search" or "log management," but it's the same core concept, repurposed to address the constantly changing security environment.

One final note for vendors from the SecOps trenches: I am not open to a replace/resell on the basis that SIM is old and whatever-you-call-it-now is new and better. My SIM, like my IDS, contains custom content that our team has developed to keep on top of changing threats, including application attacks. SIM, like IDS, succeeds when you put talented security professionals in front of it and let them tune it and manage it like a tool. But it will fail miserably if you are hands-off with it.

Useless Statistics: Nate McFeters vs. Verizon

2008-06-23T10:33:00.006-05:00

You know how much I love to tear into vendors whose studies and data analysis wouldn't pass muster for a high school statistics course. It's nice to see someone else go off for a change. Nate McFeters, Ernst & Young security dude, ZDNet blogger, con regular, and fellow Michigan native has taken issue with the results of a study on data breaches that Verizon published earlier this year.

So let's just get this out of the way:

The first thing you’re thinking is, “Wow, my consultant has been lying to me about internal threats!”, the thing is, that’s not necessarily true.

Yes it is. "Insider threat" is a red herring throughout security, but especially where data breaches are concerned. There's no breach notification law out there that defines a breach where the data ends up in the hands of someone that already works for you. Since there's no external force requiring companies to track these incidents, it's probably very safe to assume that tracking and detection of these is low, except within a handful of specific verticals.

To Nate's point about the wording of the survey and the study, I agree - it is dangerously ambiguous. However, it's probably not the cause of the improbable skew toward external attackers in the survey data.

I think I know what is. See, Nate's thinking about the Verizon study like a pen-tester, and forgetting that most data breaches and security compromises don't involve vulns and sploits, just the interesting ones. Sometimes they involve phishing, but most of the time they involve simple impersonation (the FBI calls it 'identity theft').

The thing is, if you follow basic authentication principles and practices around your self-service web apps, this stuff is hard to prevent but easy to detect and resolve. And this is how you get the disparity between the number of breaches and the amount of data breached, in the statistics. Most of those "external attacker" scenarios were someone's kid, deadbeat brother, or ex-wife impersonating them to get at their information. Not good. But not interesting.

Seriously, I don't know what it is, but it's almost always divorced/divorcing couples involved in these impersonation breaches. Nate, if you want to interview me about this some day, I've got some great stories.

Anyway, use Verizon's survey for what it's good for - getting more security funding. Because bottom line, that's a lot of breaches, no matter the circumstances.

ArcSight Logger Face-plate-lift

2008-06-16T08:17:00.005-05:00

Not only did ArcSight deliver on the improved UI and feature set for Logger 2.5, but like any good appliance vendor, they've popped their collar. And by collar I mean front bezel.

The red with blue LED scroller is definitely a better look than the previous model. Still no backlit logo, though. :-)

ArcSight's new Logger Apps

2008-06-13T12:08:00.005-05:00

ArcSight is releasing the Logger 2.5 software here soon, and along with it new appliances with some interesting variations. You can check out the vitals on the ArcSight website here.

Prior versions of Logger were available in small, large, and SuperSized, where the SuperSized box was the same spec as the large box with artificial limitations removed via license key. So really only two boxes, all self-contained, all CentOS, all MySQL.

Now, there's a whole new batch. It would appear by the naming designations that they are going after PCI compliance heavily with the L3K-PCI, which must have retention policies and capabilities that make it easier to comply with PCI-DSS 5.2. Another model supports SAN-attached storage and Oracle, so you can grow your Logger with SAN instead of NAS. And finally, there are two new L7100 models with 6x750MB drives. If I'm doing my math right, that works out, after compression, to about ~~40TB~~ 36TB of log storage. That's a significant increase over the ~~15TB~~ 12TB that the large/SuperSized L5K boxes shipped with.

Update: Talked with Ansh at ArcSight today, and aparently the 2.5 software adds columns to the CEF event view. That's a big deal for folks using CEF events in Logger, and may make CEF-only the preferred format for most Logger users. The new software also includes real-time alert views (like Active Channels in ESM), as well as a number of other enhancements to alerts and search filters and more. Current customers can download 2.5 from the software site.

DefCon 2008 Quals

2008-06-13T08:40:00.002-05:00

The lineup for this year's DefCon CTF competition has been decided.

And, better still, Doc Brown has once again posted the challenges (with answers) for everyone to try on their own. Brain calisthenics, to be sure. Get your mental leg warmers on and jazzercise kenshoto style.

From My Inbox... (ArcSight Connectors & Logger)

2008-06-01T20:14:00.004-05:00

SC left a comment on an earlier post on ArcSight Logger and CEF vs. Raw formats...

Hi Paul,

Do you know if its possible to "insert" logs into the logger or SmartConnector if the logs are on a physical storage, e.g. DVD or external storage?

Thanks.

Kind Regards,
SC

There are probably a number of ways to do this, but I've only tested one. In earlier configurations of our syslog infrastructure, there were a couple single points of failure. In order to meet log analysis commitments, we would reload lost syslog data from file.

Start by configuring a 'Syslog Pipe' Connector. Since the connector only has to be online with the Manager when you're manually inserting logs from file, you have greater flexibility about where this Connector will live. It lived on my laptop for a while. When you set it up, point to a path that isn't already used for anything else. Then you can simply start the Connector and pipe the raw log file(s) to the named pipe:

# cat oldsyslogs.txt >> /var/spool/my_arcsight_pipe

Depending on how far the date/time stamps of the events in those files are from $Now, ArcSight will probably throw some errors. It will maintain the "End Time" from the raw log events, and apply "Manager Receipt Time" as the time the manager collects the parsed events from the Connector. This will absolutely screw up any correlation rules you wanted these events to be subject to. Sorry, no easy way around that.

Game Theory and Mac Malware

2008-05-23T06:01:00.003-05:00

Cloudmark's Adam J. O'Donnell wrote a fascinating article which uses game theory to predict the tipping point for mass malware attacks on Mac OS X.

It's very hard to construct a game that can accurately represent an uncontrolled environment like the one that malware and botnets currently exist in - and to his credit, Adam fully acknowledges this. But Adam's article is great for two reasons.

First, I think that his estimate of Mac's needing to break 17% market share before they become worthwhile to malware authors isn't too far off. And second, the game he's constructed is a great model for risk-based analysis of a partially unknown threat environment (which is a fancy way of saying how likely you are to be pwned in the future).

I so often find myself ranting about infosec articles and papers that fail at basic math, let alone reasonable science, that it's nice to see something of this quality hit the trade press. Thanks, Adam.

TJX vs. CrYpTiC_MauleR

2008-05-22T18:19:00.003-05:00

rsnake reported today that TJX has fired an employee who goes by the handle CrYpTiC_MauleR. He was apparently fired for disparaging remarks he left on a sla.ckers.org message board.

So who is CrYpTiC_MauleR and why should you care? He's some college kid working a retail job at TJ Maxx, and you probably shouldn't. Unless you're TJX, that is. And not for the reasons you might think.

Sure, this kind of thing is bad PR for TJX coming and going. And sure, it's disloyal and immature of an employee to trash his employer to the public, especially when it exposes their security vulnerabilities to self-proclaimed hackers. So you might think that firing this guy is an appropriate response. And maybe it is. But I don't think so.

Now, don't get me wrong, I don't believe for a second that this guy is an actual whistleblower. PCI's not a law, and rsnake isn't a regulatory or law-enforcement agency (that I know of), so what he did doesn't even approach whistleblower status. But his now-public firing is going to have a stifling effect on employees, both retail and corporate. And that is a failure of TJX's security program (one of many if you believe CrYpTiC_MauleR).

The thing is, a company needs to have a method of intaking security concerns from staff, and whatever that looks like needs to be communicated to staff, especially from company leadership, like the loss prevention exec that CrYpTiC_MauleR claims to have spoken to. Firing this kid for airing his concerns to the only people that would listen to him is certainly TJX's perrogative and not at all unexpected, really. But it also points out that the culture that allowed the initial breach to occur in the first place hasn't changed.

I've suggested before that TJX could stand to purge themselves. This only reinforces that opinion. If TJX can't change its overall security culture, it's only a matter of months before they're all over the news again.

List of Malware Analysis Tools

2008-05-17T09:05:00.007-05:00

Update: There is an updated version of this list of tools posted to my blog here.

If you're a company that's big enough to have a security team, then you already know that client-side vulnerabilities are your biggest external attack surface. And the most common form of exploit is a drive-by download attack that drops a bot or other malware on your client. While we wait for the necessary paradigm shift in malware prevention to come along and replace ineffective AV scanners, we're stuck investigating suspicious web sites and binaries to determine their intent and impact. Part of being able to do these investigations is putting together an environment in which to analyze these web sites and binaries safely. Here's what I have done.

The first step is to build a virtual machine with VMware, VirtualPC, or whatever you prefer. It should be as similar to your corporate image as you can make it, but it should not be on your domain. Also, if you select VMware Server, do not install VMware Tools into the VM. Sure it makes things easier, but it can also make it easy for malware to determine that it's in a VM and prevent it from running. I would also recommend installing your company's AV scanner, but disable real-time scanning by default.

Once you've created your VM, you need add some tools to make analysis possible. Here's the list of stuff in my VM.

Cygwin
- Didier Stevens' SpiderMonkey
- pefile
- Jim Clausing's packerid.py
- My ieget.sh
- Mozilla rhino debugger

GMER
catchme

Mandiant Red Curtain

OSAM Autorun Manager
Mike Lin's Startup Control Panel
HiJackThis / StartupList / ADSSpy

HashCalc

HHD Free Hex Editor

OllyDBG (also: Immunity Debugger)
Plugins:
- AnalyzeThis
- FindCrypt
- Hide Debugger
- OllyDump
- OllyFlow
- OllyDbg PE Dumper

ImportREC

iDEFENSE
- MAP
- SysAnalyzer
- HookExplorer
- SniffHit
- PEiD

SysInternals
- AccessEnum
- autoruns
- Filemon
- procexp
- psexec
- psfile
- psgetsid
- Psinfo
- pskill
- pslist
- psloggedon
- psloglist
- pspasswd
- psservice
- psshutdown
- pssuspend
- Regmon
- RootkitRevealer
- tcpvcon
- Tcpview

Firefox (JavaScript Console mod)

Also, having links to VirusTotal and CWSandbox in your VM is a good idea.

Can the Media Move the Ball on HIPAA?

2008-04-30T09:01:00.007-05:00

I finally have a serious prediction for 2008: I predict that unauthorized access of medical records will be the new lost laptop story.

Reporting on the compromise of data through laptop loss/theft over the past few years has raised public awareness around data breaches and disk encryption. The upswing in incidents involving hospital employees accessing celebrity medical records will have a similar affect on awareness. I mention this because a former UCLA Medical Center employee was indicted yesterday on charges stemming from similar activity. What made this a criminal case and not just another firing is that the employee sold these records to a "media outlet" (tabloid).

The reason this is significant is that stories like this in the media raise public awareness about HIPAA requirements and medical provider capabilities. Those capabilities being the ability to review who accessed a patient's medical record and when, and that the hospitals have a way of determining whether or not the access was appropriate. The end result will likely be two-fold. First, more patients will be aware of these capabilities, and will start doing things like asking doctors and hospitals for this information. And secondly, the hospitals that aren't currently reviewing the logs from their EMR systems will feel some pressure to start doing so.

Don't Read My Blog

2008-04-28T19:37:00.002-05:00

At least, don't read it this week. I don't have anything to say that you should read before you read the transcript of Clay Shirky's "Looking for The Mouse" keynote at Web 2.0.

But I do have this to say - feeling inspired by Clay's speech, I've revived 2 old projects and started a new one and am making it my goal to finish them this year. Thanks to Dave Aitel for posting a link to this on Twitter.

Cool Things To Do In West Michigan This Spring

2008-04-16T11:46:00.004-05:00

Sorry in advance if Google sent you here looking for tourist attractions. Blame it on the title's lack of creativity or specificity. But while I have your attention, if you like security and/or beer, and will be in the vicinity of Grand Rapids within the next 6 weeks, this could be your lucky day.

That's because...

This Friday, April 18th, Matt Carpenter of Intelguardians is presenting to the Grand Rapids ISSA. His talk will basically be like drinking from the SANS 504 firehose. The best parts of a 6-day course, condensed into 90 minutes or less. Matt was my instructor for 504, and he's awesome. This will be an excellent talk.

The following Tuesday is GRSec at Graydon's Crossing. Amazing pub food and a great beer selection. I like this place a lot.

Next month, GR-ISSA has Jared DeMott coming to speak. Jared will be giving a presentation that dailydave readers have already seen a preview of. I met Jared after attending his fuzzing talk at Black Hat last year. He's freakin brilliant to start, but also a very eloquent presenter.

And there's a pretty good chance that there will be another GRSec the Tuesday after that!

It's The End of The (Security) World As We Know it... And I Feel Deja Vu?

2008-04-11T05:34:00.005-05:00

If you've been following blogs or online trade press coming out of this week's RSA conference, then you no doubt have heard about the keynote that IBM's Val Rahmani gave in which she declared that, "The security business has no future." Now, that's the punch line she used to get into the trade press and onto the blogs (including mine), but the real gist of the talk was that the future of security is for vendors to bake security into infrastructure products, and that that's what IBM would be doing.

I'm not going to dissect Val's talk, but I do want to point out two interesting things. First is that Val is Tom Noonan's replacement at the security branch of IBM Global Services (formerly ISS). So why the GM of a consulting practice is talking about her offerings' futility in a public way is a little confusing and not good for morale. I'm sure that's not what she intended, but still.

Second, and perhaps more interesting, is that this year's keynote is eerily similar to the Bill Gates keynote from RSA 2006. Now, he didn't open with a shock-jock style punch line the way Rahmani did, but he could have. And he would have had the high ground. But Gates did talk a lot about what Microsoft was doing at the time to build secure, sustainable infrastructure. He also dragged out OneCare (now Forefront) and Vista as examples of Microsoft's advances in platform security. The stories I have read seem to indicate that Rahmani did not mention specific products or tactics that IBM would be sending to market.

So I guess if you're looking for a take away, it is that platform security has gained traction at least as a talking point. And IBM is at least 2 years behind Microsoft in product positioning for security.

ArcSight Logger: CEF vs. Raw

2008-04-04T10:05:00.006-05:00

Here's something for potential ArcSight Logger customers to ponder. The issue is whether you should use CEF formatted logs (post-Connector) or raw logs (pre-Connector) or both in your Logger environment. In this case, a picture is worth at least a few hundred words:

If you look carefully at that image, you can see that it shows the same event in both its raw syslog format and it's Connector-ized CEF format. From my point of view, it boils down to use case. Analysis versus troubleshooting. Reporting versus response. The CEF formatted message is chock-full of metadata-and-labeling goodness. It's also overkill on the eyes. Log messages are already cryptic to the point of questionable usefulness. CEF amplifies that. The raw format, on the other hand, is easier to read due largely to the fact that it's what your UNIX admins are used to seeing. But that's where the positives end. Raw syslog is all but unformatted and trying to write a small chain of regexes that do a good job of parsing large quantities of syslog is a headache and a half.

Of course, you may have already realized that there is a right answer to this problem: Do both. Sure there's some overhead to consider, since you're going to pass syslog to a Connector that will then send raw events to Logger, CEF events to Logger, and CEF events to ESM if you have it. Or you could send raw syslog to Logger, have Logger forward it to a Connector and then configure the Connector to send CEF to Logger and ESM. There are probably many other complicated flows that you could implement as well, but you get the idea.

Binary File Visual Analysis Redux

2008-04-01T20:58:00.007-05:00

I got a great comment on my post regarding simple binary file visual analysis from Erik Heidt. Erik made the very valid point that visual analysis of ciphertext is not a highly reliable way to distinguish "good" crypto from "bad." He used the example of an 8-bit XOR of a file as an ineffective method of encrypting data that also has random byte distribution.

Since there's nothing good on TV, I decided to see what an XOR-ed file data looks like in gnuplot. So here's what I did.

Like before, I used the Netcat nc.exe binary. I then encrypted it using GPG and also encoded it using Luigi Auriemma's Xor utility. I then ran the three files through the Perl script from my previous post and then plotted them with gnuplot.

Here's the plot of the original binary:

Here's the plot of the GPG-encrypted file:

And here's the plot of the XOR-encoded file:

As you can see, the XOR plot has peaks and valleys that are characteristically similar to the original binary. I don't want you to take away from this that this visual analysis method is highly reliable in all situations. I only wanted to share that basic XOR encoding does stand out visually.

On Firewall Obsolescence

2008-03-30T17:31:00.012-05:00

This has been a popular topic the past week. But I think that in both Richard's and William's respective positions, it's a game of semantics that's being played here. The real debate on whether or not firewalls are still relevant is actually taking place in marketing meetings, not product development meetings, though the two are inexorably linked. I'll tell you why.

Firewalls are not an actual control, or even a technology. They are a concept. Without reposting the chronological history of the firewall here, I'll simply point out that firewalls have been made up of varying combinations of packet filters, state tables, address translators, and proxies. Vendor-driven convergence has further complicated defining what a firewall is by adding VPN tunneling, content filtering, anti-virus, and IDS/IPS functionality to their products and continuing to call them "firewalls."

So let's do away with the repackaged-converged-for-post-2002-sales firewall and break it down by function. Here's what's not obsolete:

Packet filter with a state table - Easy and fast to deploy for outbound Internet traffic. Most modern OS's come with one built in. If you're doing outbound traffic without it and instead proxying everything you are either:

Using SOCKSv4 on AIX 3.3 and your uptime dates back to 1995 ...or...
Only allowing SMTP and DNS outbound, which makes you my hero

Proxies - Content filtering or inline anti-virus with your "UTM" appliance? Transparent proxy. Web application firewall? Reverse proxy. PIX? They call yours fixups. Check Point? They call yours "Security Servers." FYI - you have proxies.
NAT - Unless you're The University of Michigan and have a spreadsheet to keep track of the public Class A's that you own, you need NAT. For that matter, University of Michigan has NAT, too. But they don't have to. You do.

Here's what is obsolete:

tcp_wrappers - Replaced by stateful firewall in the kernel.
Straight up packet filtering - Memory is cheap. Your routers all have state tables now.

What Richard and William are really complaining about is how most shipping firewalls don't already handle security for complicated app-protocol-over-HTTP stuff like SOAP, XMLP, and even the easier stuff that's getting everybody in trouble these days like SQL injection, XSS, and SEO poisoning. It's this valid, if semantically challenged, line of dialog that is fostering a perceived demand for new capabilities in a firewall and also sending vendors scrambling to position their products in a way that differentiates them from what's out there right now. It seems that every time that this happens, somebody has to declare the old stuff dead. This has happened to firewalls before as well as anti-virus and IDS.

The thing is, this isn't new functionality, it's new content. Back in 2001 I was writing patterns for Raptor's HTTPD proxy to discard inbound CodeRed exploit attempts against web servers. And since that time, I've been writing custom Snort rules to detect or block specific threats including attacks against web servers. Whether by proxy or by packet payload inspection, we've already seen the underlying concept that will make these new security features possible. The hard part isn't getting the data to the rule set. The hard part is going to be writing rules to prevent injection attacks via XML. This is where security vendors discover that extensible protocols are a real bitch.

Useless Statistics Returns!

2008-03-26T18:50:00.004-05:00

Breaking News: Information Security is still not a science and vendors still suck at statistics.

What? You already knew that? Well, somebody forgot to tell WhiteHat. You'd think they might learn from their competitor's mistakes.

I'll save you 4 of the 5 minutes necessary to read the whole thing by summarizing WhiteHat's press-release-posing-as-a-study for you. They collected vulnerability statistics from an automated scanning tool that they sell (and give away demo use of during their sales cycle). From that, they generated some numbers about what percent of sites had findings, what types of findings were the most common, and what verticals some of the sites' owners are in. Then they let their marketing folks make silly claims based on wild speculation based on inherently flawed data. Anyway, I guess this isn't the first one that WhiteHat has put out there. They've been doing it quarterly for a year. But this is the first time I had a sales guy forward one to me. Can't wait for that follow-up call.

So what's wrong with WhiteHat's "study?" First, they collected data using an automated tool. Anybody that does pen-testing knows that automated tools will generate false positives. And based on my experience - which does not include WhiteHat's product, but does include most of the big name products in the web app scanner space - tests for things like XSS, CSRF, and blind SQL injection are, by their nature, prone to a high rate of false positives. No coincidence, XSS and CSRF top their list of vulnerabilities found by their study.

Second, their data is perhaps even more skewed by the fact that they let customers demo their product during their sales cycle. And if you want to demonstrate value doing pre-sales, you will want to show the customer how the product works when you know there will be results. Enter WebGoat, Hacme Bank, and the like. These are an SE's best friends when doing customer demos because there's nothing worse than scanning the client's web app only to come up with no results. It doesn't show off the product's full capabilities, and it pretty much guarantees that the customer won't buy. Of course, what these do to the "study" is to artificially drive the number of findings up. Way up.

Finally, and perhaps best of all, when Acunetix did this exact same thing last year, it turned into a giant, embarrassing mess. Mostly for Joel Snyder at NetworkWorld. The real killer for me is that I know that Jeremiah Grossman, WhiteHat's CTO and a smart guy, was around for that whole thing.

Oh, well. Maybe we'll luck up and Joel Snyder will give us a repeat performance as well.

But just like last time, the real loser is the infosec practitioner. This kind of "research" muddies the waters. It lacks any rigor or even basic data sampling and normalization methodologies. Hell, they don't even bother to acknowledge the potential skew inherent in their data set. It's not that WhiteHat's number is way off. In fact, I'd say it's probably pretty reasonable. But if they - or if infosec as a professional practice - want to be taken seriously, then they (and we) need to do something more than run a report from their tool for customer=* and hand it to marketing to pass around to trade press.

Quicky Binary File Visual Analysis

2008-03-24T13:34:00.010-05:00

I've been reading Greg Conti's book, Security Data Visualization. If I'm honest, I was looking for new ideas for framing and presenting data to folks outside of security. But that's not really what this book is about. It's a good introduction to visualization as an analysis tool, but there's very little polish and presentation to the graphs in Greg's book.

It's what I wasn't looking for in this book, however, that wound up catching my eye. In Chapter 2, "The Beauty of Binary File Visualization," there's a comparison (on p31) that struck me. It's a set of images that are graphical representations of Word document files protected via various password methods. It was clear by looking at the graphs which methods were thorough and effective and which ones weren't. And it struck me that this is an accessible means of evaluating crypto for someone like me who sucks at math. And, hey, I just happen to have some crypto to evaluate. More on that some other time, but here's what I did. It's exceedingly simple.

I wanted to take a binary file of no known format and calculate how many times a given byte value occurs within that file, in no particular order. I wrote a Perl script to do this:

#!/usr/bin/perl
use strict;
my $buffer = "";
my $file = $ARGV[0] or die("Usage: bytecnt.pl [filename] \n");
open(FILE, $file) or die("Could not open file : $file\n");
my $filesz = -s $file;
binmode(FILE);
read(FILE, $buffer, $filesz, 0);
close(FILE);
my @bytes = ();
foreach (split(//, $buffer)) {
$bytes[ord($_)]++;
}
for my $i (0 .. 255) {
print "$i $bytes[$i]\n";
}

This script will output two columns worth of data, the first being the byte value (0-255), and the second being the number of times that byte value occurred in the file. The idea is to redirect this output to a text file and then use it to generate some graphs in gnuplot.

For my example, I analyzed a copy of nc.exe and also a symmetric-key-encrypted copy of that same file. I generated two files using the above Perl script, one called "bin.dat" and another called "crypt.dat". Then I fired up Cygwin/X and gnuplot and created some graphs using the following settings:

# the X axis range is 0-255 because those are all of
# the possible byte values
set xrange [255:0] noreverse nowriteback
set xlabel "byte value"
# the max Y axis value 1784 comes from the bin.dat file
# `cut -d\ -f2 bin.dat |sort -n |uniq |tail -1`
set yrange [1784:0] noreverse nowriteback
set ylabel "count"

I then ran:

plot "bin.dat" using 1:2 with impulses

..which generated this:

I then repeated the process with the other file:

plot "crypt.dat" using 1:2 with impulses

...which generated this:

As you can see, there's a clear difference between the encrypted file and the unencrypted file when it comes to byte count and uniqueness. Using the xrange/yrange directives in gnuplot helps emphasize this visually as well. The expectation would be that weak, or "snake-oil" crypto schemes would look more like the unencrypted binary and less like the PGP-encrypted file.

Prior Art

2008-03-20T09:18:00.002-05:00

OK, I don't have anything resembling a patent claim here, but a year ago I described a need and a potential solution for targeted phishing attacks against Credit Unions. Today, Brandimensions announced StrikePhish, a service offering in that very space.

So instead of the millions of dollars I deserve, I'll settle for StrikePhish buying me a beer at the next con I see them at. ;-)

B(uste)D+

2008-03-19T14:35:00.002-05:00

Apparently, SlySoft's new release of AnyDVD has the ability to strip BD+, Blu-Ray's DRM scheme that has gotten some very credible acclaim.

It turns out that one of the folks behind BD+, Nate Lawson, is giving a talk on DRM at RSA next month. I'm interested to see what Nate has to say about BD+ being broken. That's why I asked him. :-)

Cool Firefox / JavaScript Trick

2008-03-19T12:40:00.002-05:00

Here's an easy trick for deobfuscating JavaScript within Firefox. Via BornGeek and Offensive Computing.

Launch Firefox and browse to 'about:config'
Create a new boolean config preference named browser.dom.window.dump.enabled and set it to 'true'
Close Firefox. Now run "firefox.exe -console". A console window will open along with the browser window.
Edit file containing obfuscated JavaScript and replace "document.write" with "dump"
Open the file in Firefox.

Disable NoScript, Firebug, or other scripting add-ons and reload the file if necessary.

Switch to the JavaScript Console window. Now you can read the deobfuscated code.

You can do this with Rhino or SpiderMonkey, so it's nothing new. But the setup is really simple and easy to use, so if you've been avoiding the other tools available because they're hard to use, this may be what you've been waiting for.

My Not-So-Secret Glee

2008-03-16T16:41:00.006-05:00

When I heard that the only remaining semi-above-board sploit broker is calling it quits, I couldn't help but smile. We still have 3Com and iDefense buying exploits outright. For now. But to see that the "0Day eBay" model is failing for reasons beyond a sudden lack of staff, well that is good news.

I wish Adriel, Simon, and the rest of the folks at Netragard / SNOSoft no ill will whatsoever. I hope their business continues to prosper and that they continue to be positive, active members of the infosec community. That said, I've mentioned my stance on the buying and selling of software vulnerabilities before. There are very real ethical issues here. And, more importantly, there are very real security implications for corporations and end users, who seem to have no representation in the discussion about those ethics.

Abstract

2008-03-13T14:33:00.002-05:00

I've asked to be considered for presenting at this year's ArcSight User Conference. Today I sent my abstract over and will hopefully be on the agenda this year to talk about ArcSight Tools.

The incident handlers at [my company] use ArcSight Tools in their investigations as a way to quickly and easily collect additional intelligence from existing data stores in their environment. Come see how, with very little custom code, they have harnessed existing applications and services to quickly gather in-depth information about servers, users, workstations, and external hosts during an investigation. In addition to seeing how [my company] has leveraged ArcSight Tools, learn some of the simple tricks that will help you go back to your office and do the same.

I've blogged about Tools before and this presentation aims to be an expansion of that concept. The truth is, there's lots of great data in your environment that isn't in a log flow somewhere. And while it maybe doesn't belong in your SIM, you want it at your fingertips when investigating a potential incident. It's good to have answers to questions like, "What does this server do?," or "Is this user a local admin?," or "What is this person's boss' phone number?" close at hand.

Anyway, I hope to have more to say about ArcSight Tools soon.

Anonymous writes... (ArcSight Resources)

2008-02-24T18:07:00.004-05:00

Anonymous writes,

Hi Paul,

I would like to ask if you know of any resources I can reference for ArcSight correlation rules authoring.

In particular, I am looking for Web App and VOIP Security. Thanks in advance.

So first of all, there's an unfortunate shortage of sources on building content for ArcSight. It's part of why I blog about it, because there are only a few people putting information out there. And if SIM's in general are going to mature, then best practices and an open community are part of that maturation. Besides blogs like mine, the ArcSight forums are a good place to get questions answered and share content. Beyond that, I would highly recommend the annual User Conference that ArcSight puts on. For those that can't attend the User Conference, the slides are published to your software site, and definitely worth downloading. And of course ArcSight's own training offerings. But that is pretty much the extent of resources available at the moment.

As far as ways to monitor Web Apps and VoIP Security with ArcSight, it's going to boil down to the log sources you have available. Here are a couple of ideas I have off the top of my head.

For Web App there are tons of optiions. ArcSight works with several web security proxies, IIS and Apache, most IDS/IPS products under the sun, web app servers like Weblogic and WebSphere, and the more popular commercial databases like Oracle, MS-SQL, and DB2. Depending on what's in your web environment and which sources you're drawing from, you have lots of options here. An easy idea might be to create a filter to sift through web server logs for special characters (like < > ' or - ) or requests where the web server returned a 500 or some other obscure error (not 403 or 404).

VoIP is a trickier one to go after since there's no ArcSight connector for CallManager or whatever SIP gateway you use. You could write one with the Flex Connector SDK, but I'm not sure how great your SIP gateway logs are to begin with it comes to security. I think switches, IDS/IPS, and firewall are your best bets here. You'd want to filter firewall logs for packets sourced from your VoIP VLAN address space that might indicate a rogue device connected to your voice network. (Which reminds me, a new version of voiphopper just came out.) You might also want to filter IDS logs for traffic sourced from your VoIP VLANs as well. Hopefully you've already got "switchport port-security maximum 2" set on all of your VoIP ports (and all of your userland switch ports in general) to prevent ARP spoofing/poisoning attacks. In which case, if you send your switch syslogs to ArcSight, a rule to alert on 'NOMAC' messages could be very useful. These can be regular errors, but also occur when someone attempts ARP-based MITM attacks in a port where port-security has been configured.

Anyway, I hope that helps, Anonymous. Good luck with your projects.

Frank Boldewin Releases "More Advanced Unpacking - Part II"

2008-02-18T07:45:00.004-05:00

Just last month I posted about Frank's first release in this series, along with some other recommended reading. Here's Part II via Offensive Computing. Awesome stuff. Frank is essentially giving away what he could charge thousands for as a BlackHat Training course or something similar. It's exceptionally generous of him to do so. I've been in awe of Frank's malware unpacking skills for a while now, and I hope he continues to release more tutorials.

Ranting About Insider Threat

2008-02-15T15:36:00.006-05:00

This is another one of those posts that started out on a listserv. The original thread was about using contracting staff for security work. Along the way, someone mentioned insider threat and I went off. Enjoy.

> While he or she does not necessarily have access to
> many/all functional areas (hopefully), he/she would
> have an easier time compromising the organization's
> security. Being that internal threats are much more
> prevalent than external ones, I'd argue that this
> poses a greater risk when compared to equally-
> screened contract employees under the same NDAs, etc.

I'm not sure that I agree with your statement that internal threats are much more prevalent, or even more prevalent. Looking at the CSI/FBI Survey results over time, the Internet surpassed internal networks as the origin of attack in 2001 and has been widening ever since. I don't think the data bears this conclusion out.

If I put my tinfoil hat on here for a minute, I do think that the way that insider abuse is hyped and promoted in infosec trade press is intentionally vendor-driven. Vendors whose products address border security at layers 3-4 found their sales slumping a few years ago when everybody finally got a firewall, everybody that was going to get NIDS got NIDS, and everybody that had to comply with GLBA got DLP.

So they started telling ghost-story-anecdotes about insiders and how we need to watch our staff like they're an elite team of Chinese hackers. Of course, they never suggested how they would solve the real insider issue. Insiders getting unauthorized access to data isn't the problem. It's what they do with the data that they *are* authorized to access that's the problem. When somebody comes up with IDS signatures for bad intentions, please let me know. I'll be the first one with my checkbook out.

I guess my bottom line on insider threat is not that it's wholly imaginary, but that it's not the same as external threats. Cool appliance-based technologies don't serve you as well inside. Doing things like monitoring use of administrative accounts, auditing financial application access for proper separation of duties, and proper pre-hire screening of staff go a whole lot further than watching what files people copy to their thumb drives and guessing what they might do with them.

14.4Kbps Nostalgia

2008-02-15T11:08:00.014-05:00

I've got some decent display real-estate on my desk, roughly 576 square inches total, so choosing background wallpaper is not a task Itypically proceed into lightly. Well, last week, my selection of background aesthetic was undertaken a bit hastily. There was fallout. In what can only be described as a Bostonian fashion, Aqua Teen Hungerforce was once again poorly received. This time by my co-workers.

So today I went to DeviantArt looking for some new wallpaper, and stumbled upon these:

They're ANSI art, all done by Thor (iCE), for BBS's I used to call back when I was in high school. Legion HQ was the place, too. No real names. Just teaching each other random bits of knowledge and (mostly) cursing each other out like the angsty, angry misfits that we were. I didn't know it then, but I made some good friends during this time. I still keep in touch with a few, 14 years later. But what ever happened to Evil Dude?

Anyway, it's cool to see these again. They remind me of a much simpler time... of Desqview and Telix, of Pascal and WordPerfect, of Kings Quest and fractint, of 2600 meetings and boot sector viruses. It was all new back then. I was all new back then.

Sometimes in life, every few years or so, you stop and look back at who you used to be. The long blue hair has been replaced with short hair that's starting to speckle grey. I still listen to the same music, but it's not cool anymore. But I think that, aside from having never gotten a tattoo, the 16yr old I used to be would've been pretty jazzed to see how his life turned out.

It was 20 years ago today... (minus 19)

2008-02-12T21:27:00.001-05:00

This time last year, I decided to act on my idea to steal Matasano's idea for an informal meetup for infosec practitioners, proselytizers, patrons, and partygoers. And while I can take credit for something that boils down to, "Hey, who likes beer and security? Then let's go to the bar!", I can't take credit for all of the awesome conversation and fascinating stories from some of the best and brightest infosec minds in the midwest or anywhere for that matter.

But that's what GRSec has become. I've been to most of the 12 meetups, and I've had a good time with and learned from others at each one. And so I want to extend a genuine thank you to everyone who's come out to a GRSec.

If you're in the Mid- or West Michigan area and want to network and talk shop with other infosec types, I encourage you to join us for GRSec. Or if you're in another geoloc, check out CitySec.org and find a meetup in your area.

ARST

2008-02-11T19:50:00.000-05:00

Associated Press article via CNN Money. It's pretty favorable about this week's ArcSight IPO. Frankly, I don't care that much about their going public. I only like to blog about it because it's a thumb in Mike Rothman's eye. :-)

ArcSight 4.0 Part 2: Awesomeness

2008-02-11T08:00:00.000-05:00

All you need to do to know that ArcSight has made some big improvements to it's v4.0 ESM product is read a press release. Or launch the 4.0 console even:

And while trending, improved reporting, identity correlation, and portable content are all great feature adds, you can read about them anywhere and everywhere. So I'm not going to talk about them. What I am going to talk about are the little changes to the 4.0 console that I have found to be very useful. The stuff that you might not find if someone didn't point it out.

Let's start with filters. This one you'll probably find, but I just want to say, "Hallelujah!" Seriously, this makes me want to do backflips down the cube aisles like John Belushi in The Blues Brothers. They fixed the filter editor!

If you ever created a big, complicated filter in 3.x only to open it later to find a hideous tangled nest where your neatly organized filter had been before, then this is a victory for you. Take a look:

The other obvious improvement in the filters is automatic escaping of characters. Here's a screen shot of that:

This is an interesting feature add for a number of reasons. The big advantage to you is that when you send results to external formats (think HTML or PDF reports or CSV exports), you don't have to do escaping yourself. I suspect there are a number of advantages to the ArcSight Manager and Web components as well. I don't know, so I'll leave it to you to speculate.

One of the new features that I really like is in the right-click menu of the Events tab of the Case tool.

"Retrieve correlated events" will do exactly that. If, like me, you have cases that open with correlated events only, but you want to record or examine the events that triggered the case, this saves several minutes. Very handy.

The final feature that I wanted to talk about, and my personal favorite, has to do with an enhancement to event annotation. We use event annotation as a way to create an audit trail for log review. The selected events load in an active channel and then our handler-on-duty reviews the events, opens cases where necessary, and changes the events' annotation stage to "Closed" as a way to indicate systematically that they have been reviewed. This adds up to a good 10-15 hours of work that week at least, so anything we can do to speed the process along is greatly appreciated. Starting this month, we will be migrating to using the isReviewed event annotation flag.

The main reason for doing this, though, has more to do with a menu item and shortcut that's been added to the Active Channel / Grid view.

Now instead of 5 mouse clicks, Ctrl-R marks the events as reviewed. It's a nice streamline if you use the feature.

There are lots of other enhancements to the 4.0 console, many of which I probably haven't found yet. But these were the ones that jumped out at me and will have a positive impact on the way I use the tool.

ArcSight 4.0 Part 1: Oddities

2008-02-10T11:28:00.000-05:00

This past week, we upgraded our production ArcSight environment from 3.5 SP2 to 4.0 SP1. We've been running ArcSight 4.0 in our test environment since August 2007, but as with all things "test" there are always a few things that you won't discover in a test environment.

This is the first part of a 2-part series on the upgrade experience. I want to describe some technical challenges that we experienced that you won't find in the upgrade documentation in hopes that someone else finds this helpful.

Before I start talking about what went wrong, I should say that the process was seemingly painless aside from what I describe here. I say seemingly, because I didn't actually do it. But Tim, who did the 2-part upgrade and redesign over the course of three weeks, was still smiling on Thursday following the upgrade. He seems to have emerged from the gauntlet unscathed. I think this is because Tim's a bad-ass and also because ArcSight dramatically improved the upgrade process in 4.0.

So the first problem we ran into has to do with some changes to the Jetty code in the ArcSight manager between 3.5 and 4.0. Here's the error we got:

[2008-02-06 13:50:55,727][INFO ] [default.com.arcsight.server.Jetty311ServletContainer] [initialize] Key Store: [JKS] /opt/arcsight/manager/config/jetty/keystore com.arcsight.common.InitializationException: Exception initializing 'com.arcsight.server.Jetty311ServletContainer': The keystore may not contain more than 1 entry. Please remove excessive entries. at com.arcsight.server.Jetty311ServletContainer.initialize (Jetty311ServletContainer.java:288)

The keystore file was the same one we'd been using since 3.0. We generated our own CA key pair and certificate from OpenSSL and signed the certificate for the keystore file with it. We then added the CA certificate to the cacerts file that is used by all of the other components. While going through this process, I added the CA cert to the keystore file for posterity.

The CA cert being present in the keystore file is what caused the error, and editing the keystore with 'keytoolgui' to remove the CA cert was all it took to get it back up and running.

The other issues that we ran into occured post-upgrade and had to do with upgraded content. The first issue we saw isn't really an issue at all, rather ArcSight's improved some of its logic around Active Channels. Specifically, Active Channels that were created using one time stamp and configured to sort on another time stamp will give an error now. For example, if you created a channel and set EndTime for "Use as time stamp," and then on the Sort tab, set a sort for ManagerRecieptTime, this would cause an error in 4.0. In 3.5 it would merely hurt the channels performance, but would eventually load. This is a good change, but it may mean having to edit some of your old (and presumably really slow) Active Channels before they work again.

The second issue we saw was around filters. ArcSight has made some major improvements to Filter objects, and I'll talk more about that in the upcoming post. However, one drawback seems to be a bug in the parsing/escaping enhancements in 4.0. Filters that use a string match that contains angle brackets ( [ or ] ) will return null sets. There is no error, the only symptom is null results. In our case, all of the brackets appeared in filters matching on the Name field (gotta love undocumented syslog formats). The solution is to revise your filters to not use the brackets.

Globalization and Online Crime

2008-02-07T15:47:00.000-05:00

This article is an interesting read. Not so much for the story itself, but for what it means. The notion that a small agricultural town in Romania can, in three short years, overhaul its local economy via online fraud is fascinating to me. Not the first report of eBay having problems with Romanian criminals, but the idea that the region's economy is now heavily based on fraud is impressive.

But here's what's really blowing my mind: A poor local economy, unbalanced international economies (seriously, as bad as it may seem in the US, at least you're not trying to get by making your own wine and selling it to tourists on the train to Budapest), an open international marketplace (eBay) to connect the two economies, and $300K in technology training grants; this is all it takes to create a global crime syndicate whose victims exist 100% in a different country. It's the opposite of the well-organized, well-funded RBN activity we've seen so much of the past few years. Street hustling for the Internet age.

Fascinating. There's a sociology PhD thesis in there somewhere.

AB1298

2008-02-04T15:55:00.000-05:00

A colleague of mine sent me this article, which should be of interest to pretty much everyone in the health care or human resources fields. AB1298 is an assembly bill that updates SB1386, California's much-copied breach disclosure law. The bottom line is that now an individual's health insurance ID number (which is hopefully not also their SSN) is considered PII much the same way a credit card number is. And when that data along with the corresponding name is breached, you must notify the victim.

It makes perfect sense. That number, combined with proper billing information, is enough to receive health care services from any participating medical provider. And, while I have pretty decent credit, I don't have a platinum card with a six-figure limit. But, if it were medically necessary, my insurer could be charged that kind of bill. And I would be responsible for the deductible. And, unlike my credit card's maximum personal loss, my deductible is not $50. So as an individual I stand to suffer greater financial loss if my medical identity is stolen versus my credit card.

In an America where health coverage is a problem for 47M people and the rising cost of health care is a problem for the rest, it doesn't seem at all far-fetched that trading in stolen health insurance information could become a lucrative criminal enterprise. And that would make health care data a real target.

ArcSight / CEF Patch for Snort / Barnyard

2008-02-04T10:16:00.000-05:00

Last week Colin Grady released a patch to the Snort output tool, Barnyard 0.20, that allows you to output in ArcSight CEF format. I was going to post something here about it last week because of its sheer coolness, but then decided to hold off until I had a chance to play with it myself.

It built flawlessly, and it was easy enough to set up. It creates a new module in Barnyard named "alert_cef" that shovels CEF format messages to a syslog server (like ArcSight Logger). An example barnyard.conf might look something like this:

config daemon
config localtime
config hostname: arnold:eth1
config interface: eth1
config sid-msg-map: /opt/snort/rules/sid-msg.map
config gen-msg-map: /opt/snort/rules/gen-msg.map
config class-file: /opt/snort/rules/classification.config
output alert_cef arnold arcsight_logger.mydomain.local 16 1 514

Those arguments to the 'output alert_cef' line are, in order:
hostname
syslog server
syslog facility (integer format 16=local0)
syslog severity (integer format 1=alert)
syslog server port (UDP)

So here are my thoughts on Colin's patch. On the plus side, Barnyard is fast and lightweight, especially in comparison to the ArcSight Connector bundle which is several hundred MB on disk and in memory because it's Java. On the neutral side, its use case is pretty specific - you have ArcSight Logger collecting syslog data and forwarding to ESM. (Or you do your own thing with CEF and not ArcSight.) And on the down side, bypassing the ArcSight Connector means that you lose the categorization/prioritization stuff that ArcSight does for you with its AUP updates. And it also means no packet payload. For that you need to be running the ArcSight SnortDB Connector and logging Snort to a SQL database (preferably via Barnyard).

So that's still the architecture that I recommend for an enterprise Snort-to-ArcSight deployment. Snort doing unified logging, Barnyard shoveling logs into a SQL server on another host, ArcSight SnortDB Connector (also not on the Snort sensor host) querying that data from SQL and handing it directly to ESM. That gets you maximum scalability and functionality.

Nonetheless, Colin's patch is pretty cool, and definitely on the table for folks that have Logger and ESM. Additionally, if you use Snort and Barnyard, you should look at some of the other patches Colin has on his site. In particular, I think we will be testing and deploying his schema patch for Barnyard. Wish I had known about it 2 years ago.

30-second Malware Gathering Tool

2008-01-30T13:41:00.000-05:00

A few months ago I was trying to automate the retrieval and analysis of JavaScript exploits from a site that was designed to target vulnerable browsers. It was reading User-Agent header strings on the server side and only serving exploits to vulnerable versions of IE and displaying ads to everybody else. So my attempts to script the get-and-grep analysis I was doing weren't working with curl or wget. So I wrote this:

#!/bin/sh -f
if [ $1x = x ]; then
echo "Usage: $0 [url]"
exit
fi
/usr/bin/wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)" $1

All it does is pass url to wget while wget uses an IE6 User-Agent string when it makes its request. Nothing fancy, but it was worth the 30 seconds it took me to whip it up.

How To Develop Firewall Policy For An Existing Network

2008-01-30T13:03:00.000-05:00

So, hopefully in 2008, you're not finding yourself with an Internet connection and no firewall. But it seems to have happened to this guy. Though, I have worked on 2 projects in my past life where this was exactly the case, and we had to put in a firewall and develop policy around existing traffic, and do so without documentation or clear guidance from the company. And in fairness, I don't know that there's no firewall. It may be a situation where there's a firewall that needs replacing that nobody has access to. (That's happened to me before as well, but I digress.)

So here's my response to Ruggero. It was long, so I thought I'd also post it here:

Having done pretty much exactly this twice before, I can tell you that I wouldn't use a sniffer, and I especially wouldn't use an automated rule generator of any sort. You need to make sure any traffic you allow is deliberate and important to the organization. An automated tool won't make such judgments, and you will end up allowing all of the crap that's already on your network that you should probably be blocking.

My advice on how to proceed:

Step 1. Put the firewall in place with a policy that allows all traffic to pass. Turn on logging. Use this instead of a sniffer as analyzing firewall logs will be much easier. Putting the firewall in first will also separate the physical and routing changes you make to the network from the policy changes. This will aid in troubleshooting connectivity issues down the road.

Step 2. Analyze logs. I wrote a shell script to do this with PIX logs, and basically all I did was identify each of the unique sets of srcaddr,dstaddr,dstport and then count the number of times each unique set occurred in the log file. Sort results by number of occurrences and you will find either a) common traffic or b) crappy protocols at the top of the list.

Step 3. Investigate the results of your log analysis. Find out what each set is and then compare it to your business requirements and security policy. Determine whether or not the traffic should be allowed.

Step 4. Write rules for the traffic you decide to allow in Step 3. To help with your process, you may also want to write rules for traffic you wish to block.

Step 5. This is really a GOTO for step 2. You're going to eliminate known traffic from your logs and then analyze again. It's especially cool if the firewall includes rule numbers in its logs, so when you made those rules in step 4, it's now easier to separate known and unknown traffic. Loop through steps 2-5 for as many iterations as appropriate.

Step 6. Change from 'permit all' to 'deny all.' Once you've sufficiently analyzed data and implemented rules, cut over your default policy and test your business-related apps. If you made block rules in step 4 that are superseded by this policy change, you may want to remove them now in order to keep your firewall rule set as simple as it can be.

Step 7. Wait for the phone to ring. Depending on the size of the network and the volume of traffic, it will take you anywhere from a week to a month to get through step 6. Step 7 should last another 30-90 days, and basically you want to let the organization go through a long enough business cycle that things like payroll and billing can run through to completion at least once. You probably trampled on one or two things that are seldom used so they didn't generate traffic before, and this is when you find them and write rules for them.

Good luck. This kind of work is typically a real bear.

PaulM

There, now I don't ever have to lay that out again. :)

Quick Links

2008-01-25T14:52:00.000-05:00

Since I haven't had any time to write any new blog posts, let me point you to a couple of recent must-reads from others' blogs.

Frank Boldewin @ Offensive Computing has a new advanced unpacking tutorial. The best part is that it's "Part I" implying that there's more to come. Required reading for sure.
Ronaldo at SecuriTeam has put together a Google Calendar of 2008 Security Events.
Wouldn't it be ironic if WebSense started blocking Blogspot or GooglePages? There's good reason to. (But then you couldn't read my blog, and that's no good...)
And finally, this post to EthicalHacker.net about a new infosec cert that sounds like it might actually be hard to get.

An Interesting Budget Item

2008-01-20T12:47:00.001-05:00

The company I work for has some neat incentive programs that it makes available to its staff. One of them is the result of the company being in the health care vertical and also having a parking shortage at our headquarters. To ease the parking problem, we offer a "healthy parking" incentive where people receive a small cash payment for parking in the most inconvenient, far away lot. It works so well that there's now a waiting list to get a lousy parking spot. If small cash incentives can work this well for parking, why not other things?

I wish I had this idea last year when we were putting together the 08 budget for infosec: "healthy computing!" In this case, users who have local admin on their company-issued computers would willingly give up their elevated privileges for a cash payment. Sounds expensive, right? But what are you spending on anti-virus and other host-security products? It's probably pretty close, and at least today, there's more value in reducing local admin access than there is in running anti-virus. Not to mention the time and internal cost of proving that a user doesn't need local admin privileges in order to revoke them.

And while I'm denigrating the value of AV products, I'd like to share with you my favorite blog post of the month. It's from Amrit Williams' blog, which is one my regular reads.

Bad News for Mac Users

2008-01-07T13:10:00.000-05:00

If there is still any doubt at all about the security of Mac OS X, I think we are about to find out. Someone at the Army has resurrected an old idea. They're going to start using more Macs because...

"...fewer attacks have been designed to infiltrate Mac computers, and adding more Macs to the military's computer mix makes it tougher to destabilize a group of military computers with a single attack."

Right. I can't think of a single instance of a vulnerability that affects both Windows and Mac. And Apple have always been proactive and fair when dealing with security researchers, so that's good.

Seriously, though, the real reason Mac's aren't subjected to more drive-by-downloads and malware in general isn't that OS X is significantly more secure than XP or Vista. It's that OS X is still a tiny fraction of the potential pool of malware victims. (An all-time high of 7% as of last month.) It's not worth the money for the botnet czars to develop exploits and bots for OS X. But if somebody really big like, I dunno... the U.S. Army were to deploy Macs in large numbers, then the scale might begin to tip toward profitability.

And then hold on to your blackberry green tea frappuccino, pal. Here it comes.

Why Some InfoSec Training Is Sacred

2008-01-06T13:11:00.000-05:00

Here's another one for the CISO's to ponder.

Think about how you handle professional development across IT. I'll bet that, if you are fortunate enough to have a training budget, that it's based on the number of FTE's in each of your IT budget columns. And this makes sense - it's perfectly fair to invest equally in each person working for you. So it only makes sense that when you reduce spending on training, you do this equally as well. But this could be a mistake.

Professional development, and most IT spending for that matter, can be tied to needs that the business controls. For instance, if you think about server platforms, I'm sure you want to train your people on Windows Server 2008 before you start widely deploying it. But the decision can also be made to subsist on Windows Server 2003 for another year or two. (In fact, you probably have third-party apps holding you back anyway, but I digress.)

Now consider your incident response team. I'm sure you want to keep them trained up on malware analysis, forensics, the latest threats and exploits, etc., etc. But that's money you may want to spend elsewhere. Unfortunately, you and your business don't get to decide whether or not the new threats that come out this year are going to apply to you. They will. They do.

So you see what I'm getting at, right? You can postpone investing in new technologies and therefore the training that goes with them. But you can't postpone new threats. And so you can't postpone spending on infosec training and also expect to be as prepared to handle security threats this year as you were last year. So before you uniformly cut IT training, understand that where infosec training is concerned, there is an elevated risk level that the business may not be able to manage in other ways.

ArcSight Test Alert Connector & Replay

2008-01-03T13:59:00.000-05:00

If you use ArcSight and haven't heard of the Test Alert connector before, listen up. Especially if you have a test environment or need to perform stress testing on your ArcSight deployment/configuration, this should interest you.

There are really two components here. The first is part of your Manager, and that is the ability to generate "replay" files for use with the Test Alert connector. But replay files are actually CSV format event exports. You may find this functionality very useful even if you never intend actually replay them anywhere.

The first step is to think about what events you would like to export. Space is an issue depending on what time frame and type of events you want to export. If you need to stress test your hardware or rules, make sure you are going to get enough events to sustain the event/min or event/sec rate that you'd like to test up to. But beyond that, less is more, since it doesn't take much to get a very large (uncompressed, CSV text) replay file. Also, this feature makes use of filters defined in your ArcSight manager, so if you have specific events you wish to select, review your current filters or create a new one for your export.

The next step is pretty easy. Log in to the manager and run 'arcsight replayfilegen' from the manager/bin directory. Then follow the prompts to log in, select a file name, time range, and filter.

Now that you're generating replay files, it's time to set up your Test Alert connector. Because you're not going to run Test Alert like a service, you can install it anywhere that is convenient for your purposes, your test server, your laptop, whatever. The install is the same as any connector, just select 'Test Alert' in the type dialog and then finish the install as you would normally.

To use it, copy your replay files into the 'current' directory and launch 'arcsight agents' from the 'bin' directory. You will need a GUI display for this, and I have found the X11 display to be flaky with missing or slow redraws to remote X servers. All of the *.events files in the connector's 'current' directory will be displayed. You can turn any combination on/off, select flow rate, and then click 'Continue' to start pumping events into your manager.

Building Didier Stevens' SpiderMonkey in Cygwin

2007-12-27T10:58:00.000-05:00

Here's one for your malware analysis toolkit. For some time now, I've been using Rhino, Mozilla's Java implementation of JavaScript, to help automate deobfuscation. SpiderMonkey is Mozilla's C implementation of JavaScript, including a shell much like Rhino's.

There are a couple of things that Mozilla's engine doesn't do when it comes to deobfuscating JavaScript. Specifically, you're left to manually convert eval and document.* calls yourself. That's where this really smart guy Didier Stevens comes in. He has a modified SpiderMonkey that solves both of these issues.

So you already know that I like Cygwin for lots of things, including malware analysis. Unfortunately, SpiderMonkey is really only intended to build on Win32 with Visual Studio. However, there are a couple of quick shortcuts you can take to get it to build with gcc in Cygwin. So here we go.

1. Install Cygwin with gcc and standard C libraries.
2. Download and untar Stevens' SpiderMonkey source tarball.
3. In js/src/config/Linux_All.mk find the line that begins with MKSHLIB and change the ld linker syntax by replacing '-shared' with '-r':

$ grep -n MKSHLIB config/Linux_All.mk
50:MKSHLIB = $(LD) -shared $(XMKSHLIBOPTS)

4. Build using make with the following syntax:

$ make -f Makefile.ref OS_ARCH='Linux'

We're essentially lying to make to get it to build as if our Cygwin environment is a Linux box. This is why shared linking breaks. But it should be a non-issue.

5. The make will exit with errors, but if all went well, the JavaScript shell, js.exe, has already been built:

$ cd Linux_All_DBG.OBJ
$ ls -l js.exe
-rwxr-xr-x 1 nobody None 1493267 Dec 27 17:40 js.exe
$ cd
$ cp js/src/Linux_All_DBG.OBJ/js.exe $HOME
$ ./js.exe
js> document.write("oh word!");
js> ^C
$ cat write.log
oh word!

And that's it. Make a copy of the binary for future use and clean up.

Storms

2007-12-25T14:23:00.000-05:00

A nasty storm blew threw West Michigan on Sunday and Monday with lots of wind damage. It knocked out power to major chunks of the city, including the airport, which is literally close to home for me. Fortunately, the uptime on my OpenBSD box shows 25 days, so we never lost power (or cable, which is good, because I am trying to Tivo My Name Is Earl reruns that I missed.) So if you're family and you're reading this, we're safe and warm.

And then there are the latest mutations of that ongoing Storm thingy that fortunately doesn't leave people homeless or stranded. It's still annoying, though. You can't help but get the sense that the spammers are all taking advantage of the holidays:

That's 60 new spam messages in my Postini quarantine since Friday. That's not my GMail account, which has closer to 7K, but rather my work-only address which is seldom-published and hardly sees any spam. Additionally, Postini only shows me messages it's not sure about, so that's almost always new variations of spam messages. For me to have 60 in a month is rare, let alone a few days. The dirtbags have been busy.

On a Lighter Note...

2007-12-21T14:27:00.000-05:00

Say what you will about Bill Gates, but sometimes he does something that you just have to admire. According to Reuters, he's recently acquired a stake in FEMSA Cerveza, a Mexican beer and soft-drink conglomerate. Mexican beers don't often make it to the top of a beer snob's list, but for my taste, Bohemia is one of the better pilsners out there. Plus, it's usually cheaper than, say, Pilsner Urquell. And cheap beer is good beer when it's also good beer.

On a Personal Note...

2007-12-21T13:27:00.000-05:00

If you're one of the people that has my blog in your feeds list, then you've no doubt noticed that I have not been posting much lately. At all. I hope to get back to it in the new year, but Q407 has been insanely crazy for me, and I had to prioritize my time across the board.

But it's not bad news. Quite the contrary, actually. Made official just this week, I am now the head of infosec as well as the corporate infosec officer at the company where I work. My good friend and mentor, Tim, is returning to his technical roots but otherwise staying put. It's pretty much a job swap for the two of us, with Tim becoming the infosec team's technical lead.

I thought long and hard about the offer before accepting, and I came to a realization. I haven't worked on a team this talented in a decade. My mentor and the man I am succeeding will remain on staff as a resource to me and I to him. I will never get a better opportunity to step up to leadership. I will never have more support and more talent behind me than I do now. It's a little much to digest, really, and I think the rambling nature of this post gives you a hint at just how much my head is still swimming at the idea.

Anyhow, I hope to resume blogging in the new year as time permits. I have a couple of ideas that, if I find some time over the next few weeks, I may polish enough to post. Anyway, I hope that wherever you are, that you find peace and prosperity in the New Year.

PaulM

Deloitte Data Disclosure Study

2007-12-13T16:09:00.000-05:00

So, I can't decide what this study really means. The short version is that Deloitte did a survey of security & privacy staff from the US about data breaches and disclosures, and 85% of respondents had at least one incident, and 63% of respondents had six or more in the past 12 months.

But I don't know if this is the sky falling, or just the entropic nature of data. Clearly 85% of companies are not having TJX-sized breaches. But the 85% is apparently incidents where notification ocurred. Unfortunately, the report doesn't expand on what constitutes notification and whether that means specifically that individuals were notified.

Either way, this study raises a good point around incident response. Specifically, due to the ubiquitous nature of mandatory disclosure laws, it's time to revisit your incident response procedures and include language for determining if notification is necessary, and then coordinating and documenting notification efforts so that you can prove that you followed applicable laws.

2008 Security Blog Predictions

2007-12-07T14:35:00.001-05:00

Predictions seem to be a less popular topic this year than they were last year when nearly everybody with a blog made a stab at security predictions for 2007. There are still a few who have dusted off their crystal balls and taken a stab at it.

My blog wasn't up and going last year, so there are no poorly made guesses about security trends out there for you to hold me accountable for. This year will be no different. Instead, I present to you, dear readers...

My 2008 Security Blog Predictions

MSRC will continue to only post on the 1st Thursday and 2nd Tuesday of each month.
Matasano will burn up their clients' 2007 budgets and start posting again in January.
Richard Bejtlich will still be the only guy blogging about network taps.
Raffy will still be the only guy talking about AfterGlow, even though it works with Snort and Greg Hoglund used it in his new debugging tool.
Nate Lawson's blog will be surpassed by Chris Eng's as the most difficult to digest. Especially if Nate keeps posting exclusively about vintage computers and BaySec.
The Wired Support Intelligence blog will finally be declared abandoned and taken offline.
People will continue to read Schneier's blog, even though it's just Bruce riffing one-liners on 2-week old articles.
I will finally read WebSense Labs' blog regularly because they will add an RSS feed.
I will finally blog about my experiences upgrading ArcSight 3.5 to 4.0, because my hardware will eventually arrive and I will finally be able to do the upgrade.
...and last but not least, security blogging will continue to really just be all about Google page rank.

Thank you, and good night.

Tis The Season

2007-11-27T11:04:00.000-05:00

Sunday morning I followed up on a case involving a new mass-sploiter. It was interesting - PHP remote file inclusion attack with a hosted exploit that was targeting Windows. Of course, it didn't affect any of the systems it touched on my end, and I decided not to try for the binary. Why not? Because it was Sunday morning, I was at my in-law's house, packing up to go have a late Thanksgiving with my family.

And then it hit me. Get ready. Here it comes. As we head into the holidays, the malware folks are gearing up, hoping to catch us off guard. They've already got the design in place, the new text for socially engineering users and packing & obfuscation tricks to bypass spam filters and AV scanners. They're just waiting. Last winter it was New Year's Eve and then the SuperBowl. The timing of those attacks was no coincidence. This season I expect something similar.

Attack Surfaces and The Impending Headache

2007-11-15T11:12:00.000-05:00

If you rewind 6 years, the big security pain point for most companies was the disruption caused by worms like Code Red, nimdA, Slammer, Sasser, Blaster, etc. The common thread that made these worms so effective, and thus disruptive, was widely-deployed, unpatched Microsoft products.

Today, the threat of a catastrophic worm of this type is almost non-existent in most modern networks. Microsoft fixed code, we deployed client firewalls and automated patching, and got serious about the security of Internet-facing services. This is good news, but it's also a mixed bag. The attacks didn't stop, they just changed.

Other attack surfaces - web applications and web browsers - started to get attention. And today, an unpatched exploit for IE is worth more to the bot/adware crowd than one for IIS 6. But lately there's been an upswing in exploits against third-party apps that integrate with web browsers. QuickTime, RealPlayer, Acrobat Reader, Shockwave, have all had remote code execution vulnerabilities discovered - and exploited by the bad guys - in the past few months. And this is exacerbated by the fact that at least half of your QuickTime or RealPlayer installs are from folks that installed iTunes or Rhapsody so they could sync their MP3 player at work, so you don't even know that they're out there.

But here's the real teeth-kicker. There was also a vulnerability in Viewpoint Media Player announced last week. With an exploit circulating. And I'll bet that until you read about it being vulnerable, you had never heard of Viewpoint Media Player and didn't have (and perhaps still don't have) any idea where it's installed throughout your network.

So now I have to defend mobile workstations against attacks on software I don't even know is out there? We have a pretty tight workstation management regimen where I work, and I was able to poll our software management tool for Viewpoint. And sure enough, there are a half-dozen installs.

So the picture this paints for the near future isn't pretty: even more time spent trolling mailing lists and RSS feeds for new vulnerabilities, expensive software to inventory your workstations and manage the software that's installed on them, a politically charged fight to take away local administrator privileges anywhere you can, and developing new ways to triage and mitigate vulnerabilities while you wait for some tiny software shop to fix the vulnerability.

Or, you could just focus on the insider threat. ;-)

Fixes For ArcSight Console on Linux

2007-11-15T10:04:00.000-05:00

If you're like me and you prefer to run a distro other than CentOS or RedHat Enterprise on your laptop or workstation, you may have run into problems trying to install and run ArcSight Console. So here are a couple of quick hack/fix tips that can get you up and running.

1) The problem: The installer won't run. It gives the following error:

error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory

The fix:

sed -i 's/export LD_ASSUME_KERNEL/#xport LD_ASSUME_KERNEL/g' ArcSight-4.0.0.5206.0-Console-Linux.bin

2) The problem: When I try to run the installer or a previously installed console, I get the following error:

java: xcb_xlib.c:50: xcb_xlib_unlock: Assertion `c->xlib.lock' failed.

The fix:

sed -i 's/XINERAMA/FAKEEXTN/g' $HOME/arcsight/Console/current/jre/lib/i386/xawt/libmawt.so

3) The problem: Some windows don't draw correctly or at all when running the nvidia X11 video driver. I run 'arcsight console' and it hangs.

The fix: Switch back to the lame, non-AIGLX nv driver.

The hack: For some reason, if you run Java inside of strace, it works. (I suspect this has to do with Java threading.) Edit the execjava.sh script in the current/bin/scripts directory. The very last line begins with "$JAVA_HOME/bin/java". Put 'strace' at the beginning of that line. Now run 'arcsight console' like you normally would. Using 'strace' generates a lot of overhead, and will slow the console down, but it runs, which is more than you had before. And if you just can't give up compiz's wobbly windows, this may work for you.

Snort Turns 9, Marty Talks About 3.0

2007-11-09T15:01:00.000-05:00

Snort turns 9 years old this month. It's come a long way and gotten a lot bigger:

paul@arnold ~/snort-0.96$ find . -type f |wc -l
21
paul@arnold ~/snort-2.8.0$ find . -type f |wc -l
1311

Today Marty blogged about the changes that Sourcefire has in mind for the 3.0 engine. Some of this is old news, some of it's brand new. Here's what I found to be of specific interest:

0) Rewrite the core frameworks for Snort from the ground up to clean out code base cruft and leverage external libraries where possible to [...] effectively reduce the size and complexity of the code base making it easier to extend and ultimately lending the security benefits of a smaller code base.

Amen. The Snort 2.8.0 binary alone is over 8MB, to say nothing of the dynamic preprocessor libraries. It can be more if you compile in support for MySQL or PostgreSQL. That said, 2.8 and stream5 are significant improvements over their predecessors. If you're still on 2.6 or 2.7, don't wait for 3.0.

1) Build an "contextually aware engine", one that has the ability to understand what it's defending built around the concept of network context. Network context is essentially data about the environment that is being defended by Snort, the composition of the hosts in the network as well as the local network composition.

I'm glad to hear Marty say this, though frankly it's part of what we've built with the help of Oinkmaster. Trying to get the right rules turned on and looking at the right traffic is tough and requires deep knowledge of your network and how to configure your IDS. And it takes time. But it's worth it, and it is definitely the hardest part of tuning out false positives (and avoiding overtuning so that you miss real attacks). Anything Sourcefire can do to make this process more intuitive is a good thing in my book

2) Abstract and compartmentalize Snort's subsystems to make components "separable".

Sure. I think that we started seeing this with 2.6 and the dynamic preprocessors. I would like to add that I think Snort is due for an update or replacement for barnyard. Something more flexible and more easily integrated (and with better documentation) would be nice.

4) Add an interactive shell to the system so that it may be more fully orchestrated at runtime.

He's talking about Lua. I like the idea of an interactive shell interface to the engine. Honestly, though, I'm not sure what I'd do with it.

5) Multithread the engine to take better advantage of multi-core platforms that are standard today.

If you wouldn't hate me for it, I'd embed dancing hamsters or puppies or something equally ridiculous as a symbol of my elation. In other words, it's about effin time. Snort being single-thread only is, in my opinion, the single greatest scalability barrier that it has.

Data Source API - An abstraction API between the facilities provided by the data source and the rest of the Snort 3.0 software framework. This API exists to that the rest of Snort 3.0 can work without caring whether the Data Source is implemented as hardware or software.

And last but not least, what may be a second I-told-you-so for me this week. I don't know, but I suspect the purpose of this API is to eliminate the need for a LibPcap-bound network interface and open up the possible ways Snort can acquire network data like, say, disk?

Targeted Phishing, You Don't Say?

2007-11-08T11:51:00.000-05:00

I hate to say it... Oh, who am I kidding? I LOVE to say, "I told you so!" This is actually pretty neat, so long as you're not salesforce.com.

(Via Schneier) Salesforce.com admitted today that one of their employees was the victim of targeted phishing. And that once his account was compromised, it was used to get lists of e-mail addresses for... wait for it... more targeted phishing attacks!

So as targeted phishing attacks pass from the realm of pen-testers-who-can't-use-debuggers to actual criminals, the anti-spam/phishing segment is going to have to catch up. And it's not going to be easy, because traditionally collecting spam and phishing e-mails has been remarkably easy. But once the attacks become targeted, it's exponentially harder to get samples before the damage is done.

Enter the custom-tailored anti-phishing service. Gonna call those VC folks back.

And They Were All Yellow

2007-11-07T16:30:00.000-05:00

Symantec bought Vontu. Never heard of Vontu? They are an established player in the data-leakage security niche. Primarily deployed on networks that fall under the purview of the Gramm-Leach-Bliley Act, Vontu's flagship product works like an IPS, but instead of loading it up with vulnerability signatures, you load it up with keywords and snippets of your confidential data.

For $350M, this is is a gamble for Symantec for a couple of reasons. First, the expansion of the data-leakage market is very much a question-mark. Sure Vontu's poised to dominate if it does blow up, especially with Symantec's Panama Canal of a channel. But Symantec is a desktop client company. They've killed every network device they've ever acquired, and some that they built themselves. Sure Vontu has a desktop client as well, but it's not their leader.

What I find most interesting about this acquisition is that Symantec is known for paying pennies for secondary niche players and trying to pump them on their brand recognition against primary niche players. Their whole product strategy can be summed up as "one brand, one vendor." In this case, they bought one of the best-of-breed players in the niche, if not the top dog. And they paid good money for them, too. Recent acquisitions like Altiris and Revivio were more of the old Symantec trying to find a bargain buy into a new market. So the Vontu purchase leaves me confused. I would've expected Symantec to buy somebody like Tizor and stay away from Vontu and PortAuthority.

By the way, there's an excellent Forrester paper on Symantec's ongoing shopping spree. If you work for a Forrester subscriber, or own a lot of Symantec stock, it's worth reading. (I am the former and, at not the latter, for what that's worth.) If you're keeping track, Symantec has acquired no fewer than 31 companies since 2000.

Also, Vontu co-founder (and recent multimillionaire!) Joseph Ansanelli testified before a House subcommittee about combating identity fraud. (PDF Link) Another interesting read, but when you contrast this with the recent ID theft study that Bruce Schneier blogged about today, you have to wonder if there's a decent sales line for these products beyond GLBA compliance.

Am I Not In On The Joke?

2007-11-06T16:33:00.001-05:00

So I just found Security Mike's Guide to Internet Security.

You have to understand that I respect the hell out of Mike Rothman. Which is why I am choosing to believe that this is an elaborate tongue-in-cheek joke that I'm just not able to extract the punchline from.

This quote in particular has me convinced that this is some sort of hoax:

"You certainly can pay your local Geek to come over and configure your computer and sell you lots of software you have no idea about. Bring your checkbook – it’s going to run you hundreds You can do it yourself of dollars. And you get to pay every year to renew your software as well. Don't forget the Geeks get paid when you buy software as well, so they have an interest in loading you up with stuff you don't need.

It’s not right. So I decided to do something about it."

That something is selling a 6-month website subscription for $37. So either I have just seen the Lone Ranger take a bribe and slap an old lady, or I am still not in on the joke. Mike's selling a book for the mom set on how to secure their own computer? Because paying for McAfee is some sort of injustice?

I teach a course very similar to Mike's book through my employer's corporate training program. If you would like a copy, e-mail me, and I will send you the slide deck. Steal my bullet points. Pass my advice around. I don't want any money. If you feel like giving me credit, that's cool. The people this is really for don't know who I am anyway.

Be free, common sense, be free!

I'll just leave you with this:

"Best of all, there is NO RISK to you. You don’t like Security Mike's Guide? Get your money back. [...] Regardless of the reason, if you are unhappy – I will send your money back. That’s right. If you aren’t happy, you can have your money back. I’ll wish you good luck because Security Mike’s Guide isn’t for everyone. It’s all good."

It conjures images of clowns and ponies and free hot dogs at a used car lot.

For the Paranoid

2007-11-05T16:06:00.001-05:00

Been too busy to blog lately. Got a few things half-ready to post. Just need to find the time, motivation, and answers to get them posted. So this is just a proof-of-life post, I guess.

This story from Radar Mag made my day, sort of. It's an excellent story, but if you're paranoid like me, it may take you some place you'd rather not go. Maybe I should move my blog to typepad. :-)

The Heartbreak of Nondisclosure

2007-10-26T10:38:00.001-05:00

Look what I've got in the test lab this week:

It's a little more recognizable with the front bezel on it:

That's right, it's ArcSight Logger 2.0 beta! Alas, the non-disclosure agreement prevents me from telling you any more than that. OK, I'll also tell you that, much to my disappointment, the cool logo bezel does not light up.

Addendum: If I Could Tell Your CISO 3 Things

2007-10-21T21:30:00.001-05:00

On the issue of spending on monitoring versus prevention, I stand by what I said about spending on monitoring equal to prevention. But there's another point worth making that I missed the first time around. So, if I may, I'd like to tell your CISO another thing.

1b) Let the results of your 2007 monitoring determine what you spend your 2008 prevention dollars on. Simply put, no consultant, auditor, or magazine is going to know better than you what your security problems are. So, unless you still don't believe me about monitoring, don't let them tell you how to spend your money. (Remember that "deep pac ket inspection firewall" you bought in 2005? That's what you get for listening to a magazine.)

Set aside time each year to review what your big messes were as well as where your analysts spent the majority of their time. Then look at the market for technologies that can cut the amount of time your talent spends doing the same thing over and over by hand. Also look at technologies that can help you keep the promises you made under your breath to never let _____ happen again.

So while there may be no Security-ROI-Santa-Claus, comprehensive operational security is self-supporting. Leverage it to the maximum extent that you are able.

A Little Wi-Fi Hacking With Your Half-Caf Nonfat Mochachino?

2007-10-21T21:11:00.000-05:00

So like, literally right now Vivek and Sohail from AirTight networks are presenting on a new attack on WEP at Toorcon. This new technique, cheekily dubbed Cafe Latte, attacks clients instead of access points. But according to an interview that the researchers gave prior to Toorcon, the attack can take from a few minutes to a few hours, making it no more efficient than existing techniques.

Cool research guys, but I guess the question I have is this. If I need to attack a mobile client instead of an access point in order to avoid detection by, I dunno, a wireless IDS of some sort - and I have to struggle with position and availability of the target, no less - won't I be shocked to discover that your technique works because this highly secure wireless network uses WEP?!

I'm just saying. Attacks against wireless clients in the field are interesting, and fertile ground for all sorts of cool hacks and lucrative crime. But - and maybe I'm missing the obvious here - I don't get it.

A Little YouTube Nostalgia

2007-10-15T10:16:00.000-05:00

Nothing serious, just some computing throwbacks.

Remember when Bill Cosby sold computers? Or when Windows 1.0 came out? (Yeah, that is Steve Ballmer in the godawful jacket.) What about when Commodore 64 got a joystick? Did you even know that Atari made computers?

I had a TI-99/4A back in the day. With the 300bps acoustic coupler and the cassette storage cable to record my BASIC programs for later retrieval. I'm so friggin' old I could cry.

State Penn

2007-10-12T10:26:00.000-05:00

I just got this story off of Engadget. It only has a little something to do with security, and my rant even less so.

Penn State has developed a high-security environment for students to take exams in. This is a total waste of technology. The point of this is to ensure that students cannot cheat on tests by using iPods or cell phones to store potential answers to questions. In my day, it was graphing calculators, and in my folks' day it was arms up shirt sleeves.

My point is not that invasive, high-tech monitoring can't work, though it probably can't. My point is that it only allows the continued perception of validity of the worst testing higher education has to offer - memorization. Computers are for data storage. Human minds are for imagination, applying concepts, and learning. None of this can be stored on an iPod. Professors who insist that students learn by regurgitating facts that can be digitized and retrieved with Ctrl-F only serve as a barrier to learning.

On George Clooney and HIPAA

2007-10-10T09:32:00.001-05:00

Palisades hospital in New Jersey has suspended 27 employees for accessing actor George Clooney's medical record after he was treated there following a motorcycle crash. I don't disagree with the employees' suspension, but the hospital spokesperson told reporters, "What these individuals did was violate a HIPAA regulation. We can not say that they actually released any of this information to the media."

It's clear that someone did leak to the media information from his medical record, but the hospital doesn't know who. Additionally, these employees had access to patient EMR data as employees of a covered entity (the hospital). So I'm picking a nit here, but I do believe the hospital has admitted that it doesn't know which of the 27 employees suspended, if any, actually violated HIPAA. As far as I can tell they were, under the law, authorized to view Clooney's medical record. Of course, what they did was still inappropriate, unprofessional, unethical, and probably a violation of hospital policy.

But perhaps the best-slash-worst part of this whole situation is that a union rep defending some of the suspended employees has been quoted as saying, "There are hospital obligations to have security systems so that a breach can't occur -- obviously that failed."

Phishing Secure Email Portals

2007-10-09T12:05:00.000-05:00

Here's a new twist on an old scam:

Lots of companies have implemented some form of "secure e-mail" solution. If you haven't seen this before, a user at Megabank or Gotham Hospital sends you a message about your personal information. Instead of arriving directly over SMTP (which is, among other things, as clear a text protocol as any), you receive a notification via SMTP that tells you to click on a link to a web site (encrypted with SSL) where you can log in and retrieve your message. This is extremely common in the health care vertical because the HIPAA Privacy Rule that went into effect in 2003 explicitly forbids sending personal information unencrypted over the Internet.

So it makes perfect sense that these portals are worth phishing - they are almost guaranteed to contain some sort of valuable data. But it got me thinking about something else. I work in the health care vertical, and we have a secure e-mail solution in place. And when we evaluated products a few years ago, we discovered some sort of session handling flaw in better than half of the products we looked at. Not to mention that a number of the vendors out there support what can only be described as a "letter-of-the-law" configuration*.

Anyway, I wonder if phishing is all that necessary for sites like these. I would bet that there are enough vulnerabilities in enough of these portals that hacking them straight up is a better bet for the criminals that want the dumps to sell on IRC. Especially since some of the third-party products out there are appliances that insist on SSL termination at the appliance. What's that mean to a hacker? A blind spot to the IDS plus permission from the firewall. Oh, and we all know how good the logging on an appliance like that is bound to be.

* In this mode, the portal sends a link that contains a hash of some kind. Send that link back with the valid hash, view the message. Well, technically, the private data's not sent unencrypted. Instead, a link to the private data is sent unencrypted. If you have deployed something like this and you feel that you can justify it, I'd love to hear from you. Obviously there was enough demand for it since most of the vendors in this space have something like it.

If I Could Tell Your CISO 3 Things

2007-10-08T20:29:00.000-05:00

This is me on my soapbox. Preaching to the choir.

1. Buy more monitoring.
It's necessary to spend security dollars on prevention and protection technologies. But it's very easy (and thus very common) to overspend on these technologies as well. Budget and spend at a prevention-to-monitoring ratio of 1:1. Security monitoring is the cornerstone of security response, and in many ways response is more important than defense.

Think of it this way. As CISO, you are the mayor of Securityville, which is on the border of North Korea, Iran, Chechnya, Darfur, and Canada. When you spend on prevention products, you are buying fences and sprinklers to keep bad guys out and keep fires from spreading. When you don't buy monitoring tools, you lack cameras and smoke alarms to tell you that the fence has a hole in it and everything is on fire. To say nothing of the police and firefighters. Which brings me to...

2. Hire more firefighters.
And by firefighters I mean security analysts that can monitor for and respond to security incidents. In 2007, if you haven't experienced a security breach yet, you probably don't believe me when I tell you it's an inevitability. But when you reread this 2 months from now, you'll know I'm right. Or you'll smugly chuckle at how this post is all FUD while Chinese hackers rifle through your e-mail unhindered. Either way, if your security folks are all busy managing firewalls and doing vulnerability scans and nobody's monitoring your network, then you can't argue my point because you don't even know that you've been pwned.

Also, hire good people. Talented people. Security monitoring is not a help desk job, so you can't pay help desk pay for it. I'm proud of our team's incident turnaround time and ecstatic about the fact that in most cases we detect and respond to incidents before the impacted employees are aware there's a problem. But this is the natural order of things, because...

3. Security is not everybody's job.
So stop saying it is. Cindy's job is processing expense reports. Tom's job is developing new client accounts. Jim's job is, well, I don't know what Jim does, but he runs Fantasy Football each year, so he can stay. Oh, right, back to you and how security is your job.

If you want employees to act securely, then you must do the (very unpopular, unfriendly, unfun) job of writing and by God enforcing data security policies. It's really cool if you can write them, design the oversight and monitoring controls, and then hand enforcement over to the compliance or audit departments. Then you'll still get invited to happy hour every once in awhile. But not by Jim. He's not talking to you since he was written up for distributing NCAA brackets printed on the blank side of old payroll reports.

Is Your IP Address Personal Info?

2007-10-03T14:46:00.001-05:00

According to a German court it is. (via Eric Fitzgerald's blog)

The remedy that this ruling implies - not logging IP addresses to a web site beyond the duration of the user's session - is either unsustainable or crippling to site security.

If it becomes standard practice in Germany to not log IP addresses anywhere for any length of time, they will essentially be declaring open season on themselves. There will be no network evidence trail and therefore no case to prosecute. I can't imagine it'll come to that, but it is interesting to ponder.

Paris Got a Raw Deal

2007-10-02T10:30:00.001-05:00

OK, so this might be proof that Paris Hilton's prison sentence was too harsh. An MIVD official (read: high ranking Dutch spy) was sentenced at The Hague for losing some part of an NSA intelligence feed he had access to in his role (as a high ranking Dutch spy). The sentence? 120 hours of community service. So, uh, I guess if you live in Utrecht, keep an eye out for a guy in a tuxedo picking up trash along A27.

TJX: A Glimmer of Clue?

2007-10-02T08:08:00.000-05:00

This is the first time I've heard anyone say anything about TJX doing something about their network security posture. But read between the lines here. WEP has been thrown under the bus, they've implemented WPA, but all of these credit card numbers lived in a database.

Is it safe to assume that the sa or sysdba password was different than the WEP key? OK, then maybe WEP wasn't the only problem? It's disingenuous to make WEP the scapegoat for what is a larger security failure. But, hey, at least they're using WPA now. Anybody taking bets as to whether or not it's WPA-PSK?