BackTrack (EH.net remix)

I was waiting for my download to finish before I told you that EthicalHacker.net has released a version of the BackTrack live CD that A) is also a VMWare appliance and B) has Metsploit 3 ready to run. (It also has the latest aircrack-ng for those long airport layovers.)

OK, my download is done.

Firewalls, SIM, and Visualization

Saudi asks for help on the loganalysis mailing list:

"Looking for help in identifying meaningful/actionable reports that we can get from Firewall log analysis."

Normally, I would've replied to the list, but attaching a bunch of jpeg files that will be sent to hundreds of people is poor etiquette. So instead, I'll spam the list with a link to this blog post. :-)

Reports are great and all, and you've gotten some excellent suggestions so far. But I'm a believer in mjr's artificial ignorance model for log analysis, so I put a high value on finding things that I don't know that I'm looking for. And when you want to do that with millions of events, visualization is the way to go. So here are some ArcSight data monitors that I have that are specific to firewall data.



This is a pair of moving average graphs. The green one is 'accept' messages and the red one is 'drop' or 'reject' messages. Big spikes or dips in these graphs are interesting. The other thing you can't see in these is that there's a second line along the bottom. That line is the failover firewall. When it fails over, both graphs draw a pretty 'X' with intersecting lines.




This is another moving average graph. I love these things! This one isolates workstation VLANs (so this is user-land only) and pairs srcaddr/dstport. Big spikes and long plateaus are usually interesting. The plateaus have traditionally been malware trying to scan or send spam. We've gotten better at catching this stuff on the front end, though, so I rely on this less today than I did 2 years ago. Also, if multiple lines are doing the same thing, that's interesting, too, since it can mean multiple infections.




This data monitor shows, to-scale, firewall events by hour, by severity. Any place you have visible orange or red or green is probably interesting. Also an abnormally high or low event count per hour is also interesting. This one above shows the overnight, so the yellow, orange, and red appear more prevalent because there are fewer events in those buckets.




This data monitor is a pie graph that shows last-hour firewall events by target country code. This probably doesn't work for all organizations, but my company is based and does business exclusively in the US. That means that any large amount of traffic destined for RU or CN is probably the start of a bad day for me.




This data monitor is just a chart that displays the Top 10 sources of blocked traffic. I've whited-out the actual IP's, but you can see the zone details. (The top 3 DMZ servers are due to a recent change in the firewall that the servers haven't caught up to.)

One of the cool things about SIM visualization gadgetry like ArcSight's data monitors is that these displays are in near-realtime. So it's like a report that's always running, and that's really easy to operationalize - "Here, stare at this for a few minutes every so often. If it looks weird, click on it and find out why."

A Message for Digital Flow

I was following up on some suspicious JavaScript content and found this:

//****** Advanced DHTML Popup Pro Version 2.40.096.201.019, Build: 130 ******
// Copyright (c) Digital Flow Software 2005-2006
// The present javascript code is property of Digital Flow Software.
// This code can only be used inside Internet/Intranet web sites located on *web servers*, as the outcome of a licensed Advanced DHTML Popup application only.
// This code *cannot* be used inside distributable implementations (such as demos, applications or CD-based webs), unless this implementation is licensed with an "Advanced DHTML Popup License for Distributed Applications".
// Any unauthorized use, reverse-engineering, alteration, transmission, transformation, facsimile, or copying of any means (electronic or not) is strictly prohibited and will be prosecuted.
// ***Removal of the present copyright notice is strictly prohibited***

And subsequently, this:

Unblockable popups

The popups that are created with Advanced DHTML Popup are not blocked by standard external window blocking software as they are part of the web page and not windows on your visitors desktop.

So, first of all, I would like to say that for as long as your "intellectual" property appears on my network just like a malware dropper, I will continue to reverse engineer its content to verify its intent. Second of all, you guys seem pretty smart. Why couldn't you find real jobs?

eBay Hacked?

Sounds like somebody may have hacked an eBay server or two and dumped member dox to one of the forums. Keep a close eye on your credit cards that are associated with eBay and/or PayPal. Also keep an eye out for an announcement from eBay of the personal info disclosure.

Update: Maybe not. May have been a problem with eBay software. Either way, personal info seems to have been disclosed. Here's a post (that's subsequently been pulled) from an eBay employee:

"xman@ebay.com View Listings Report 26-09-07 00:47 EST 82 of 88The site wasn't actually hacked... it was a server issue where the system displayed the poster's information rather than the post itself. Being that the credit card information was on a different server, that info came up incorrect. It was not some hacker sitting there entering in someone's information and using a card generator."

More here.

Another Update:

Check out some video of the actual data. Look out for the cheezy Nintendo music. It'll sneak up on you!

TJX Settlement Close?

According to a boston.com article, a tentative settlement has been reached in the TJX breach class-action lawsuit. If the judge accepts the settlement, consumers will get:

1. Up to a $30 voucher per customer who can show time/money spent dealing with the breach (at a rate of $10/hr).
2. 3 years of credit monitoring and identity theft insurance for about 450K customers who had lots of info (including DL# and SSN) stolen.
3. Marshalls and TJ Max will hold a 3-day "Customer Appreciation 15% Off Sale." (I kid you not!)
   

Unfortunately, the settlement lets TJX avoid admitting breach of contract and negligence with regard to its data security practices. Also apparently missing from the settlement is any commitment from TJX to improve security. Of course, with the settlement costing an estimated $256M, we can hope that the board and execs at TJX have seen the light on security spending.

Expert Advice

I feel a little strange about being proud of this achievement, but anywhere my name appears along with the word "expert" in the same context as folks like Kevin Kadow and Lenny Zeltser, it makes my head swell.

It's Official

The Grand Rapids ISSA chapter has secured local evil* genius Matt Carpenter to present at our October meeting. Matt is a SANS instructor and security analyst at Intelguardians. He will be presenting the research he and Tom Liston debuted at SANSFire 2007 on VM escaping.

If you're in Michigan or even Northern Indiana or Northeastern Illinois (or Eastern Wisconsin - the ferry drops you in Muskegon, just 40 minutes away!), and couldn't get to SANSFire, this is your chance to catch part of it for free.


* Matt's not actually evil, in fact he's a genuinely great guy and I'm honored to call him my friend. But if he were evil, we'd all be in big, BIG trouble.

September GRSec Announced!

Go here for details.

Mike Rothman on the ArcSight IPO

I'm probably the last person to blog about this. I was at the conference when Robert Shaw announced that ArcSight had filed with the SEC, but I missed the actual announcement thanks to a trip into Adams Morgan the night before. The rest of the day there were conversations about who could or would buy ArcSight once they go public, what the initial share price would be, how the stock would perform, etc. The buy-up probably won't happen, but interesting stuff to hypothesize about.

In all honesty, I was going to skip posting about it until I found Mike Rothman going all sour grapes on ArcSight's IPO filing. And that made me smile, so I thought I'd share. I would like to crown Mike Rothman "The Official SIM Playa Hater." It's like some SIM vendor backed over his puppy with their car when he was a kid. :-)

Defcon 15 Talks Online

Couldn't get time off work? Customs denied you entry to the US? Banned from the casinos in Vegas? Not to worry. Now you can watch the Defcon talks with the same blurry vision that most attendees had at the time!

Here are my picks:

1) Bruce Potter
2) Major Malfunction
3) Jared DeMott
4) Johnny Long
5) Bruce Schneier

But for a complete list, check out the full list of speakers.

The Future of Vulnerability Disclosure

If you're not already talking about this issue, this is your wake-up call. Going back 15 years or more the debate, if you can call it that, around disclosure has been the issue of when and how much to disclose. The two extremes were CERT who wouldn't disclose until the vendor released a patch, and the early days of the full-disclosure mailing list where people simply posted proof-of-concept exploits without bothering with the vendor.

But this year I've heard more about disclosure than I have in almost a decade. And for those of us in security operations, it's not good news. The monetization of the Internet is having an impact on vulnerability research and consequently the issue of disclosure. This has lead to a number of developments all focusing around the issue of whether or not vulnerability researchers should be paid and by whom. In the past, the model was responsible disclosure and if you worked independent of the vendor, you could release your advisory in conjunction with their patch. Researchers got credit and hoped that it would draw in business.

Of course, the pool's a lot more crowded for researchers these days. And where a researcher could've drawn a nice salary at a security consultancy like ISS or @stake and still gotten on-the-clock time to research bugs, they've been consolidated and exist under the profit-driven umbrella of a publicly-traded company. This means a lot of new as well as a good number of "old" (which makes them roughly my age) vulnerability researchers now have to pay their own way. Combine this with positions taken by large vendors like Microsoft and Cisco that they will not pay third-party researchers, and this means lots of smart people - make that smart people that hack stuff - are out there trying to find a way to get paid for their work.

So now, researchers are selling to companies like 3Com/TippingPoint but it's safe to say that at least a few of them are selling to malware kitmakers on the black market. There have also been a couple of attempts at agnostic auction houses that let researchers sell to the highest bidder, whoever that is. This means that at some point in the future, the next big vulnerability could go not just first, but exclusively to organized crime. If that doesn't scare you, it should.

But what should scare you more is that the biggest stakeholders - the infosec ops folks on the front lines, the IT orgs they work for, and software customers everywhere - literally don't have a seat at the table on this one. Black Hat and Defcon both had roundtable discussions on disclosure, and you heard a lot from researchers, lawyers, software vendors, and infosec product vendors. But so far as I can tell, the people that will be impacted most haven't been asked for their opinion. I wish I had a good idea for how to get us heard on this issue. But I don't. So I guess all I have for you is something more to worry about. Sorry.

The Bride of Useless Statistics

It's Friday, so that means it's time to pick on another Dark Reading story. I've decided to dust off an old theme and once again confront bad statistics in the press.

Tim Wilson's story, "Insider Threats Increase, But Damage Is Minimal" and the CSI/FBI Survey that he sites describe the losses from incidents involving insiders as being significantly lower than those caused by outsiders. The problem here isn't really the math as the survey itself, which , in my opinion, incorrectly categorized a number of incident types and then averaged all of their costs, skewing the data significantly.

From Tim's article, "Insider abuse of Internet access was the most frequently-cited incident among the CSI survey respondents, at 59 percent. Fifty percent cited the loss or theft of laptop or mobile devices, while 25 percent cited misuse of instant messaging services."

So they counted up all of the losses from people trying to surf porn, lost equipment, and used chat software in violation of company policy. And then they lumped them in with, "Another 25 percent said they had experienced "unauthorized access to information" in the past 12 months, and 17 percent said they have suffered loss or theft of customer/employee data."

So a smaller subset of the incidents described involves malicious intent, not just boorish behavior and bad luck. And as a result, the losses aren't very big on a per-incident basis. But comparing unauthorized chat sessions with electronic embezzlement is apples and felonies.

I also can't help but notice that lots of this stuff is of the easy-to-detect variety. Sure, by now you should be able to catch users trying to access Internet sites that aren't appropriate for a professional setting or know when a laptop goes missing. But of those that responded to the CSI survey, how many even possess the ability to detect when an administrator, accountant, or executive siphons off valuable corporate data and sells it? Data that they are authorized to access?

How will you detect the breach? How will you know unauthorized disclosure occurred? How will you calculate financial losses from it? I would guess that most of the respondents to the CSI/FBI survey can't answer all of those questions in a way that would satisfy their corporate leadership. Hell, I'll bet their vendors can't answer those questions, either.

"Headless" ArcSight Installs

This was brought up by a conference attendee yesterday, and I thought I'd throw my two cents out there for general consumption.

ArcSight has a very nice Java GUI interface for most of the things you need it to do. But if you work in an optimized and/or hardened UNIX environment, chances are pretty good your servers don't have an X-Windows display. This can make installing, upgrading, and managing ArcSight a bit of a hassle, but there are options.

All of the ArcSight installers support a text-only install mode. This is triggered by running the installer with the '-i console' flag. This gives you an old-school text interface (not curses, though having vt100/ANSI emulation is helpful for the password entry prompts) that walks you through the install wizard. You can also run $ARCSIGHT_HOME/bin/runxxxsetup.sh (where xxx is manager, agent, etc.) in text mode. It's a good idea to make sure that $DISPLAY is not set in your shell before you do. If it doesn't detect $DISPLAY, it will run in a text mode also.

However, there are a small number of tools that cannot be run in text mode for whatever reason. One example is 'arcsight database pc' which launches the PartitionArchiver configuration tool. In order to do run this tool, you need an X11 display. So here's how I do it.

1. Boot Linux workstation with X11 display. This is the default GUI for nearly every Linux distro - using something like Knoppix will work fine. It does not need to accept connections to port 6000 across the network (traditional X11 $DISPLAY does, but you don't allow X11 across security domains, right?).

2. Connect to UNIX server with 'ssh -X' which manually configures X11 tunneling over ssh.

3. If you will be su-ing to another user that is not root, run 'chmod 666 $HOME/.Xauthority' WARNING: Do not do this on any system with multiple users, untrusted services, etc. It should go without saying, but...

4. Run su to the user you need to become (root, arcsight_user, nobody, oracle, 4DGifts, whatever) and set 'XAUTHORITY=/your/ssh/user/home/dir/.Xauthority && export XAUTHORITY'

5. Run ArcSight GUI tools like 'arcsight database pc'

I have encountered (with Beryl on Fedora 7) some redraw issues. So be prepared that you might get a window with nothing in it. Also, be patient. The display is being redrawn over an old, inefficient protocol that is additionally being encrypted and tunneled. Performance is good on a LAN or fast WAN connection, but slower connections or connections with latency problems will have slower redraws.

CyberWar is, uhh... what is it?

Oh, yeah, it's just like all other hacking, except it's gov-to-gov.

Richard Stiennon wonders in his Wednesday blog post if the PLA hackers that pwned Ministry of Defense e-mail systems wanted to get caught. His theory goes, government-quality hackers should be so good that they are undetectable unless they intend to disrupt systems. Since neither thing happened in this case, Richard suggests that maybe China meant to get caught in order to send a message.

Personally, I think there's a simpler explanation. It goes back to a conversation that I had with Richard Bejtlich about the detectability of breaches. The "Titan Rain" attacks have been going on for the better part of three years (as far as we know). And there were bound to be good hacks that were still detected. Especially since this story made Time Magazine in 2005, we should assume that all of the Allied Forces' three-letter branches have been on the lookout for Chinese hackers.

We often hear (and some of us often say) that, "We have to get it right every time, the bad guys only have to get it right once," when security folks talk about defending against network attacks. But as soon as the attack starts, that equation flips on hackers' avoiding detection, especially if you're going to stay on a system for an extended period of time in order to gather intel. The number of chances that formal security detection mechanisms will catch you increases exponentially with time. Not to mention curious admins, auditors, and plain old dumb luck.

Bottom line is that the best hackers in the world can penetrate nearly any system, and can cover their tracks well. But eventually they'll get caught, whether they mean to or not. And that's what happened to the Chinese hackers in this case. Oops.

Firing Up The Rumor Mill

So last week we saw the first post from a new MSDN blog - "hackers @ microsoft." It's in my RSS feeds for now. Microsoft hiring hackers is hardly a newsworthy rumor. It's pretty much common knowledge. The big success story of infosec has been Microsoft's product turnaround over the past 5 years. The message there, that you as an infosec professional should take back to your organization, is that throwing money at security works. So tell them to throw more money at you and your projects.

The rumor I want to start has to do with the hiring of new hackers by Microsoft. Specifically, I'm going to loudly whisper that Microsoft may have hired Mark Litchfield. Here's the evidence I have compiled:

1) Mark was supposed to teach at BlackHat with his brother David, but couldn't. According to David, he was denied entry into the US because Customs felt he may have abused the visa waiver program (like Halvar). Apparently, the reason for his frequent trips to the US prior to BlackHat had to do with purchasing a house in WA.

2) But maybe Mark is moving to the US to focus on growing NGS in the states, you say. Except that NGS already has its US headquarters in Dallas.

3) If you dig around in bugtraq archives, you will see that Mark has published vulns in all variety of Microsoft products, from 2003 Server to SQL Server to IIS to IE to Outlook. Of course, Mark has spent a good amount of time publishing vulns in Oracle products as well. But Oracle's not headquartered in Washington. Microsoft is. Plus, Oracle still doesn't "get it." Microsoft does.

ArcSight User Conference

The ArcSight 2007 User Conference is upon us! Well, next week anyway.

A lot of the hits I get are from people searching Google for 'arcsight ...' If you're an ArcSight user that stumbled on my blog and will be at the conference next week, drop me a line or just stop me and say, "Hi." Feel free to e-mail me (p melson at g mail dot com), though I'll be pretty easy to find - I'll be the tall pasty guy in the obnoxious hawaiian shirt standing next to the free beer.

It's probably too late for me to tell you this, but last year's conference was excellent. I am not a big fan of the venue since it is pretty isolated (so rent a car!), but the facilities are plenty nice. The presentations last year were excellent, and are reason enough to attend. Add to that the chance to trade stories and ideas with other users in all sorts of industries along with the access you get to ArcSight developers and support staff, and it's 3 days very well spent.