Paul Melson's Blog

Queries: Excel vs. ArcSight

2009-09-23T22:07:00.020-05:00

Since ArcSight ESM 4.0, reports and trends have been based on queries. Considering that ESM runs on top of Oracle, a query in ESM is exactly what you think it is. Queries are an extremely flexible way to get at event data. But as the name implies, they go against the ARC_EVENT_DATA tablespace, and therefore you can't use them to build data monitors or rule conditions, since those engines run against data prior to insertion into the database.

Anyway, I've got a story about how cool queries are. And about how much of an Excel badass I am. And also about how queries are still better. Last month, I got a request from one of our architects who was running down an issue related to client VPN activity. Specifically, he wanted to know how many remote VPN users we had over time for a particular morning. Since we feed those logs to ESM, I was a logical person to ask for the information.

So I pulled up the relevant events in an active channel and realized that I wasn't going to be able to work this one out just sorting columns. So, without thinking, I exported the events and pulled them up in Excel. So here's the Excel badass part:

If you want to copy it, here it is:
=SUM(IF(FREQUENCY(MATCH(A2:A3653,A2:A3653,0),MATCH(A2:A3653,A2:A3653,0))>0,1))

So A is the column that usernames are in. This formula uses the MATCH function to create a list of usernames and then the FREQUENCY function to count the unique values in the match lists. You need two MATCH lists to make FREQUENCY happy because it requires two arguments, hence the redundancy. It took about an hour for me to put it together, most of that was spent finding the row numbers that corresponded to the time segment borders.

But as I finished it up and sent it off to the requesting architect, I thought, there must be an easier way. And of course there is. So here's how you do the same thing in ESM using queries:

So, it's just EndTime with the hour function applied, and TargetUserName with the count function applied, and the Unique box (DISTINCT for the Oracle DBA's playing at home) checked. And then on the Conditions tab you create your filter to select only the events you want to query against. That's it.

Once the query is created, just run the Report Wizard and go. All told, it's about 90 seconds to the same thing with a query and report that it took an hour to do in Excel.

Malware Analysis Toolkit for 2010

2009-12-28T15:09:00.013-05:00

Back in 2008 I posted a list of the tools I use for doing malware analysis. The tools I use have changed over time, and rather than just talk about a couple of recent additions, I decided I'd put a current complete list up with links. This is by no means a comprehensive list of malware analysis tools, it's just what I like and use.

Platform

VMWare Workstation
The "vulnerable stuff:"

Windows XP
Internet Explorer 7/8
Firefox
Acrobat Reader
Flash Player

General Tools

Analysis Tools

Binary Tools

JavaScript & HTTP Tools

PDF & Flash Tools

Web Sites as Tools

Reversing JavaScript Shellcode: A Step By Step How-To

2009-11-09T11:26:00.028-05:00

With more and more exploits being written in JavaScript, even some 0-day, there is a need to be able to reverse exploits written in JavaScript beyond de-obfuscation. I spent some time this weekend searching Google for a simple way to reverse JavaScript shellcode to assembly. I know people do it all the time. It's hardly rocket science. Yet, I didn't find any good walk-throughs on how to do this. So I thought I'd write one.

For this walk-through, I'll start with JavaScript that has already been extracted from a PDF file and de-obfuscated. So this isn't step 1 of fully reversing a PDF exploit, but for the first several steps, check out Part 2 of this slide deck.

What you'll need:

A safe place to play with exploits (I'll be using an image in VMWare Workstation.)
JavaScript debugger (I highly recommend and will be using Didier Stevens' modified SpiderMonkey.)
Perl
The crap2shellcode.pl script, which you'll find further down in this post
A C compiler and your favorite binary debugger

I'll be using one of the example Adobe Acrobat exploits from the aforementioned slides for this example. You can grab it from milw0rm.

Step 1 - Converting from UTF-encoded characters to ASCII
Most JavaScript shellcode is encoded as either UTF-8 or UTF-16 characters. It would be easy enough to write a tool to convert from any one of these formats to the typical \x-ed UTF-8 format that we're used to seeing shellcode in. But because of the diversity of encoding and obfuscation showing up in JavaScript exploits today, it's more reliable to use JavaScript to decode the shellcode.

For this task, you need a JavaScript debugger. Didier Stevens' SpiderMonkey mod is a great choice. Start by preparing the shellcode text for passing to the debugger. In this case, drop the rest of the exploit, and then wrap the unescape function in an eval function:

Now run this code through SpiderMonkey. SpiderMonkey will create two log files for the eval command, the one with our ASCII shellcode is eval.001.log.

Step 2 - crap2shellcode.pl
This is why I wrote this script, to take an ASCII dump of some shellcode and automate making it debugger-friendly.

---cut---


#!/bin/perl
#
# crap2shellcode  - 11/9/2009 Paul Melson
#
# This script takes stdin from some ascii dump of shellcode
# (i.e. unescape-ed JavaScript sploit) and converts it to
# hex and outputs it in a simple C source file for debugging.
#
# gcc -g3 -o dummy dummy.c
# gdb ./dummy
# (gdb) display /50i shellcode
# (gdb) break main
# (gdb) run
#

use strict;
use warnings;

my $crap;
while($crap=<stdin>) {
  my $hex = unpack('H*', "$crap");

  my $len = length($hex);
  my $start = 0;

  print "#include <stdio.h>\n\n";
  print "static char shellcode[] = \"";

  for (my $i = 0; $i < length $hex; $i+=4) {
    my $a = substr $hex, $i, 2;
    my $b = substr $hex, $i+2, 2;
    print "\\x$b\\x$a";
  }
  print "\";\n\n";
}

print "int main(int argc, char *argv[])\n";
print "{\n";
print "  void (*code)() = (void *)shellcode;\n";
print "  code();\n";
print "  exit(0);\n";
print "}\n";
print "\n";

--paste--

The output of passing eval.001.log through crap2shellcode.pl is a C program that makes debugging the shellcode easy.

Step 3 - View the shellcode/assembly in a debugger
First we have to build it. Since we know that this shellcode is a Linux bindshell the logical choice for where and how to build is Linux with gcc. Similarly, we can use gdb to dump the shellcode. For Win32 shellcode, we would probably pick Visual Studio Express and OllyDbg. Just about any Windows C compiler and debugger will work fine, though.

To build the C code we generated in step 2 with gcc, use the following:

gcc -g3 shellcode.c -o shellcode

The '-g3' flag builds the binary with labels for function stack tracing. This is necessary for debugging the binary. Or at least it makes it a whole lot easier.

Now open the binary in gdb, print *shellcode in x/50i format, set a breakpoint at main(), and run it.

$ gdb ./shellcode
(gdb) display /50i shellcode
(gdb) break main
(gdb) run

Two-For-One Talk: Malware Analysis for Everyone

2009-10-18T15:10:00.001-05:00

These two mini-talks were originally going to be blog posts, but I needed a speaker for this month's ISSA meeting. So I volunteered myself. Here are the slides.

Two-For-One Talk: Malware Analysis for Everyone

View more presentations from pmelson.

The 'Cyberwarfare' Problem

2009-09-20T23:24:00.003-05:00

Last week I attended ArcSight's annual user conference in Washinton DC. More about that in a later post. During the conference, ArcSight hosted a panel discussion on cyberwarfare. In DC, where many of ArcSight's biggest customer are based, this is a hot topic, and there will be a lot of time spent discussing it and a lot of money spent on defending against it, maybe.

What struck me about the panel discussion were two comments, both made by James Lewis, one of the panelists, and a director at the Center for International and Strategic Studies. At one point, Mr. Lewis invoked Estonia as an example of state-sponsored cyberwarfare, and made the comment that, "the Russians are tickled that they got away with it." Not ten minutes later, an audience member asked a question about retaliation against cyber-attacks. Mr. Lewis responded to the question by pointing out the problem of attribution. That is, from the logs that the victim systems generated, the IP address(es) recorded can't reliably be used to identify the actual individual(s) responsible for the attack.

Now, I don't intend to pick on James Lewis. It just so happened that one person on the panel expressed the paradox of cyberwarfare. The attribution problem is a big problem for all outsider attacks, not just cyberwarfare. A decade ago, security analysts were calling it "the legal firewall" because US-based hackers would first hack computers in China, Indonesia, Venezuela, or another country that doesn't openly cooperate with US law enforcement, and then hack back into the US from there, causing an investigative barrier that would hinder or prevent an investigation being able to get back to the attacker's actual location.

So knowing that there's a very real problem with being able to identify the source country for Internet-based attacks, it stands to reason that using the same limited forensic data to not only identify the actual source of an attack, but to determine that it is in fact state-sponsored, and not, say, a grassroots attack armed by a teenager, is a stretch. And for that reason, the question of cyberwarfare is an open one. Until a government actually comes forward and claims responsiblity for an attack, it's unprovable.

So as the government spends $100M on cyberdefense over the next six months, it's important to try and answer the question, "What is the military actually defending against?" At the very least, it's fair to say nobody knows for certain.

Inbox 3

2009-08-12T15:57:00.004-05:00

Teguh writes,

Hi Paul,
could you give some guide to administering logger? i searched thru
google, but found nothing significant. How to(s) and tutorial would be enough i
guess. Does it have to have syslog server for the logger to be able to read data
from?
Thanks..

The documentation for Logger is available from ArcSight's download center. Only registered customers have access, but I assume that if you've got a Logger box, that generally qualifies you.

With regard to your second question, yes Logger has a syslog server. It actually has a few. In Logger nomenclature these are "receivers." Logger supports UDP and TCP syslog, FTP and SSH file pull, NFS and CIFS remote filesystem. Logger also supports some ArcSight-specific receivers including a SmartMessage receiver for events forwarded from ESM and CEF-over-syslog (OK, ArcSight wouldn't agree that this is specific to their products, but despite the C standing for Common, CEF is anything but. At least right now.)

Configuring Logger to act as a syslog server is pretty straightforward.
From the web interface, navigate to Configuration, Event Input/Output.
On the "Receivers" tab, click the Add button.
Name your connector and set the type as "UDP Receiver" then click Next.
The defaults for Compression Level and Encoding are fine. Select the IP address you want the listener to reside on, and set the port number. The default syslog server port is UDP/514.
Click Save.
On the "Receivers" tab, click the little no-smoking image next to the new receiver to enable it.

Nobody Sells Laptops for The Price of Silver

2009-06-23T12:47:00.005-05:00

If you haven't already, I recommend that you take 20 minutes and read "Nobody Sells Gold for the Price of Silver" by Cormac Herley and Dinei Florencio. (PDF Link) This is an excellent analysis of the research into and press coverage of the underground economy. It's a fascinating read, and they make a cogent argument that the underground economy is more myth than reality. I don't want to say more because it will ruin it for you.

Now I have an excercise for you. First, read the Herley/Florencio article. Then, read Bruce Schneier's experiences with trying to sell a laptop on eBay. Now think about the implications of the "Ripper Tax" on eBay. Now ask yourself why you haven't already sold any stock you own in eBay.

PCI-DSS and Encrypting Card Numbers

2009-06-18T22:22:00.009-05:00

OK, I'm about to do something dumb and talk about cryptography and cryptanalysis. I'm an expert in neither of these things. But despite the fact that somebody smarter than me should be telling you this, you're stuck with me, and I think I have a point. So here goes.

I had a bit of an "A-ha!" moment earlier today around PCI-DSS, specifically requirement 3.4 from v1.2 of the standard. Here's the relevant language from that requirement:

3.4 Render PAN, at minimum, unreadable anywhere it is stored (including on portable digital media, backup media, in logs) by using any of the following approaches:
One-way hashes based on strong cryptography
Truncation
Index tokens and pads (pads must be securely stored)
Strong cryptography with associated key-management processes and procedures

The bottom line is that this requirement fails to provide adequate protection to card numbers. Here's why.

Truncation and tokenized strings with pads have limited use cases. In the case of truncating card numbers, PCI-DSS recommends only storing the last 4 digits of the card number. You wouldn't choose truncation for a program that validates a card number because there would be too great a potential for false matches. It would only be helpful for including in receipts, billing statements, and for use in validating a customer identity in conjunction with other demographic information. Database tokens only provide adequate protection in environments where there is a multi-user or multi-app security model, and if there are flaws in the applications that have access to the pads, then your data is pwned.

So for the sake of maximum versatility and security, you're likely (or your software vendor is likely) to opt for hashing or encryption. But you still have a serious problem. While one-way hashes like SHA and block ciphers like AES can provide good protection to many forms of plaintext, credit cards aren't one of them. That's right, the problem isn't actually in the way you encrypt credit card numbers, it's that credit card numbers make for lousy plaintext to begin with.

Take for example the following row of data from my hypothetical e-commerce application's cardholder table:

LNAME,FNAME,CTYPE,EXP,HASH,LASTFOUR
Melson,Paul,DISCOVER,06/2009,e4b769607856a2f30b57fd26079dfefb,1111

In this case, we have what we need to use the card, except the card number is hashed with MD5. (Ignore what you know about MD5 collisions for a moment, since this problem also exists for SHA or any other method of encrypting the card number.) If we calculate the possible number of values that could be on the other side of that hash, it would be 10^16, or about 10,000 trillion for the 16-digit card number. That's roughly twice as many possibilities as an 8-character complex password (96^8), which is an acceptable keyspace size, but also completely doable for a tool like John The Ripper.

But if you know credit card numbers, then you've already realized that it's even worse than that. The first 4-6 digits of the card number are a misnomer in calculating keyspace. There aren't 1 million actual possible values. Since that row from my e-commerce app's database told me the card issuer, I know within 4-5 guesses the first two to four digits of the card number, and the last four are right there as well for inclusion on statements, etc. In this case, since it's a Discover card, we already know that the card number is 6011XXXXXXXX1111. Now we've cut the possible values we must guess in half, from 10^16 down to 10^8, which is a mere 100 million possibilities. There are other clever things we can do if it's encrypted with a stream cipher like RC4 or FISH, because we know the beginning and end values of the plaintext. But guess what? It's cheaper and easier to brute-force it even if lousy crypto is used. Even on the scale of millions of records. Even with salting, it's still worth it to brute-force the middle digits.

But wait, there's more! As if publicly known prefix values weren't enough, credit card numbers are also designed to be self-checking. That is to say, the numbers contain something like a checksum that, when a known algorithm is applied to the 7-digit account number, 3 digits of which we know from our last-four field, can be used to validate the card number. This was designed as an anti-fraud mechanism that would allow cards to be checked without a need to communicate with a clearinghouse. But this algorithm allows us to only generate valid account numbers, combined with partially-known prefixes, to reduce the keyspace significantly. And since this is a known algorithm I can (and someone already has) very easily write a tool that combines a brute-force password cracker with a credit card generator.

The bottom line is that, because of the already-partially-known nature of credit card numbers, simply encrypting card numbers inside a database or extract file is insufficient protection. The PCI Security Standards Council should revisit this requirement and modify it to, at the very least, require symmetric-key block ciphers and disallow stream ciphers and one-way hashes. But even then, I suspect, encrypted card numbers will be at risk. Certainly row-level encryption of card numbers should not qualify for "safe harbor" when it comes to breach notification laws.

PS - Extra credit if you crack the full card number from the hash above and post it below.

From The Inbox 2

2009-06-11T18:56:00.003-05:00

lmran writes:

Hi Paul,
Do you know any reason why ArcSight ESM does not support the Cisco MARS? Right now, all my firwalls send the syslog feeds into Cisco MARS and I'm trying to set the Cisco MARS to send thoes raw feeds data to ArcSight local connector but I just found out that ArcSight does not support the Cisco MARS. Thanks in ADV for any info reading this subject.

Starting in 4.x, MARS can forward events to another remote syslog listener. ArcSight has a syslog connector. So you ought to be able to forward events from MARS to ArcSight via syslog assuming MARS doesn't change the format of the log events too much. Even if MARS does mangle the event format, ArcSight will still receive them, but then most or all of the event will be parsed into the CEF Name field and categorization and prioritization won't be accurate.

If you are unable to upgrade your MARS appliance to 4.31 or later (I think that's the rev you need), another option would be to use a syslog-ng server out front. It supports forwarding events by source to other syslog servers. You could use this to send the stuff you want in ESM to ArcSight's syslog Connector and the stuff you want in MARS to MARS.

Or, you could do the environmentally conscious thing and unplug then recycle your MARS appliance. ;-)

From The Inbox

2009-06-09T12:32:00.004-05:00

Anonymous writes,

Hi Paul, I am one of those who, as you say, found your blog by googling ArcSight, trying to do some recon on the product for my employer. (I think I see that the most recent posts here are from 2007 so who knows if you or anybody will be seeing my question.) I'm trying to find out, can Arcsight's data be queried programmatically; i.e. is it stored in a relational database, hopefully SQL Server or Oracle, or if not, is there an API or ADO.NET provider that can allow it to be queried, preferably with SQL? Thanks for any info anyone reading can provide.

ArcSight ESM uses Oracle 10g for its back-end database. At one point, and this may still be true, DB2 was also supported. You can query the database directly, and the schema is pretty straightforward. The table ARC_EVENT_DATA is where most of the event data lives, for example. But depending on your use case, that might not be the best way to get data out of ESM.

Also, since you didn't specify, it may be worth mentioning that the same is not true of the ArcSight Logger platform, which is flat storage. Instead of querying the log store directly, Logger can be configured to forward events based on source, type, etc. to another destination, if you need them in real-time. There is a PostegreSQL database on Logger, but it's my understanding that it supports the reports engine, and doesn't store the raw or CEF events in any comprehensive way.

The interesting thing is that the storage technology behind Logger 3.0, because of its performance and relative "cheapness" may become the data store for ESM down the road. It would only make sense, since you could handle MUCH higher event rates with less disk and no Oracle license fee. If it can be done while maintaining the stability and feature set that the Oracle-based data store has, it's a walk-off home run for ArcSight.

New Rules

2009-06-01T22:02:00.004-05:00

After many months off, I'm jumping back in to the blog with both feet. Mostly in a Howard Beale sort of way. Didja miss me? Anyway, stealing a meme from Bill Maher, I've got something to say to security vendors. Without further ado, New Rules.

If you are a vendor, especially a vendor of security products or services, these are the rules I expect your product to follow. These are common sense, and I feel a little condescending telling them to you. But if recent experience is any indicator, you need to hear them. And you deserve the condescension.

Do not store credentials in clear text! Seriously, you can get free libraries to hash credentials or store them in a secure container file that requires a secret key. There's no reason for a password to be in a text file or HKLM Registry key. None.
Do not hardcode passwords! If I can't change every single password associated with your product simply and easily, then there should be a law that strips all of your developers of any degree they hold and forces them to go back to college and learn file IO methods.
Do not use HTTP/Telnet/FTP/LDAP for authentication! Seriously, more than enough free libraries for SSH, TLS, IPSec exist. Use one. Or buy the one you really like. It beats having to issue a "patch" to sell to government and regulated industry.
Don't run as root/SYSTEM/sa/DBA! Your product is not so special that it actually needs administrative privileges to run on the server or database that hosts it. Unless by "special" you mean "coded by lazy fools that don't want to define even the most basic security model." OK, then it is special.
Don't use broken crypto algorithms! Sorry, but if you are shipping new product that uses 56-bit DES, RC4, or ROT13, please see rule #3.
Don't send passwords in e-mail! Remote password reset is easy enough to do properly, there's no reason to be lazy and just send me my password if I forget it. Also, it means you're breaking rule #1. Busted.

There are no excuses for any product to not follow these rules, but especially security/compliance products. Gee, thanks. I just spent six figures on a product to help me manage or achieve compliance, and the product itself can't comply with the regulation I'm trying to address.

The Next Phase

2009-01-19T23:37:00.006-05:00

For those of you who haven't given up on my blog (or forgot it was still in your feed list), I want to let you know that I will be back to it later this year. More punditry, more metrics, more SIM, more cool random technical stuff. I'll try anyway. I've been missing it, but I had too much going on, had to prioritize, and this blog has rusted as a result.

A lot has changed since my last blog post in November - a new position at work, a new baby daughter - and the one thing that I've come to realize is that changing is hard work, but if you want it, it's worth it. There's been an excessive amount of talk about change this past year, and on the eve of President Obama's inauguration, I've decided to share with you this story of a moment I had recently.

On November 5th, the day after Election Day 2008, I spoke at the SecureWorld Expo conference in Detroit. I've been in West Michigan for the past several years, but I used to live and work on the East side of the state. It was a gorgeous Wednesday, clear and unseasonably warm for November. And as I was driving westbound on I-96, into the dusk between me and the sunset, I looked up and found myself in familiar territory - Webberville.

You've probably never heard of Webberville, Michigan. That's OK. It's a rural town on the automotive corridor where in the 1990's, companies got huge tax breaks to buy up farmland and build factories. And in 2001, I had an office in one of those factories. That company (a "Tier One" in industry lingo because we sold directly to car makers), like many automotive suppliers, has since gone out of business. And despite working there only a year, I have some very fond and vivid memories of that job. Perhaps the most vivid, however, is driving that stretch of I-96 between Webberville and Wixom and hearing the radio newscaster describe the second plane hitting the World Trade Center on 9/11.

That day changed everything for Americans. I was living in the Midwest, working in a one-story office that had highway on one side and cows on the other, but for the weeks that followed the attacks, I was afraid. We all were. I recall making that drive to Webberville again a week later while all of the planes were still grounded and thinking to myself, "How long until we recover? Can we recover? What will it take for us to move forward?"

Not get over it. Not forget. But move forward - take the next step as a society, as a culture, as a country.

So back to 11/5/2008, and my drive home from SecureWorld, less than 24 hours after learning that Barack Obama - a young, African-American man - would be our next president. And it was there, on that piece of highway in rural Michigan that I answered my own question. Seven years and two months later, I knew America was moving forward. We were moving forward.

Guilty Pleasures, Social Networks, and Event ID's

2007-04-04T14:45:00.000-05:00

So, one of my guilty pleasures is I like to read and answer the Information Security questions people ask on LinkedIn. It's like infosec Jeopardy without having to go to Vegas. Sometimes I even know the answer and have time to post it. The other day was one, and I'll share it here in a little more detail.

Venkatesh asks: "What are you monitoring on Active Directory/SQL Server as part of IT compliance?"

The cool thing about Microsoft EventLog format is the Event ID field, which for the most part tells you what is happening, and the details are things like who or what is doing that thing to who- or what-else. An example is Event ID Security:628. Any time you see that code, you know that A changed the password of B, and it is possible that A == B or A != B.

So get your left pinky finger ready for Ctrl-C & Ctrl-V action. Here's my big list of Security EventLog ID's that you should monitor as part of your log review processes.

EventID == Security:535
EventID == Security:624
EventID == Security:628
EventID == Security:629
EventID == Security:630
EventID == Security:631
EventID == Security:632
EventID == Security:634
EventID == Security:636
EventID == Security:637
EventID == Security:639
EventID == Security:641
EventID == Security:642
EventID == Security:644
EventID == Security:647
EventID == Security:659
EventID == Security:660
EventID == Security:671
EventID == Security:685
Target Account Name == Administrator
Target Account Name == [real admin user name] (you renamed Administrator, right?)

Here's how it looks in the ArcSight filter editor:

In our environment (200+ Windows servers, another 80-100 UNIX servers that authenticate against AD, and 1200+ Windows workstations), this represents about 70-100 events per day out of roughly a half million EventLog entries that we collect per day. That's so totally manageable. The rest of it you can subject to trending, thresholds, and so on to find weirdness worth investigating.

It's also a good idea to go through your EventLog data every couple of months and look for new Event ID's that you haven't seen before. I use a filter that matches all of the Event ID's that I've already identified and excludes them. Then it's just a matter of researching the new Event ID's and determining their cause and relevance.

If you've got other ideas of good EventLog content that you focus on, post it up here. I'd love to hear about it!

My ArcSight Toolbox

2007-04-20T09:41:00.000-05:00

I'm not shy about the fact that I use ArcSight at work, though when talking about SIM's and logging, I try not to make it all about them. But this post is all about ArcSight, but also maybe not. Maybe you use another SIM that has this same type of functionality - it wouldn't surprise me if this was standard on most SIM's shipping today.

Anyway, ArcSight has a "Tools" feature that basically allows you to pass the contents of any cell in a table view (ArcSight calls them Active Channels) to an external program. This is unbelievably handy. So here are some of my favorite ArcSight Tools.

1. Cygwin Whois - ArcSight comes with a built-in, java-based whois lookup tool. But for whatever reason, if the address is outside the US, say in an APNIC block, ArcSight just returns the NIC. Cygwin's whois will look up the registrant from the correct NIC.

2. EventID.Net Lookup - Takes a field containing EventLogType:ID ('Device Event Class ID' by default) and passes it to a shell script that launches IE with a properly f0rmatted eventid.net URL:

#!/bin/bash
PATH=$PATH:/cygdrive/c/cygwin/bin:/usr/bin:/bin
if [ "$1" = "" ];
then echo "usage: $0 [ArcSight EventLog ID Tag]";
exit 0
fi
query=`echo $1 | sed 's/$.*$:$.*$/eventid=\2\&source=\1/'`
if [ "$query" = "" ]; then echo "Error in field format";
exit 1
fi
/cygdrive/c/Program\ Files/Internet\ Explorer/IEXPLORE.EXE "http://www.eventid.net/display.asp?$query" &

3. LDAP Server/User Lookup - This is a Perl script that I wrote that takes a server or user name field and searches AD via LDAP for it and returns things like distinguishedName, operatingSystem, description, memberOf, and so on. This runs in Cygwin as well.

4. VHost Live Search - Got this idea from a post to the pen-test mailing list. Sometimes whois and nslookup don't cut it. This is a great way to figure out what vhosts might be present on a given IP address.

5. IP2Asset - On our network, workstation names and asset ID's are the same. So here's a script that takes an IP address, runs nslookup, and then launches Altiris web console to search for the asset.

#!/bin/bash PATH=$PATH:/cygdrive/c/cygwin/bin:/usr/bin:/bin
if [ "$1" = "" ]; then echo "usage: $0 [ip address]";
exit 0
fi
asset=`nslookup $1 |grep ^Name |sed 's/.*$it[0-9]*$\.wks.*/\1/'`
if [ "$asset" = "" ]; then echo "Error resolving address";
exit 1
fi
/cygdrive/c/Program\ Files/Internet\ Explorer/IEXPLORE.EXE "http://altiris_svr/Altiris/NS/Console.aspx?NameMatch='$asset'" &

GeekTool

2007-04-25T13:38:00.000-05:00

So I was playing with my Mac today, trying to make a desktop that's as useful as my KDE/SuperKaramba setup on my Kubuntu laptop. I didn't find all the stuff I was looking for, but I did find GeekTool, which is a cool little app that lets you display command-line tools or tail a log to your desktop.

Quick & Dirty JavaScript Sandbox

2007-05-07T11:19:00.000-05:00

It seems like more and more browser attacks are using obfuscated JavaScript to make analysis harder. Some things are as simple as UTF encoding, others are far more inventive and confusing. Just like packed executables before, there are legit reasons for wanting to obscure JavaScript, like making it harder for people to steal your code.

So when your IPS alerts on suspicious JavaScript (which is almost never blocked in a default configuration), you can:

A) Investigate, get a sample of the offending page and potentially spend hours trying to work back through it by hand.
B) Investigate, browse the page with your browser to see what happens, and potentially get pwned.
C) Ignore it, and hope the local AV got it.

What I have historically done, and continue to do in some cases, is option B from inside a VMWare machine.

Today, however, I ran into a higher-than-usual volume of alerts, all of which were based on the presence of an unescape() call. In anticipation of having to do this again, and the VM being a poor solution to begin with, I built a Java sandbox, starting with a JavaScript interpreter.

Here's the recipe:

1. Cygwin (optional, but you know I love it, and it makes certain things easier)
2. Current Sun JRE for Win32
3. Rhino JavaScript engine

Create an unprivileged local user who's not even a member of 'Everybody'. You're never going to log in as this user anyway. Now unpack the JRE and Rhino to a directory where that user can view them. If you have Cygwin, build a home directory for your user, and then create a bash shell shortcut with that directory in the "Start In" line. Now use RunAs to launch your shell as the unprivileged user, and start Rhino:

Now you can dump JavaScript to the shell and watch it execute with relatively low risk of pwnage. Rhino also has a GUI debugger that's ideal for stepping through more advanced JavaScript trickery.

One for the RSS aggregator: Chinese bot/sploit blogs

2007-05-08T15:45:00.000-05:00

OK, so I can't read Chinese (or Japanese, or Korean, or...) characters to save my life. But in the course of my recent adventures in obfuscated JavaScript droppers, I stumbled across something interesting. I put the first piece of some obfuscated JavaScript in Google, and I got 2 hits!

I was really hoping to find a page on this particular type of encoding and where and how it's been used in the past. Instead, I found it posted to a pair of blogs in China, with no accompanying perl scripts for decoding the payload, so I can only assume the intent of the poster(s).

ArcSight 4.0 Released

2007-05-17T12:13:00.000-05:00

As far back as a year ago there was working code that ArcSight was calling "four-point-oh." So there's no big news here other than the fact that this week ArcSight released installers and docs and all of that good stuff to their software site for general consumption. Add to that the fact that I can't find the announcement on their web site or in general press, and I figure that makes it blogworthy.

What's also blogworthy is the fact that if you're an existing customer and want to upgrade, you're stuck until August when ArcSight releases the upgrade-capable installers with SP1. Or, like with 3.5, you can pay their pro services team to do the upgrade for you before then. Anyway, I'm spoiling the feature list here:

Key features of ArcSight ESM v4.0 include :

Identity Correlation

ArcSight ESM v4.0 identity correlation can model the typical behavior of groups, machines, or individuals (as reflected in events) and provides a framework to access any other form of session data through mappings with dynamic variables. This information can be used or shown in rules, reports, active lists, active channels, and data monitors.

Improved Asset Management & Scalability

ArcSight ESM v4.0 introduces the ability to manage up to one million assets while maintaining performance, including maintaining memory usage in-line, processing, correlation, and ensuring sustained EPS (events per second).

Trend Reporting & Report Generation Performance

Trend Reporting enables the ready historical trending often required for regulatory compliance reporting. Trend reporting can track a trend over a specified period of time, and highlight changes in risks or threats during that period. Trend reporting improves report generation performance for regularly scheduled reports by tracking trends over a user-specified time and by keeping the data easily accessible.

New Report and Template Designer

ArcSight ESM v4.0 provides a new, more powerful and highly flexible reporting system. You can use this design capability to create well-defined reports for different scenarios or audiences. This feature offers options for unique queries and to define the overall look-and-feel for presenting information. These new features include the ability to report on several data queries simultaneously, using multiple charts and grids in one report. Report formats, layout, and overall look-and-feel can be customized to your needs.

Historical Correlation

ArcSight ESM v4.0 enhances the Verify Rules with Events capability (previously known as Replay with Rules) so you can define actions based on processing historical data through the correlation engine.

ArcSight Packages

ArcSight ESM v4.0 introduces a new feature called packages. A package is an ArcSight resource that acts as a portable container for group resources or content (e.g., rules, filters, data monitors, reports, etc).

Resource Validation Enhancements

ArcSight ESM v4.0 enhances resource validation beyond rule- and network-modeling, adding the ability to validate cross-resource dependencies automatically, and interactively, through the Console. This enables the ArcSight Manager to detect resource conflicts introduced during resource modification, creation, upgrading or importing.

ArcSight ESM v4.0 64-bit

The 64-bit JVM version of ArcSight ESM v4.0 will be made available as part of a controlled release. Customers who are interested in participating should contact Technical Support for additional information.

Like an Orange on a Toothpick

2007-05-24T13:00:00.000-05:00

My ego's going to have to do some sit-ups this weekend. It's getting huge. I'm on CNN via the WOODTV WiFi Arrest story.

Supastah!

SIM Sizing - CPU (+ Performance Tuning)

2007-06-05T09:20:00.001-05:00

There are a couple of places across a traditional SIM that are more susceptible to performance degradation than others. Here's the short list:

The Database
There basically two things that can beat up your database. The first is basically drowning it with INSERT's of new log data. While this will manifest as poor performance and high CPU utilization, the problem is most likely disk array write performance. The answer to that problem is most likely expensive. Sorry.

The second thing that can kill database performance is event searches. This can happen in reporting or table views or pattern discovery or even in charts and graphs. (It can also happen in correlation rule filters - more on that below.) Think of it this way; whatever means you are using to search events, especially historic events (double-especially if there's compression in the mix here) has to be translated in to some horrid, fugly SELECT statement, probably with multiple JOIN's. Use these in rules, graphs, or regularly scheduled reports and you can drown your database server in work to the point that the stuff you're actively doing is unusably slow. The answer here is a combination of giving lots of CPU to your database servers and writing smart search/filter statements.

Correlation Rules
Correlation rules are the heart & soul of SIM technology. For more on what they do, check out my old ISSA 'Intro to SIM' preso deck (PDF Link). There are a number of things that can screw you here, and I already mentioned the first one above. Writing filters that are too complex or simple filters that are too vague will come back to haunt you.

Like an IDS, you will need to tune the correlation rules that your SIM ships with. A lot of this will be about eliminating noise and false positives, just like IDS. But also like IDS, some of the tuning will be performance-related. In addition to the filters you write you will also want to think about things like the number of events to match on, time frame (how long to wait for event2 after event1 occurs), etc. A cool thing that ArcSight includes is a real-time graph partial rule matches. In the example below, you can see there are two rules there that need tweaking and will probably free up measurable memory and CPU cycles once I do.

One last tip on rules and performance: If your rule creates a new meta-event, make certain that the new event does not match its own correlation filter. Trust me on this one. It's worth the extra time to double-check before turning it on.

Log Agents/Parsers/Handlers
The final place where CPU load can grow quickly is your log collection points. Somewhere between the log source and the database is code that your SIM uses to convert the log from its original format to a standard format for insertion into the database. The frequency with which log entries hit this code can have an impact on performance. This is where all of the regex matching, sorting, asset category searching, severity calculation, and so on occurs. For well-defined log formats and sources (like firewalls), this tends not to be that intense a process since you have very little diversity to be handled. But for UNIX servers with a variety of services running, there is the potential for serious friction as these parsers try and figure out what the actual log source is and what the message means.

If you have something like a UNIX a server farm that generates thousands of events per second and you want to push it through your SIM, you will need to spread this load out or buy big hardware to handle it.

Malware Season Pt. 1

2007-06-20T14:35:00.000-05:00

Like I said last week, I'm going to write up my experience from last week with taking some "0h-day" malware (read: undetected by IDS or AV) from log finding back to analysis of the dropper and binary. This is a 2-parter, the first part covers from discovery through the dropper to getting a copy of the binary.

If you're aware of what's been going on in the malware arena for the past few years, and has visibly worsened over the last 6-9mos, then you know that you can't rely on your AV vendor to catch it all. (Remember all of that 'defense-in-depth' stuff from the Information Assurance "awakening" 4-5 years ago? Yeah, this is where it should be saving your bacon.) So one thing I've taken to doing is looking at firewall logs for outbound web requests that end in ".exe". I found one that was a "http://IP:PORT/bin.exe" It would be nice if the FQDN were captured here, but it's not. And as such, that file is just out of reach.

Using ArcSight, it was pretty easy to give that download some context by pulling up all of the Internet traffic to and from that workstation for 30 minutes before and 30 minutes after. After digging around, I found one of the web sites the user had visited that contained a single line of JavaScript on each page that contained an unescape() call that looked glaringly suspicious.

In addition to the unescape call, there's also another function named dF() that has a whole mess of obfuscated code in it. The first step is to find out what the unescape function will actually do. I used Rhino JavaScript shell:

The unescape creates a second set of script tags and defines the dF function. Now I can define dF in Rhino's shell and see what that does:

Oops. I needed to replace document.write with print in order to actually see output of the dF function:

There, that should work:

Now I've got a human-readable VBScript dropper with a URL. Grabbing the binary malware at this point is as easy as using wget. Despite the fact that the malware URL can easily be read, there's still obfuscation at work here. Check out the a1,a2,a3,a4 variables and then str1 = a1&a2&a3&a4 used to hide the string "Adodb.Stream." I wonder why. Like I said, both the JavaScript payload and the binary went undetected by the workstation's antivirus. Our IDS fired on both, but they were fairly generic detects - one for the JavaScript unescape function and another for the packed executable download.

Tomorrow I'll write up the basic analysis of the binary that I did along with some info on the type of external follow-up.

Malware Season Pt. 2

2007-06-20T15:18:00.000-05:00

When we last left our heroes, they had de-obfuscated some JavaScript and downloaded a malware binary file named 'bin.exe'.

As you might have guessed, this binary was packed in order to make detecting its contents more difficult. I ran it through PEiD to determine what packer was used:

At this point, I don't even bother trying to unpack it. Instead, I try to load it in the GenOEP and ScanEP PEiD plugins and then I try to open it in OllyDbg. They all fail. Now I start to fear that I'm doomed to repeat past frustrations. But, what the hell, I'll try and unpack it anyway:

That was lucky. It can't be this easy. You may have noticed that the file in that screen shot is svhost32.exe, not bin.exe. This is because I was playing with it, trying to get it to run in SysAnalyzer. Since the VBScript dropper saves the file as svhost32.exe, I thought that might be worth a try. Anyway, to make sure there aren't more layers of packing going on here, I take another whack at it with PEiD:

I didn't see that coming, but I'm not looking a gift horse in the mouth. So now we should be able to do stuff like launch it in SysAnalyzer or OllyDbg. Sure enough, it runs from SysAnalyzer and we get the goodies:

Once it's running with SysAnalyzer, we can get the scoop. It uses AutoItv3 to download itself again as svhost.exe, modifies a mess of registry keys to run at start up as well as hijack Explorer and IE startup pages, presumably to drive up ad hits for the distributor.

A quick Google search, and we have a name for it: Sohanad. So it's not new malware, really, just slightly modified from the original so as to get by more AV scanners. I wonder how many:

The packed executable that we downloaded is detected by 13/31 AV products used by VirusTotal. Just for kicks, what happens if we try the unpacked file from earlier:

Ugh. Only 5/31 detect it now, when it's not obfuscated. The irony is overwhelming. Quick, somebody in the AV R&D field write a paper on using un-obfuscated code as a means of bypassing AV detection. This is hot!

Lastly, I contacted McAfee for an EXTRA.DAT file for both the packed and unpacked binaries and notified SANS ISC of the hacked web site with the dropper as well as the site hosting the binary.

I'd like to say, if you do run into sites hosting malware, the handlers at ISC are a great resource for coordinating response and clean-up. In this case, the hacked site was cleaned up and the malicious site was taken down within a day of my contacting ISC. They contacted the responsible parties and got it done. Doing this by yourself is hard and annoying work, and I am grateful to the ISC folks that they're willing to let us offload this stuff to them. So when you're out and about at conferences this summer and you see any of the ISC handlers, remember to thank them and maybe buy them a beer or something.

ISN Funny

2007-06-26T12:20:00.001-05:00

I <3 The Onion. But I especially love it when they show up mixed in with the "real news."

http://www.infosecnews.org/pipermail/isn/2007-June/014889.html

Little Stuff

2007-07-02T12:29:00.000-05:00

So in between playing whack-a-mole with ecard.exe urls and trying to figure out which BlackHat talk you're going to now that The Brothers Kumar have backed out of their talk on bypassing TPM (BTW, these guys are kinda tipping the shady-meter, no?), you're looking for something interesting to read.

Try http://www.securls.com/

And since I mentioned ecard, here's a write-up on some other oft-repacked malware that won't go away. Lyberty Miller does a nice job of pointing out practical countermeasures, something researchers don't always do.

Also, more ecard. Guess what I'm neck-deep in today. It's all new since the weekend!

Welcome Back!

2007-07-05T10:31:00.000-05:00

Welcome back from the holiday! Pffft. I enjoyed the day off, but I'm starting to dread coming back to work after holidays. New Years. Superbowl Sunday. Lately, any random weekend. And now Independence Day.

So for those keeping score, we're now on variant 4 of ecard.exe, all new for July 4th! And look, with the new version, no two binaries are exactly the same: