ArcSight Logger: CEF vs. Raw

Here's something for potential ArcSight Logger customers to ponder. The issue is whether you should use CEF formatted logs (post-Connector) or raw logs (pre-Connector) or both in your Logger environment. In this case, a picture is worth at least a few hundred words:



If you look carefully at that image, you can see that it shows the same event in both its raw syslog format and it's Connector-ized CEF format. From my point of view, it boils down to use case. Analysis versus troubleshooting. Reporting versus response. The CEF formatted message is chock-full of metadata-and-labeling goodness. It's also overkill on the eyes. Log messages are already cryptic to the point of questionable usefulness. CEF amplifies that. The raw format, on the other hand, is easier to read due largely to the fact that it's what your UNIX admins are used to seeing. But that's where the positives end. Raw syslog is all but unformatted and trying to write a small chain of regexes that do a good job of parsing large quantities of syslog is a headache and a half.

Of course, you may have already realized that there is a right answer to this problem: Do both. Sure there's some overhead to consider, since you're going to pass syslog to a Connector that will then send raw events to Logger, CEF events to Logger, and CEF events to ESM if you have it. Or you could send raw syslog to Logger, have Logger forward it to a Connector and then configure the Connector to send CEF to Logger and ESM. There are probably many other complicated flows that you could implement as well, but you get the idea.