Sunday, June 1, 2008

From My Inbox... (ArcSight Connectors & Logger)

SC left a comment on an earlier post on ArcSight Logger and CEF vs. Raw formats...

Hi Paul,

Do you know if its possible to "insert" logs into the logger or SmartConnector if the logs are on a physical storage, e.g. DVD or external storage?

Thanks.

Kind Regards,
SC

There are probably a number of ways to do this, but I've only tested one. In earlier configurations of our syslog infrastructure, there were a couple single points of failure. In order to meet log analysis commitments, we would reload lost syslog data from file.

Start by configuring a 'Syslog Pipe' Connector. Since the connector only has to be online with the Manager when you're manually inserting logs from file, you have greater flexibility about where this Connector will live. It lived on my laptop for a while. When you set it up, point to a path that isn't already used for anything else. Then you can simply start the Connector and pipe the raw log file(s) to the named pipe:

# cat oldsyslogs.txt >> /var/spool/my_arcsight_pipe

Depending on how far the date/time stamps of the events in those files are from $Now, ArcSight will probably throw some errors. It will maintain the "End Time" from the raw log events, and apply "Manager Receipt Time" as the time the manager collects the parsed events from the Connector. This will absolutely screw up any correlation rules you wanted these events to be subject to. Sorry, no easy way around that.
at

3 comments:

  1. AnonymousJune 19, 2008 at 9:28 AM

    Hi Paul,

    Thanks for your advice... Just checking if you have tried the FTP functions on the logger.

    I wanna use it but it keeps telling me connection failed.

    Do you have any idea? Thanks again.

    SC

    ReplyDelete
  2. PaulMJune 19, 2008 at 2:19 PM

    Not really. We looked at using the SCP file transfer event receiver when we did the 2.0 beta, but it didn't work for our needs.

    For one thing, there have been problems with it handling variations in initiating connections. It's really just an interface to cron and scp or ftpget using expect scripts. For another thing, you can't schedule the transfers more often than hourly (which is not a limitation of cron, I might add). These two factors make it a poor choice for retrieving logs on servers where we can't run a Connector, which means that you're stuck looking elsewhere. For the time being, anyway. We submitted bug fixes and a feature requests around this issue last fall. Maybe in 3.0?

    ReplyDelete
  3. AnonymousJuly 6, 2010 at 7:33 PM

    Hi Paul,

    Seeking your advice on what could potentially be the problem.

    I have a connector as the source IP is generating lots of denied traffic targetting public IP address. The traffic is through port 137/udp.

    Had ruled out the possibility of infection, but the TCP/IP netbios is set to default. Could this be due to it trying to do lookup, or the computer browser service?

    Thanks for your time and advice.

    Kind Regards,
    SC

    ReplyDelete