SIM Sizing - RAM

Sorry for the pregnant pause on this topic. We went camping this weekend and my brain, well, rebooted. Good for me and my blood pressure. Bad for anyone waiting for the final episode of this gripping trilogy.

Anyhoo, there are basically two places you need to plan for RAM expansion in your SIM. Again, the database. Not much of a surprise. The two drivers here are the log frequency as this will cause the database to store INSERTs into RAM while waiting for disk. This problem will be obvious as your performance will suffer noticeably. The other thing that can cause the amount of RAM your database is using to increase are standard things like the number of simultaneous users or the number of simultaneous reports that run. Also reports or table views (ArcSight calls these 'active channels') that look back through weeks or months worth of events will definitely burn RAM on the database server.

The other component that can run into RAM problems is the point on your SIM where correlation rules run. The reason correlation rules eat memory is because they are based on matching 2 or more events over a window of time. This is not unlike the state tracking table on your firewall.

Let's say you have a rule that looks for a firewall allow message and an IDS alert that have the same source IP address. Then let's say that that rule has a 2 minute window because you have time sync issues. Your SIM is going to track every firewall allow message and every IDS alert that match your filter for 2 minutes, comparing each new IDS event to each firewall event in the table as well as comparing each new firewall event to each IDS event in the table. Now imagine someone fat-fingers that rule or you have really bad time sync issues and you set that window to 20 minutes. I think you get the idea.

The solution to keeping this from spiraling out of control is simply:

1) Write correlation rule filters as simply and precisely as possible.
2) NTP!@ (If you haven't already. This is important for your SIM to function well for so many reasons.)
3) Since you have time sync figured out now, use small windows in your correlation rules.