Review: Chuvakin's Database Log Analysis Paper

Dr. Anton Chuvakin was kind enough to share with me a copy of his 2-part paper on database log analysis, which was published in CSI Alert Newsletter, Feb 2007. I enjoyed his paper(s), but I have to acknowledge that this is a topic near and dear to my current work. In Part I of his paper, Dr. Chuvakin gets right to the heart of database log analysis: You should do database logging better than your database does by default because it's probably a compliance issue for you.

Chuvakin also explores some of the pitfalls of database logging:

• Performance hits from turning on auditing
  
• A databases stores its different log data in multiple formats (in a table, in a redo file, in a text file, etc.)
• Finding data in database logs that is meaningful to security ops and incident response

"Introduction to Database Log Analysis, Part II" lays out some practical considerations for any organization preparing to implement database logging. It's hard to keep this topic general, since the differences in audit logging between Oracle and Microsoft SQL Server, for example, are significant enough that the strategies for tackling log gathering, storage, and analysis are very different.

As you might expect from a guy in his line of work, Chuvakin goes on to describe why you might want to use a SIM or similar tool to aggregate and read database logs. I happen to agree with him that you need something more than LogMiner if you want to use your logs for anything more than troubleshooting. But Chuvakin gives a list of example reasons for using a SIM for compliance and operations purposes around database logging, not all of which wash with me:

Change management: modern RDBMS is a complicated piece of software with plenty of configuration files and other things to change. And being aware of all the changes constitutes one of the control objectives for IT governance as well as essential for protecting the environment.

True that COBIT and ITIL both rely on change management and that it is a common compliance requirement for things like SOX and SAS. However, I don't know that a SIM is a good candidate for this. Operationally, I use our change control documentation to explain, for instance, why a bunch of files were changed or new users created. I don't use the SIM's ability to detect these events as the impetus for documenting the change - the cart's way out ahead of the horse by the time the SIM gets to it, and if you do it right, change docs always precede the change. So, at best, this is a secondary control that the SIM is providing. Not that it's a bad idea, it's just not a business driver for log analysis.

Authentication, Authorization and Access: logging access control decision such as login failures and successes as well as access to data and various database objects is of interest for auditors as well as important for security, such as insider privilege abuse. While logging all access to data is less common, it is one of the emerging trends in logging.

Good idea. Do this with your directory and server OS, too. And RADIUS. And client VPN. And prox cards. And voicemail. And... oh, you get the idea.

Threat Detection: while both unauthorized changes and access control decisions are essential for security, database logs may be used to discover and analyze direct exploitation attempts as well, at least in some cases. Even accessing the database system at an unusual time will likely be of interest to as security team.

Log analysis is probably your best option here, unless you stick an IDS sensor between your database servers and everything else on your network. Which if your database supports a web app, you should definitely do.

Performance: DBAs are tasked with monitoring database performance and logs provide one of the avenues for doing so. This is especially important to those orgs with strict service level agreements (SLA) for database performance

While a SIM may be able to create averages and other statistics in near-real time, this is not the tool for database performance monitoring. There's lots of stuff out there that does this already and much better than your SIM can.

Business Continuity: knowing of database software starts and stops is essential since business depend on databases for their revenue streams and this a downed database directly leads to losing money

Again, lots of stuff that can do this better than a SIM. Moreover, SIM's use data analysis instead of active checks. If your database shuts down, a SIM can tell you about it once it sees a log entry for it. If it hangs, then maybe it will detect a drop in activity and determine that the event per second delta is abnormal. Of course, this can occur during normal use. IMHO, SIMs suck for tracking service up/down states.

Overall, I enjoyed reading this paper. It's clear that Dr. Chuvakin has some good ideas on this topic, and I would recommend that anyone who is considering database log auditing read this paper prior to the initial design meeting. It's not a comprehensive guide, but it will get you thinking in the right direction about the issues your organization will likely face.