Variable Severity Logging 1

Posted by F 24/09/2006 at 19h21

Most software products and logging frameworks produce entries in their log files with different severity levels (say, from Debug to Severe). Then, you can configure which level of messages you want the application to log.

At development time, you may want all messages to be logged. But once in production, you normally configure things so that only Warning and higher messages get logged (for performance, space and other reasons).

The problem is that when a Severe message arises you may need more detailed (Debug level) information in order to understand the actual cause of the problem. So, a common practice is to increase the logging level at that point, and wait for the problem to happen again –not very effective.

This is one of the problems that motivated the guys behind LogBag. The idea they propose is nice and simple:

Make the system write only Warning (and higher) messages to the log, but when such a message is logged, it should also include some lower-severity messages that occurred right before and after this one.

It’s one of those good ideas that might seem obvious, but for some reason nobody thought (or did anything) about before.

To implement this, the logging system could keep a buffer of the last N messages generated by the application, and when a Warning (or above) message comes in, the whole buffer is flushed to disk. This should not be too hard to implement, since most logging frameworks already use some kind of buffering, in a producer-consumer pattern, to improve concurrency and performance.

From the LogBag site:

You keep on putting things in a Log bag. Then whether the entire log bag gets written or not depends on the highest severity level in the entire bag. So either you write everything or you write nothing.

I think it’s a nice idea. I bet the technical support guys would like it too :-).