Variable Severity Logging 1

Posted by F 24/09/2006 at 19h21

Most software products and logging frameworks produce entries in their log files with different severity levels (say, from Debug to Severe). Then, you can configure which level of messages you want the application to log.

At development time, you may want all messages to be logged. But once in production, you normally configure things so that only Warning and higher messages get logged (for performance, space and other reasons).

The problem is that when a Severe message arises you may need more detailed (Debug level) information in order to understand the actual cause of the problem. So, a common practice is to increase the logging level at that point, and wait for the problem to happen again –not very effective.

This is one of the problems that motivated the guys behind LogBag. The idea they propose is nice and simple:

Make the system write only Warning (and higher) messages to the log, but when such a message is logged, it should also include some lower-severity messages that occurred right before and after this one.

It’s one of those good ideas that might seem obvious, but for some reason nobody thought (or did anything) about before.

To implement this, the logging system could keep a buffer of the last N messages generated by the application, and when a Warning (or above) message comes in, the whole buffer is flushed to disk. This should not be too hard to implement, since most logging frameworks already use some kind of buffering, in a producer-consumer pattern, to improve concurrency and performance.

From the LogBag site:

You keep on putting things in a Log bag. Then whether the entire log bag gets written or not depends on the highest severity level in the entire bag. So either you write everything or you write nothing.

I think it’s a nice idea. I bet the technical support guys would like it too :-).

Graphical Thread Dumps 10

Posted by F 24/11/2005 at 23h00

I am surprised by the high number of Java developers I meet that do not know what a Java Thread Dump is or how to generate one. I find it a very powerful tool, and it is always available as part of the JVM. I haven’t played much with Java 5 yet, but it comes with jstack, a new tool that makes it easier to generate thread dumps.

Earlier this year, I was working on a load test for for a well-known airline. We were tunning the environment all we could, monitoring and profiling to know where to focus our optimization efforts. The solution involved a fairly high stack: Apache httpd, WebSphere, FuegoBPM, Tibco messaging, Oracle RAC.

The system was holding load pretty well up to a certain point in which it immediatly halted and stopped processing new requests. Every time we run the load testing scripts we experienced the same symptoms. Not even the official testers –with allegedly powerful testing and monitoring tools– were able to identify the cause of the problem.

So, I decided to get a few Thread Dumps of WebSphere’s JVM. On Unix, you do ”kill -3 <pid>” and the dump goes to WebSphere’s native_stdout.log. We inspected the dumps but couldn’t identify dead-locks or any other obvious anomaly, although the answer was right before our eyes.

Micro and Blind Optimizations 1

Posted by F 03/11/2005 at 22h16

Yesterday, a good friend of mine and ex-coworker contacted to me to share his frustration.

(he hates to be called “Polino”, so I won’t.. doh!)

He finished a software solution for a customer, and now an expert is reviewing his Java code.

The expert code reviewer insists on small performance optimizations, but he is way off target. He wants to micro-optimize, and to do it blindly.

For example, he reported that the following code was doing “inefficient String concatenations”:

String myString = "Some text here "+
                  "Some text there "+
                  "Some more... ";

And that this was an “inefficient way of creating Longs”:

 myList.add(new Long(1));

These examples are probably well optimized by modern Java compilers. But even if they weren’t, they probably don’t affect much to the performance of the system as a whole.