Tuesday, February 03, 2009

Logging in Your System

Logging is one of the more controversial aspects of a large system.  There are some who hold that you should not need to have log file - everything should be presented in some UI or other.  Others take it to the other extreme, logging every action at almost the code line level.

Not surprisingly, the answer lies in between.  You need to log enough to be able to track what is going on, and save enough info about the error conditions to make it clear what went on, but you don't want to flood your logfiles with chaff.

One common thing is to make your logging dynamic - you can adjust it on the fly, so that if an error is recurring, you can turn it up to get more detail.  For this to work, you need to define various levels of logging to present, and have a means to store the current level of presentation.  Another facet of logging is the files the logs are written to.  UNIX system have the syslog() facility, which is good for serious issues - ones that may shut down the system, or cause complete failure, like full disks, etc.
But it can be a pain to fill up a system file with the minutae of your most verbose debug level, so the best is probably to route logs to a file under your control.  Of course, it goes without saying that you should have a file aging system in place to keep the file from growing too large, and having too many old ones.

There is usually a debate about how logging levels should be managed - as increasing levels, or as bit flags to cover areas like disk I/O, database access, RPC communications, etc.  It's something of a judgment call, but make a set of categories and enforce it across all processes in the system, or you'll have chaos.

If you go with levels, you'll have the question of how many levels.  There are a number of logging systems that use levels, and if you are using a language with such a facility, use those levels.  If not, look to one of them for suggestions.

Have a command-line utility to adjust logging levels, so that if you have to go in remotely you can change things - there's nothing worse than having to wake up a site staffer just to navigate to a single GUI and click one button.

Technorati Tags --
, , ,

No comments: