Logging

Standard logging mechanism

All of the dæmons in the Internet Utilities write their log output to standard error. They thus leave the disposition of those data up to the user. This provides the user with the choice of what to do with the log output. One can:

• close standard error, resulting in the log output going nowhere;
• redirect standard error to a file, resulting in the log output going to that file (which will grow indefinitely); or
• redirect standard error to a pipe connected to a logger process, such as the CYCLOG command from the OS/2 Command Line Utilities version 2.2.

Most commands prepend each line of their log output with a timestamp. This timestamp is in a machine-parseable, compact, and timezone-independent form which can be converted into a human-readable time using the UNSTAMP command from the OS/2 Command Line Utilities version 2.2.

Avoid syslog

The Internet Utilities provides a SYSLOGD command that proides a "syslog" UDP service.

Avoid using the "syslog" UDP service if at all possible. It is there for compatibility with badly written programs.

Some programs write their log output by sending it encapsulated in UDP datagrams to a "syslog" dæmon. This design is a poor one, for several reasons:

• It performs fan-in, only to require fan-out further on.

  All of the streams of logging output from all of the individual processes on the system are merged into one single stream, with the log entries from different processes interleaved arbitrarily with one another. In order to separate the logs, postprocessing must be done on the resultant data. (Unix "syslogd" dæmons perform this postprocessing themselves. IBM's SYSLOGD does not provide mechanisms to even perform such postprocessing.)

  A better design would involve not merging the separate streams of log data in the first place.

• Voluble programs can swamp, or even completely expel, the output of quieter programs.

  Consider the case where two processes are writing to the log. One generates a large volume of mostly trivial log entries - thousands per day, let us say. The other generates a couple of very important entries per week. Because the "syslog" approach combines all of the streams of logging output into one big single stream, the important log entries from the quieter process are lost in amongst the noise generated by the more voluble process.

  Worse, if the log files are being rotated in order to cap the amount of disc space that they consume (Unix "syslogd" dæmons do this. IBM's SYSLOGD provides no mechanism to do so, however.), the unimportant log output of the voluble process can cause the important log entries of the quieter process to "scroll off" and be lost.

  A better design would ensure that each stream of log data is separate, and not obscured by being interleaved with others. It would also ensure that log entries in one stream are not purged because of high levels of logging activity in other streams, and (furthermore) allow for having different degrees of data retention for different streams.

• The UDP protocol is, by design, an unreliable transport.

  UDP doesn't guarantee that log entries won't be discarded. The UDP/IP stack is free to throw away UDP datagrams for various reasons — such as if it runs out of buffers to hold the datagrams, for example. There is no retry and no acknowledgement. There is no way to determine what log entries are lost.

  A better design would use a reliable transport mechanism.

• The scheme requires at least one (loopback) IP address to be available.

  Because the service uses UDP, programs that use the "syslog" service require that the UDP/IP stack be installed and loaded, and that there be at least one (loopback) IP address in existence, before logging can occur.

  A better design would not require that IP networking be installed and running in order for it to work.

Fortunately, there is a far better mechanism for logging output. It is called "standard error". This is a very familiar concept that exists on most operating systems. Unix has it. (In fact, error and event logging is one of the purposes for which standard error exists on Unix in the first place.) MS/PC/DR-DOS has it. Even Win32 has it (just barely). IBM OS/2 most definitely has it.

All of the programs in the Internet Utilities use this simple mechanism. None of the programs in the Internet Utilities require the use of the SYSLOGD command.

Some OS/2 programs that use "syslog" also have an option of sending their log output to their standard error instead of to a "syslog" UDP service. It is strongly recommended that this option be enabled in those programs that provide it.

However, some badly written softwares that log to "syslog" cannot be persuaded not to do so. This disease is endemic in Unix softwares — alas! — despite standard error being a well-known Unix mechanism, and so is endemic in ports of those softwares to OS/2. For such programs, the SYSLOGD command provides a remedy, allowing one to turn the stream of UDP datagrams into output to standard error (of the SYSLOGD process) that can then be redirected as desired using the normal file handle redirection mechanisms.

This remedy is necessarily imperfect, since it can do nothing about the potential for loss of log entries or the merging of all of the streams of log information.