108.2 System Logging
Candidates should be able to configure the syslog daemon. This objective also includes configuring the logging daemon to send log output to a central log server or accept log output as a central log server.
Key Knowledge Areas
- syslog configuration files
- standard facilities, priorities and actions
The /var/log/ directory
Many services report their activities by writing messages to log files. Some services such as Apache, Samba and the X server write their own log files directly. Many others do their logging through a logging service called syslog. This service is provided through two daemons – syslogd handles application logs and klogd handles kernel logs. The advantage of using syslog is that it provides a central place to control where the logged messages will go.
Most log files are in the directory /var/log. The exact set of files that you will find here varies between Linux distributions but those that you will commonly find include:
cronkeeps track of messages generated when cron executes
mailmessages relating to mail
messages logs all messages except private authentication authpriv, cron, mail and news
securelogs all failed authentications, users added/deleted etc.
The most important log file is messages where most activities are logged.
The /etc/syslog.conf file
When syslogd is started it reads the configuration file /etc/syslog.conf. (It is possible to start syslogd with the -f option to specify an alternative config file.) This file contains a list of rules. The first part of each rule identifies which messages the rule applies to and the second part of each rule specifies an action (what to do with the message).
Syslog can send messages to:
- A file
- A logged-in user
- Another syslog running on a remote machine
Figure 108.2-1: Format of rules in /etc/syslog.conf
Messages are identified using a facility and a priority level. The facility indicates approximately where the message came from and the priority indicates how important the message is.
The available facilities and priorities are shown in the figure. They are predefined and cannot be extended; however note that facilities local0 to local7 are provided for general use.
Valid priorities are: (from highest to lowest)
In addition, the wild-card '*' can be used to mean “all facilities” or “all priorities”.
Consider the following four example lines in syslog.conf. (These are not meant to represent a working configuration but simply to provide examples of the syntax. Also the line numbers are for reference, they are not part of the file):
Line 1 matches messages from the cron facility at the notice priority or higher. These messages are appended to /var/log/cron
Line 2 matches all messages except those from the authpriv facility (note the use of 'none' here).
Line 3 matches all messages from the mail facility. These messages are appended to /var/log/mail. The '~' in front of the file name suppresses syslog's normal behaviour which is to flush the output file after every message is written. Flushing degrades I/O performance and is probably not appropriate for low-priority, high-volume messages such as a mailer daemon might generate.
Line 4 matches all messages at level crit (or above) and writes them to all logged in users. (On a server this is unlikely to be an effective way of getting anyone's attention!)
Line 5 shows how messages can be forwarded to another machine (“neptune” in the example).
Line 6 shows the use of '=' to match just a single priority level.
Note that a single message can match more than one rule, in which case, more than one action will be taken.
Listing of /etc/syslog.conf
| # Log all kernel messages to the console.
# Logging much else clutters up the screen.
# kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
Note that if the syslog.conf file is changed, the syslogd daemon should be signalled to re-read it:
# pkill -HUP syslogd
Using a central logging server
On large networks it may be useful to forward messages to a secure central server. This has two advantages. First, it is easier to conduct log file analysis if the messages are all in one place. Second, it makes it much harder for an intruder to “cover his tracks” by doctoring the log files on a compromised machine. If the messages have also been forwarded to another machine the intruder will not be able to delete them unless he is able to compromise that machine also.
To do this, on the “upstream” machines, in the syslog.conf file simply specify an action of the form “@10.1.1.254” or “@loghost” for those messages that you want to forward. (“loghost” must be a resolvable name for your central logging server). On the “downstream” machine, you must start syslogd with the '-r' flag to tell it to listed (on UDP port 514) for forwarded messages. This is not the default, so you may need to modify the start-up script (probably /etc/init.d/syslog) to add this flag.
The logger command
The command-line utility logger may be used to send messages to syslogd. It can be used interactively (usually to test your syslog configuration), or it can be used within a shell script.
# logger "invalid request from client"
By default, logger sends messages with a priority of user.notice and on a typical system these messages will end up in /var/log/messages:
# tail -1 /var/log/messages
Sep 15 07:07:04 neptune chris: invalid request from client
In this message, "neptune" is the host name and "chris" is the user name.
The logger utility logs messages with a priority of user.notice by default. You can use the '-p' option to specify a different priority. In the example below, we log with a priority of local4.notice. The local4 facility is one of eight (local0 to local7) defined for general use. These can be used to create your own log files. (Though note that RedHat uses local7 to log boot-time information in /var/log/boot.log).
First, we add the following line to /etc/syslog.conf:
Restart the syslogd or force it to re-read its' configuration file as follows:
|#pkill -HUP syslogd|
Now, messages from the local4 facility will be written to the terminal /dev/tty1:
#logger -p local4.notice "This script is writing to /dev/tty1"
The following is a partial list of the used files, terms and utilities: