Introduction
The Apache web server can be configured to give the server administrator important information about how it is functioning and what issues, if any, need to be addressed.
The main avenue for providing feedback to the administrator is through the use of log files. Apache has a very configurable logging mechanism that can be used to output messages to different places based on instructions.
In this guide, we will look at how to utilize Apache‘s logging functionality to set up structured, easy-to-parse logs. We will be using a default Apache2 installation on an Ubuntu 12.04 VPS. Other distributions should operate in a similar fashion.
Apache Log Levels
Apache separates all informational messages into categories depending on how important it considers the information.
For instance, for the most important messages, considered emergencies, Apache designates the log level as "emerg". The "info" tag, on the other hand, just shows helpful information that can be useful to look at occasionally.
Here are the log levels that Apache recognizes, from most important to least:
- emerg: Emergency situations where the system is in an unusable state.
- alert: Severe situation where action is needed promptly.
- crit: Important problems that need to be addressed.
- error: An Error has occurred. Something was unsuccessful.
- warn: Something out of the ordinary happened, but not a cause for concern.
- notice: Something normal, but worth noting has happened.
- info: An informational message that might be nice to know.
- debug: Debugging information that can be useful to pinpoint where a problem is occurring.
- trace[1-8]: Tracing information of various levels of verbosity that produces a large amount of information.
When you specify a log level, you are not choosing to log the messages labeled in that category, you are choosing the least important level that you wish to log.
This means that any levels above the selected level are also logged. For example, if you choose the "warn" log level, messages tagged with warn, error, crit, alert, and emerg will all be logged.
We specify the level of logging desired with the "LogLevel" directive. We can see the default log level in the default configuration file:
sudo nano /etc/apache2/apache2.conf
. . . LogLevel warn . . .
As you can see, by default, we have Apache configured to log messages with a priority of "warn" and above. We will learn where Apache logs its messages in the following section.
Where does Apache Keep Its Logs?
Apache can be told where to place its logs using server-wide logging specifications. You can also configure logging individually for each separate virtual host.
Server-Wide Logging
To find out where the server logs information by default, we can open the default configuration file. On Ubuntu, this is "/etc/apache2/apache2.conf":
sudo nano /etc/apache2/apache2.conf
If we search the file, we can find a line that looks like this:
ErrorLog ${APACHE_LOG_DIR}/error.log
This directive names the file where Apache will keep its error messages. As you can see, it utilizes an environmental variable called "APACHE_LOG_DIR" to get the prefix of the directory path.
We can find out what the "APACHE_LOG_DIR" is set to by examining a different file, the aptly-named "envvars" file:
sudo nano /etc/apache2/envvars
. . . export APACHE_LOG_DIR=/var/log/apache2$SUFFIX . . .
We can see in this line, that the "APACHE_LOG_DIR" variable is set to the directory "/var/log/apache2". This means that when combined with the directive in the "apache2.conf" file, Apache will log into a file called "/var/log/apache2/error.log":
sudo ls /var/log/apache2
access.log error.log other_vhosts_access.log
We can see the error.log file and some other log files as well.
Virtual Host Logging
The "access.log" file at the end of the previous section is not configured in the "apache2.conf" file. Instead, the package maintainer decided to place the directive specifying its use within a virtual host definition.
Examine the default virtual host definition to see the access log declaration:
sudo nano /etc/apache2/sites-available/default
If we scroll through the file, we will see three separate values concerning logging:
. . .
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
. . .
The ErrorLog definition matches the one in the default configuration file. It is not necessary to have that line in both places, but it does not hurt to be specific incase you change the location in one place or another.
Defining Custom Logs
In the previous section, the line describing the "access.log" file uses a different directive than the preceding log lines. It uses "CustomLog" to specify the access.log location:
CustomLog ${APACHE_LOG_DIR}/access.log combined
This directive takes the following syntax:
CustomLog log_location log_format
The log format in this example is "combined". This is not an internal Apache specification. Instead, this is a label for a custom format that is defined in the default configuration file.
If we open the default config file again, we can see the line that defines the "combined" log format:
sudo nano /etc/apache2/apache2.conf
. . .
LogFormat "%h %l %u %t \"%r\" %>s %O \"{Referer}i\" \"%{User-Agent}i\"" combined
. . .
The "LogFormat" command defines a custom format for logs that can be called using the "CustomLog" directive as we saw in the virtual host definition.
This log format specifies a format known as a "combined" format. Learn more about available format string variables by going here.
As you can see, there are several other common formats that have been created for use within your virtual host definitions. You can uses them exactly like you see the CustomLog declaration earlier. You can also create your own custom log formats.
Rotating Apache Log Files
Because Apache is capable of logging a large quantity of information during the process of handling requests from clients, it is necessary to set up a system for rotating the logs.
Log rotation can be as simple as switching out logs as they get too big, or it can be a system of archiving and storing old copies to reference at a later date. This depends on your configuration.
Below, we will discuss some different methods of achieving this.
Manually Rotating Logs
It is not possible to move log files while Apache is running, so instead, the server must be restarted in order to swap the old logs for fresh logs.
This can be accomplished manually by moving the files, then soft-restarting Apache so that it can begin to use the new logs for new connections.
The example used in the Apache documentation is here. You may need to precede these commands with "sudo" to gain appropriate privileges:
mv access_log access_log.old mv error_log error_log.old apachectl graceful sleep 600 [post-processing of log files]
This will move the files, reload the server, and wait 600 seconds. This will allow Apache to continue using the old log files to complete logging from old requests. During this time, new requests will be logged to the new, refreshed files.
While it is good to know how to do this manually, this would be unsustainable for larger server environments.
Managing Log Rotation Using Logrotate
By default, Ubuntu sets up its own log rotation plan with logrotate.
This program can take parameters and rotate logs when certain criteria are met. We can see what events cause logrotate to swap the Apache logs by looking in "/etc/logrotate.d/apache2":
sudo nano /etc/logrotate.d/apache2
Here, you can see some of the parameters given to logrotate. First of all, notice the first line is:
/var/log/apache2/*.log {
This means that logrotate will only operate on those logs in "/var/log/apache2". Keep this in mind if you have chosen a different directory for your logs in your Apache configuration.
We can see that the logs are rotated weekly and that they save a years-worth of logs by default. We can also see that there is a section that reloads Apache after the rotation:
postrotate /etc/init.d/apache2 reload > /dev/null endscript
These lines make sure that apache is reloaded automatically whenever a rotation is complete. The parameters within this file can be changed at will, but the configuration is outside of the scope of this article.
Logging Using Pipes
Using a pipe instead of a file is an easy way to allow a separate logging program to handle the output. This solves the log rotation problem too, since that can be handled by the backend logging program instead of by Apache itself.
If we wanted the access log to be handled by a logging program that accepts standard input, we could change the line to this:
CustomLog "| logging_program logging_program_parameters" combined
Apache starts the logging program when it begins and restarts it if the logging process fails for any reason.
Although multiple programs can be configured to rotate logs, Apache includes one called "rotatelogs" by default. You can configure it as follows:
CustomLog "| /path/to/rotatelog /path/of/log/to/rotate number_of_seconds_between_rotations" log_level
Similar configuration can be achieved with other logging utilities.
Conclusion
It is important that you are logging everything that you need to correctly administrate your servers, and that you have a good log rotation mechanism in place in order to prevent files from growing too large.
You should now understand how to create and configure logging for Apache. The information that you have logged can be used to troubleshoot problems and anticipate when actions need to be taken.
By Justin Ellingwood