Sumologic

Apache

Archagent collects performance metrics from the server status page of apache web server. To monitor Apache, you need to enable server status page, by adding the following to httpd.conf / apache2.conf

<Location "/server-status">
    SetHandler server-status
    Require host localhost
    Allow from 127.0.0.1
</Location>

Additionally, you may specify username and password (if set), in data.json for http request authorization.

Having updated data.json on your server (no need to restart archagent), enable this plugin from the server's rule config page. Set thresholds for metrics that you care about and want to be alerted on. Enabling a plugin.

Apache Logs

Some important metrics such as request latency, errors, and other per-host metrics are not reported on the server status page, but can be obtained from apache log files. To set up log collection (optional, but recommended), you simply need to enable log collection in archagent and install fluentd (which can forward log data to archagent).

Enable Log Collection in Archagent

Enable log collection in the agent's config file. The relevant fields in archagent's config are shown below

"apache": {
  "enablelogcollection": true,
  "logport": 8118,
  "hosts": ["www.domain-1.com:80", "www.domain-2.com:80"]
}

logport can be any port that is free. Fluentd will send data to archagent on this port. Note down this port, as you will need to set the same port in Fluentd's config file (described later).

hosts field specifies a list of hosts (maximum of 10) which you wish to monitor. hosts could either be a list of apache virtual hosts (logged as %v:%p) or a list of remote hosts (logged as %h), depending upon which you wish to monitor. But you need to specify here, the exact names (hosts / virtual hosts, as the case may be) as they appear in your apache log files.

Restart the agent. The agent is now configured to receive logs from fluentd, and send them to ArchSaber. Now its time to setup fluentd to send the logs to archagent.

Install Fluentd

Fluentd is a popular production-ready open source log collector.

# Install ruby (>= 1.9.3), if not already installed, using RVM (ruby version manager)
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

\curl -sSL https://get.rvm.io | bash -s stable --ruby

source ~/.rvm/scripts/rvm

# Install fluentd ruby gem, along with the archagent plugin
gem install fluentd fluent-plugin-archagent --no-ri --no-rdoc

Configure & Start Fluentd

Create a config file for fluentd containing the two sections <source> and <match> as shown below.

<source>
  @type tail                            # Plugin for continuosly tailing the log
  format apache2                        # For default format (combined) of apache access logs, else use regex
  time_format %d/%b/%Y:%H:%M:%S %z      # Time format in access logs
  path /var/log/apache2/access.log      # Path (or comma-seperated paths) of log files
  tag apache.access
</source>

<match apache.access>
  @type archagent                       # ArchSaber's plugin for sending the log to archagent
  port 8118                             # Should be same as one in archagent's config file (logport)
  flush_interval 6s                     # Intervals at which data is forwarded to archagent.
</match>

format field

format specifies the expected format of the logs. Your LogFormat in apache config determines the value of the format field in fluentd's config. Have a look at some examples.

apache LogFormat fluentd format
combined format apache2
vhost_combined format /^(?<vhost>[^ ]*) (?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$/
common format /^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)?$/
"%v:%p %t %D %>s %O" format /^(?<host>[^ ]*) \[(?<time>[^\]]*)\] (?<latency>[^ ]*) (?<code>[^ ]*) (?<size>[^ ]*)?$/

Note that archagent only looks at log fields named host, latency, code and size, so name your regex capturing groups accordingly. Also, the data is grouped per host. In case you need to group data by virtual host, simply name its regex capturing group as host, like the last example in the table (which matches the starting %v:%p to host capturing group in the regex). You should test out the regex with a line of your log to ensure a full match.

Note that you need to make sure fluentd has the permissions to be able to read your apache access log files.

Start fluentd fluentd -c <path/to/config/file> -d <path/to/pid/file> -o <path/to/log/file>.

You should now be able to view your average response size, status codes and average latency (if you log it) grouped by host in the ArchSaber dashboard.

results matching ""

    No results matching ""