Skip to content
share

Varnish Cache Logs Integration

To use the Sematext Varnish Cache Logs integration simply install the Sematext Agent. The agent will discover your logs and you'll be able to configure Varnish Cache log shipping via the UI. When doing that, you will want to create or select an existing Varnish Cache Logs App because that is what will provide you with all the out of the box dashboards, some of which you can see below. Moreover, the pre-configured anomaly alert rules will notify you about increasing 4xx and 5xx response rates.

Log Collection

  • To enable Varnish logging uncomment the following in /etc/default/varnishncsa:

  VARNISHNCSA_ENABLED=1

  • Create st_varnishncsa_format file and add the following:

"Timestamp": "%t", "network_client_ip": "%h", "varnish_hit_miss": "%{Varnish:hitmiss}x", "varnish_side": "%{Varnish:side}x", "age": %{age}o, "handling": "%{Varnish:handling}x", "http_request": "%r", "varnish_time_first_byte": %{Varnish:time_firstbyte}x, "http_method": "%m", "http_status_code": %s, "response_reason": "%{VSL:RespReason}x", "fetch_error": "%{VSL:FetchError}x", "x_forwarded_for": "%{x-forwarded-for}i", "remote_user": "%u", "network_bytes_written": "%b", "http_response_time": %D, "http_user_agent": "%{User-agent}i", "http_referer": "%{Referer}i", "x_varnish": "%{x-varnish}o", "x_magento_yags": "%{x-magento-tags}o"

  • Execute the sudo systemctl edit varnishncsa command to edit ExecStart property and paste the following:

[Service]
ExecStart=
ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -D -c -b -f /home/user/st_varnishncsa_format -P /run/varnishncsa/varnishncsa.pid

You should provide the full path to the st_varnishncsa_format file in the command above. In this example, the file was located within the /home/user directory.

  • Restart the varnishncsa to apply the changes.

Exploring logs

Once data is in, you can explore it via the built-in reports or create your own.

Varnish Cache Logs Overview Report

Traffic Insight Report

You can use the Traffic Insight report to see top clients, referrers, average response time and time to serve the first byte based on servers and "zoom in" to the ones you're interested in:

Varnish Cache Traffic Insight Report

Traffic Distribution Report

You can use the Traffic Distribution report to see the traffic distribution across servers for resource planning by analyzing data volume and bytes served:

Varnish Cache Traffic Distribution Report

Client Side Logs Report

You can use the Client Side Logs report to see the requests made to the client side:

Varnish Cache Client Side Logs Report

Backend Side Logs Report

You can use the Backend Side Logs report to see the requests made to the backend side:

Varnish Cache Backend Side Logs Report

Troubleshooting

If you have trouble sending logs, try out the latest version of Sematext Agent. Also, make sure Sematext Agent is configured to send logs to your Varnish Cache Logs App. Lastly, check the Log Agents panel for any errors, and refer to our Sematext Logs FAQ for useful tips.