NGINX is a highly popular web server considered one of the fastest solutions on the market. You can use it for serving data for small, personal projects, as a reverse proxy, or as an enterprise-grade solution powering large e-commerce sites handling millions of users daily. It supports a wide range of capabilities, such as HTTP cache, load balancing, or even a mail proxy, and offers features such as HTTP/2, TLS offloading, WebSockets and gRPC.
To fully leverage its capabilities, you need to be aware of what is happening to your environment, and handling NGINX isn’t different. You should keep an eye on all the metrics it exposes and use its log files to fully understand the traffic coming to your servers.
In this blog post we will look at the top NGINX log analyzer tools that will help you ship, centralize, aggregate, and finally analyze log events to get a detailed picture of what is happening with your NGINX servers.
1. Sematext Logs
Sematext Logs is a log aggregation and analysis tool with first-class support for NGINX, allowing you to slice and dice through NGINX access and error logs to find the ones that are most important at the moment. Sematext Logs provide out-of-the-box dashboards with pre-configured visualizations showing you the most important information coming from your logs, like requests summary, number of requests with given response code and much more. You can correlate logs with events and metrics, live-tail logs, add alerts to logs, and use Google-like syntax for filtering. Its powerful search and filtering capabilities give you the log context needed to find log events occurring before and after the one you are currently looking at.
Sematext features auto-discovery of logs to help you centralize logs in highly scalable and dynamic environments, whether you are using bare metal, virtual machines, or containers. Along with the
Pros:
- Provides an agent with pre-configured parsing rules for NGINX and other common formats like Apache common or Java garbage collector.
- Ease of integration with the rest of the Sematext Cloud offerings like Experience (RUM), Synthetics, and Infrastructure Monitoring, giving you a full observability solution in a single tool. This is especially useful if you want to correlate the log data with NGINX metrics for quick root cause analysis.
- Accepts data from various sources that can ship data in Elasticsearch-compatible format allowing integration of log events from various sources inside your environment.
- Once shipped, you can manipulate the data via user-friendly Logs Pipelines that enable further data processing, such as numerical data extraction, IP to geolocation mapping and many more.
Cons:
- No support for unstructured text. You can only send JSON-formatted log events, use syslog to send data, or use the provided agent that supports a limited set of common formats. It won’t be an issue with NGINX log events but may become a limitation when dealing with many different log sources.
- It doesn’t allow mixing Kibana and Sematext native UI widgets in a single dashboard – it is one or the other.
Price:
Sematext Logs starts with a Basic plan allowing up to 500MB/day worth of logs and 7 days of data retention for free. The Standard plan has a limited set of features and starts at $50/month and includes 1GB/day and 7 days retention, which translates to around $0.1/GB of received data and $1.56/GB of stored data. The fully-featured Pro plan has Logs Pipelines, Log Archiving, and more, starting at around $60/month for 1GB/day of data and 7-day retention, meaning $0.1/GB of received data and $1.90/GB of stored data.
2. GoAccess
GoAccess is an open-source, real-time web log analyzer with NGINX log analysis capabilities. It works in the terminal if you lack access to the visual environment or via browser using a real-time generated HTML file. Incredibly fast, written in C, and supporting a wide range of web server formats, including NGINX, Apache and Amazon S3, it could be the only tool you need to process your NGINX logs and get all the necessary information out of them. The generated HTML files can be updated dynamically and provide details about requests, visitors, top URLs, 404 pages, IPs and hostnames of your visitors – these are only a few examples of the visualizations you get out of the box of this solution. On top of that, GoAccess supports virtual hosts, allowing you to easily identify which consumes the most resources.
Pros:
- First-class NGINX support that doesn’t need to be adjusted or configured.
- Minimal configuration needed to set up for NGINX logs.
- Beautiful terminal dashboard for hard-core site reliability engineers that prefer the dark terminal over rich web visualizations.
- Support for incremental log processing with the on-disk database.
Cons:
- Logs need to be centralized in a single shared place if you want to visualize all your NGINX hosts together.
Price:
GoAccess is an open-source and free-of-charge NGINX log analyzer.
3. Elastic Stack
The Elastic Stack, formerly known as ELK Stack, is one of the most widely adopted tools used for analyzing logs for most of the available technologies, including NGINX. It started as the ELK trio combining Elasticsearch, the search and analytics engine; Logstash, the log processing and shipping tool; and Kibana, the visualization layer. Since then, it has evolved to include additional tools, such as Beats, a lightweight log shipper with NGINX support included.
The solution enables you to aggregate logs, slice and dice them and process them to enable rich analysis. You can extract every piece of NGINX log events and enrich them with location and security information. With its powerful aggregation engine, you can create dashboards showing you all the necessary details about your NGINX and the users interacting with it. Depending on your use case, you can install and run it in your environment or the cloud. And all these are available from a single vendor.
Pros:
- Logging support for virtually any kind of technology available, allowing easy correlation between events across your infrastructure, thus enabling faster troubleshooting.
- Scalable to support large data volume and long retention.
- Mature and configurable log shippers with native support for NGINX.
- Rich, virtually unlimited visualizations.
Cons:
- May be challenging to maintain at a larger scale, requiring in-depth knowledge.
- Limited features of the Basic version.
Price:
The Basic version of Elastic Stack is free but self-managed, so you need to consider what comes with it – maintenance. It has limited features, including machine learning which becomes available with the platinum-grade tier, which starts at $125 a month.
4. Datadog
Datadog is a unified solution that allows you to navigate from your NGINX logs to metrics, traces and back for efficient root cause analysis and performance comparison. It also provides live tail functionality to monitor your logs in near real time. All from a single vendor.
Datadog offers log analyzers with support for NGINX logging capabilities, giving you everything you need to analyze your NGINX log data. With support for more than 170 technologies, you can centralize your logs in a single place and extract the metrics out of it for dynamic altering. Further, you can prioritize the high-value logs to get a cost-effective and scalable approach to centralized log management.
Pros:
- On data ingestion processing allows you to ship virtually any format of log events and parse them on the solution side, including NGINX.
- Possibility to generate metrics out of NGINX logs allowing for rich dashboarding for your access and error logs.
- Real-time alerting with log-based custom metrics.
- TCP/SSL support with SOC 2 compliance for the highest security when it comes to logs.
- Rich extensibility of processing pipelines with more than 170 common technologies available.
Cons:
- Only self-hosted data archiving in the Ingest plan.
- Unlimited user accounts are available only in the more expensive plan.
Price:
Datadog offers two options for their log analyzer. The Ingest plan pricing starts at $0.10 per ingested or scanned GB of uncompressed data a month with process, live time and archive available. You also benefit from self-hosted archiving parsing the log events on data ingestion and log-based metrics. The second option is Retain or Rehydrate, an on-demand billing plan that’s priced at $2.25 per million log events per month and includes 15 days of logs retention based on tags or facets, log patterns and analytics, and log rehydration for auditing and historical analysis.
Want to see how Sematext stacks up? Check out our page on Sematext vs Datadog.
5. SolarWinds Loggly
Loggly is a cloud-based log aggregation and management service that allows you to access and analyze your NGINX log data while giving you real-time insights into every detail of your NGINX logs. It’s a simple, yet efficient solution that uses well-established protocols such as HTTP or Syslog to ship the NGINX log data, so that you can choose the best way to ship each type of NGINX logs. The dynamic field explorer gives a real-time overview of your logs categorized by structure or customized view. And with its powerful full-text search capabilities you can search on individual fields to find the log events you are interested in – for example, the ones related to your most valuable URL handled by the NGINX server.
Pros:
- Server-side logs parsing, meaning that you don’t have to waste your server resources for log preparation.
- Agent-free logs collection enables easy shipping of NGINX logs.
- Support for popular log shippers.
- Parsing support for common log formats, including NGINX.
- Query time field extraction.
Cons:
- Some features, like API access, are only available when using higher plans, which can limit your log analysis process.
- Restrictive overage rules.
Price:
Loggly pricing starts with a free tier that includes 200MB/day of data ingestion with 7 days retention and includes basic functionality lacking alerting and customized dashboards. The paid plan starts at $79/month, billed annually for 1GB/day of data ingestion and 15 days retention, giving you access to a solution enriched with customized dashboards and email alerting. The Enterprise version starts at $279/month billed annually and includes all the features of the earlier plans, webhooks, custom daily data volume, and 15 to 90 days retention.
Want to see how Sematext stacks up? Check out our page on Sematext vs Loggly.
6. Sumo Logic
Sumo Logic is a unified logs platform with the first-class NGINX support. It features machine learning and predictive algorithms that enable daily processing of large amounts of data. The platform allows you to evaluate your NGINX server health, gain insights into the traffic origin and plan for resource allocation. Use various dashboards to track your users, see where most errors come from, and track requests and their trends across the platform.
Pros:
- Powerful query language.
- Automatic log patterns detection for easy NGINX integration.
- Centralized agents with easy setup supporting autoscaled environments.
- Rich visualization support to give you insights into every piece of NGINX logs.
Cons:
- Not available on-premises disqualifies the solution for some companies.
- No overage support.
Price:
Sumo Logic pricing is based on features and data ingestion. You can start with a free plan with limited features and up to 500MB daily data ingestion. The paid plans start with the Essential, which has log analytics, real-time alerting, and live and historical data dashboarding, costing you around $93/GB (an estimated cost based on annual commitment and 30 days data retention).
7. Papertrail
SolarWinds Papertrail is a cloud-hosted log aggregation and management tool with NGINX support and great features for log analysis. It allows you to search and analyze any type of log file, text log file, or data coming via the Syslog protocol. Using this protocol, Papertrail enables you to ship large amounts of log data very efficiently, making it a very good choice as an NGINX log analyzer.
Papertrail feels very much like using the console – its live tail search helps detect issues faster and trace back the chain of events to enable real-time root cause analysis. You can filter your NGINX log events by source, date, log severity level, user IP address, or message contents to focus on the most meaningful data.
Pros:
- Simple and user-friendly interface that mimics the console experience will help you focus on what is crucial in your NGINX log events.
- Built-in archiving of the data makes it easy to work with large volumes of NGINX log events.
- Data ingestion is calculated monthly, making the service spike-friendly, which is advantageous when dealing with NGINX and services having spiky behavior, such as e-commerce.
Cons:
- No rich visualization support.
- Overage is limited to 200% of the plan, which can result in logs dropping, especially for services with spiky traffic.
- Higher volume plans become expensive compared to other services especially considering limited visualization capabilities.
Price:
The pricing starts at $0 for 2 days worth of searchable data and 7 days archive with 50MB/month of data (with 16GB free as a first-month bonus). The paid plans start at $7 for 1GB/month of data, with the data being searchable for 1 week and archived for a year. The most expensive non-custom plan we can see is $230 for 25GB/month of data available for two weeks for searching and one year of archiving.
Want to see how Sematext stacks up? Check out our page on Sematext vs Papertrail.
Conclusion
Understanding the traffic hosted by your NGINX instances is crucial for having a healthy working environment and reacting to the trends your users are following or creating. Understanding the logs can help you quickly identify issues in your environment and intervene immediately, resulting in the best user experience for everyone using your services. This is why you need a tool for gathering, centralizing, processing and analyzing the log events coming from your NGINX servers. Hopefully, this blog post gave you an overview of the potential solutions you can consider when looking for the right NGINX log analyzer for your use case.