logging
Best Logging Practices: 14 Do’s and Don’ts for Better Logging
Ever found yourself drowning in a sea of log data, struggling to make sense of the overwhelming noise? Or perhaps faced a major system breakdown, only to find that your
Elasticsearch to OpenSearch Migration Facilitated by Sematext Cloud
OK, so you've decided to move from Elasticsearch to OpenSearch. Maybe our comparison helped you decide and maybe you've checked our guide on how to perform the migration. But how
Python Logging Tutorial: How-To, Basic Examples & Best Practices
Logging is the process of keeping records of activities and data of a software program. It is an important aspect of developing, debugging, and running software solutions as it helps
5 Logstash Alternatives [2023 Review]
When it comes to centralizing logs to Elasticsearch, the first log shipper that comes to mind is Logstash. People hear about it even if it’s not clear what it does:
20+ Best Log Management Tools for Monitoring, Analytics & More: Pros & Cons Comparison [2023]
Whether you capture them for application security and compliance, production monitoring, performance monitoring, or troubleshooting, logs contain valuable information about the health of your apps. But it all comes down
Best Java GC Log Analyzers: Top Analysis Tools You Need to Know in 2023
When an application written for the Java Virtual Machine is running, it constantly creates new objects and puts them on the heap. Well, at least in the vast majority of
15 Best Log Analysis Tools & Log Analyzers of 2024 (Paid, Free & Open-source)
Log analysis and management tools have become essential in troubleshooting. With logging analysis tools – also known as network log analysis tools – you can extract meaningful data from logs
9 Best Cloud Logging Services for Log Management, Analysis, Monitoring & More [2023 Comparison]
Log management stopped being a very simple operation quite some time ago. Long gone are the "good old days" when you could log into the machine, check the logs, and
HAProxy Logging Configuration Explained: How to Enable and View Log Files
HAProxy is generally the frontend layer of your application, which means it plays a critical role since all traffic first lands on this layer. Because of this, you need to
How to Tail Kubernetes Logs: Using the Kubectl Command to See Pod, Container, and Deployment Logs
Logs are a critical aspect of any production workload, as they give you insight into what is happening in your system and tell you which components may be having issues.
Black Friday log management (with the Elastic Stack) checklist
For this Black Friday, Sematext wishes you: more products sold more traffic and exposure more logs :) Now seriously, applications tend to generate a lot more logs on Black Friday,
NGINX Error & Access Logs: Tutorial for How to View and Configure Them for Efficient Logging
NGINX is one of the most widely used reverse proxy servers, web servers, and load balancers. It has capabilities like TLS offloading, can do health checks for backends, and offers
What Is Log Aggregation: A Complete Guide to Best Practices & Tools You Need to Know
Every developer's worst nightmare is digging through a huge log file, trying to pinpoint problems. The troubleshooting most likely won't stop there. They'll have to follow the trail to multiple
Log4Shell: How We Protect Sematext Users
On December 9, 2021, a vulnerability was reported that could allow a system running Apache Log4j 2 version 2.14.1 or below to be compromised and allow an attacker to execute
Understanding Apache Logging: How to View, Locate and Analyze Access & Error Logs
Apache - the technology that powers the web. I'm not sure if that is correct, but I think that we wouldn't see the world wide web in its current shape
Exception Handling in Java: How-to Tutorial with Examples & Best Practices
As developers, we would like our users to interact with applications that run smoothly and without issues. We want the libraries that we create to be widely adopted and successful.
Key Kubernetes Metrics and Resources to Monitor for Peak Cluster Performance
Monitoring is not easy. Period. In our guide to Kubernetes monitoring we explained how you need a different approach to monitoring Kubernetes than traditional VMs. Also, just in case you
SLF4J Tutorial: Loggers, Levels & How to Configure for Java Applications with Examples
Logging is a crucial part of the observability of your Java applications. Java logs combined with JVM metrics and traces give full observability into the application behavior and are invaluable
Logback Configuration Example: Tutorial on How to Use It for Logging in Java
Troubleshooting issues in your applications can be a complicated task requiring visibility into various components. In the worst-case scenario, to understand what is happening and why it is happening you
Java Logging Tutorial: Basic Concepts & Examples to Help You Log Efficiently
When it comes to troubleshooting Java application performance, JVM metrics are no longer enough. To fully understand the environment you also need Java logs and traces. Today, we're going to
Log4j2 Configuration Tutorial: Appenders, Levels, Examples
When it comes to troubleshooting application performance, the more information you have the better. Java logs combined with JVM metrics and traces give you full visibility into your Java applications.
Log4j Tutorial: How to Configure the Logger for Efficient Java Logging [Example]
Getting visibility into your application is crucial when running your code in production. What do we mean by visibility? Primarily things like application performance via metrics, application health, and availability,
Node.js Error Handling Made Easy: Best Practices On Just About Everything You Need to Know
Don't you hate it when you see an uncaughtException error pop up and crash your Node.js app? Yeah... I feel you. Can anything be worse? Oh yeah, sorry, unhandledRejection I
Solr Performance: Troubleshooting Solr Slow Queries Using Logs and Metrics
Let's say you get an alert that one or more queries is slow. Or that your users complain, whichever comes first 🙂 We've all been there… How do you find
Node.js Logging Tutorial: How to Check, Write & Debug [Best Practices]
Building SaaS products is hard. Making customers happy is even harder. I should know, I've built a start-up that failed! But, not everything is that bad. I learned a lot.
Understanding Logging Levels: What They Are & How To Use Them
Making sense out of logs is not an easy task. Log management solutions gather and accept data from multiple sources. Those sources can have different log structures, providing a different
What Is Log Analysis: Definition, Logging Use Cases & How-to Guide
Logs are one of the most valuable assets when it comes to IT system management and monitoring. As they record every action that took place on your network, logs provide
Java Logging Best Practices: 10+ Tips You Should Know to Get the Most Out of Your Logs
Having visibility into your Java application is crucial for understanding how it works right now, how it worked some time in the past and increasing your understanding of how it
What Are Linux Logs & How to Check Them: Logging Basics Explained with Examples
TL;DR note: if you want the bzip2 -9 version of this post, scroll down to the very last section for some quick pointers. If you want to learn a bit
Tutorial: Logging with journald
I'm sure you bumped into journald: it's what most distros use by default for system logging in Linux. Most applications running as a service will also log to the journal.
Where Are Docker Container Logs Stored?
Where are Docker container logs stored? There’s a short answer, and a long answer. The short answer, that will satisfy your needs in the vast majority of cases, is: /var/lib/docker/containers/<container_id>/<container_id>-json.log
Logging vs Monitoring: How are They Different & Why You Need Both
By Chris Tozzi Logging or monitoring? If you deploy and manage an application, these are the two key techniques available to you for helping to ensure that the application meets
Logstash Tutorial: Getting Started Guide for Shipping Logs
Looking to learn about Logstash as quickly as possible? This Logstash Tutorial is for you: we’ll install Logstash and push some Apache logs to Elasticsearch in less than 5 minutes.
14 Monitoring and Logging Best Practices and Standards for Monitoring
When managing cloud-native applications, it’s essential to have end-to-end visibility into what’s happening at any given time. This is especially true because of the distributed and dynamic nature of cloud-native
Monitoring Linux Audit Logs with auditd and Auditbeat
The Linux Audit framework is a kernel feature (paired with userspace tools) that can log system calls. For example, opening a file, killing a process or creating a network connection.
Centralized AWS Lambda Logs with Kinesis and Serverless
The key to gaining serverless observability is sending all AWS Lambda Logs to a central location where you can later group, filter and make sense of them. Sematext is a full-stack
5 Benefits to Run Elastic Stack in the Cloud
By Chris Riley What is the Elastic Stack? Elasticsearch, Logstash, and Kibana — the trio better known as Elastic Stack (or ELK, if you prefer a term that is now
The New Version of Logagent Enriches Container Logs with Metadata and GeoIP
Logagent is a modern, open-source, light-weight data shipper with out of the box and extensible log parsing, on-disk buffering, secure transport and bulk indexing to Elasticsearch and Sematext Cloud. Its
Elasticsearch Ingest Node vs Logstash Performance
Unless you are using a very old version of Elasticsearch you’re able to define pipelines within Elasticsearch itself and have those pipelines process your data in the same way you’d
Recipe: How to integrate rsyslog with Kafka and Logstash
This recipe is similar to the previous rsyslog + Redis + Logstash one, except that we’ll use Kafka as a central buffer and connecting point instead of Redis. You’ll have
Handling Multiline Stack Traces with Logstash
Here at Sematext we use Java and rely on Elasticsearch for our hosted ELK logging SaaS, a lot. We like them so much that we regularly share our logging experience
Logtrail + Kibana = Retro Terminal Experience
Logtrail is a popular Kibana plugin that makes Kibana look like Papertrail. Sematext’s Logtrail view in Kibana gives you the terminal-like experience of exploring logs without the pain of needing
Docker Log Driver Alternatives
"Why does the 'docker logs' command fail?", is one of our frequently asked questions. The answer is simple and mentioned in the Docker documentation: "The docker logs command is not available for drivers
GDPR: Top 5 Logging Best Practices
The rather broad definition of personal data in the GDPR requires paying special attention to log data. GDPR and personal data in web server logs is a popular topic in
Elasticsearch online training classes in 2018
Since we’ve been working with Elasticsearch on a daily basis for more than 7 years now and know so much about it, we like helping others improve their knowledge of
Top 10 Docker Logging Gotchas
Docker changed not only how applications are deployed, it also changed the workflow for log management. Instead of writing logs to files, containers write logs to the console (stdout/stderr) and
Embracing Kubernetes Successfully
Kubernetes is a really hot topic at the moment. All major cloud providers adopted it as a solution for deploying cloud native apps. Just a few weeks ago, AWS at
How to ship Kibana Server Logs to Elasticsearch
When dealing with log centralization in your organization you have to start with something. Often times people start by collecting logs for the most crucial pieces of software, and frequently
Logagent Meets Apache Kafka
This is a guest post from Filippo Balicchia. Filippo contributed Logagent plugins for Apache Kafka the details of which he is sharing in this post. Filippo is a software engineer
How to Detect Malicious Traffic in Your Server Logs: Sematext + Access Watch
Do you know what portion of your traffic comes from bots? Do you know which bots are good and which ones are bad? Do you know what the bad bots
Introducing Sematext Cloud
While some are hallucinating about building walls, we at Sematext are looking to knock them down. One of the Sematext differentiators is that it’s not just a monitoring solution or
Introducing Instant Templates and Mappings
Logsene, our hosted ELK stack now part of Sematext Cloud, exposes the Elasticsearch API. As a Logsene user you may or may not be aware of this. If you have
Metrics & Logs – Together or Separate?
Every modern organization depends on software, and where there is software there are metrics and logs and, one would hope, monitoring of these two critically important types of data. Operations
S3 Archiving & Longer Retention for Logs
Need to have your recent logs searchable for your DevOps team(s) to troubleshoot issues, but also need to archive old logs for compliance reasons? Many organizations need that and
Navigation, Discoverability, and Accessibility
We’ve restructured the UI …. bigly :). Everything you need can be quickly accessed via tabs on the left. Account section is exposed, and Team-related functionality (invitations to apps
Tighter Metrics & Logs Integration
One of the Sematext differentiators is that it’s not just a monitoring solution or just log management solution. It’s monitoring AND log management in a single solution. This single pane
[VIDEO] Elasticsearch for logs and metrics: A deep dive @O’Reilly Velocity
Since we work with Elasticsearch and Solr on a daily basis and know so much about them we like helping others improve their knowledge of these technologies, not only through
Network Security with Bro (now Zeek) and Elasticsearch
Intrusion detection systems generate highly valuable logs with network usage details and alerts. They collect vast amounts of data and typically store them in structures with a large number of
Exploring Windows Kernel with Fibratus and Sematext
This is a guest post by Nedim Šabić, developer of Fibratus, a tool for exploration and tracing of the Windows kernel. Unlike Linux / UNIX environments which provide a plethora of open
Elasticsearch for logs and metrics: A deep dive – Velocity 2016, O’REILLY CONFERENCES
We are known worldwide for our Elasticsearch, ELK stack and Solr consulting services, and we are always happy to help others improve their skills in these technologies, not only through
Tuning Solr & Pipeline for Logs – Video & Slides
Not everyone uses Splunk or ELK stack for logs. A few weeks ago, at the Lucene/Solr Revolution conference in Boston, we gave a talk about using Solr for logging, along
How to Ship Heroku Logs to Logsene / Managed ELK Stack
Update: We made it much easier to ship Heroku logs. Please read 5 Minute Recipe: Heroku Log Drain Setup for more details. Heroku is a cloud platform based on a managed
Sending your Windows Event Logs to Sematext using NxLog and Logstash
There are a lot of sources of logs these days. Some may come from mobile devices, some from your Linux servers used to host data, while other can be related to your
How to forward CloudTrail (or other logs from AWS S3) to Logsene
This recipe shows how to send CloudTrail logs (which are .gz logs that AWS puts in a certain S3 bucket) to a Logsene application, but should apply to any kinds
Using Grafana with Elasticsearch for Log Analytics
Grafana is an open-source alternative to Kibana. Grafana is best known as a visualization / dashboarding tool focused on graphing metrics from various data sources, such as InfluxDB. Even though
Presentation: Log Analysis with Elasticsearch
Fresh from the Velocity NYC conference is the latest presentation from Sematext engineers Rafal Kuć and Radu Gheorghe — “From zero to production hero: Log Analysis with Elasticsearch.” The talk
Recipe: Apache Logs + rsyslog (parsing) + Elasticsearch
More than two years ago we posted a recipe on how to centralize syslog in Elasticsearch in order to search and analyze them with Kibana, all by using only rsyslog.
Recipe: rsyslog + Redis + Logstash
OK, so you want to hook up rsyslog with Logstash. If you don't remember why you want that, let me give you a few hints: Logstash can do lots of
Introducing Logsene CLI
[Note: this post was written by Sematext engineer Marko Bonaći] In vino veritas, right? During a recent team gathering in Kraków, Poland, and after several yummy bottles of țuică, vișinată,