At the end of November, we’ll be migrating the Sematext Logs backend from Elasticsearch to OpenSearch

logging

Best Logging Practices: 14 Do’s and Don’ts for Better Logging

Ever found yourself drowning in a sea of log data, struggling to make sense of the overwhelming noise? Or perhaps faced a major system breakdown, only to find that your

Elasticsearch to OpenSearch Migration Facilitated by Sematext Cloud

OK, so you've decided to move from Elasticsearch to OpenSearch. Maybe our comparison helped you decide and maybe you've checked our guide on how to perform the migration. But how

Python Logging Tutorial: How-To, Basic Examples & Best Practices

Logging is the process of keeping records of activities and data of a software program. It is an important aspect of developing, debugging, and running software solutions as it helps

5 Logstash Alternatives [2023 Review]

When it comes to centralizing logs to Elasticsearch, the first log shipper that comes to mind is Logstash. People hear about it even if it’s not clear what it does:

20+ Best Log Management Tools for Monitoring, Analytics & More: Pros & Cons Comparison [2023]

Whether you capture them for application security and compliance, production monitoring, performance monitoring, or troubleshooting, logs contain valuable information about the health of your apps. But it all comes down

Best Java GC Log Analyzers: Top Analysis Tools You Need to Know in 2023

When an application written for the Java Virtual Machine is running, it constantly creates new objects and puts them on the heap. Well, at least in the vast majority of

15 Best Log Analysis Tools & Log Analyzers of 2024 (Paid, Free & Open-source)

Log analysis and management tools have become essential in troubleshooting. With logging analysis tools – also known as network log analysis tools – you can extract meaningful data from logs

9 Best Cloud Logging Services for Log Management, Analysis, Monitoring & More [2023 Comparison]

Log management stopped being a very simple operation quite some time ago. Long gone are the "good old days" when you could log into the machine, check the logs, and

HAProxy Logging Configuration Explained: How to Enable and View Log Files

HAProxy is generally the frontend layer of your application, which means it plays a critical role since all traffic first lands on this layer. Because of this, you need to

How to Tail Kubernetes Logs: Using the Kubectl Command to See Pod, Container, and Deployment Logs

Logs are a critical aspect of any production workload, as they give you insight into what is happening in your system and tell you which components may be having issues.

Black Friday log management (with the Elastic Stack) checklist

For this Black Friday, Sematext wishes you: more products sold more traffic and exposure more logs :) Now seriously, applications tend to generate a lot more logs on Black Friday,

NGINX Error & Access Logs: Tutorial for How to View and Configure Them for Efficient Logging

NGINX is one of the most widely used reverse proxy servers, web servers, and load balancers. It has capabilities like TLS offloading, can do health checks for backends, and offers

What Is Log Aggregation: A Complete Guide to Best Practices & Tools You Need to Know

Every developer's worst nightmare is digging through a huge log file, trying to pinpoint problems. The troubleshooting most likely won't stop there. They'll have to follow the trail to multiple

Log4Shell: How We Protect Sematext Users

On December 9, 2021, a vulnerability was reported that could allow a system running Apache Log4j 2 version 2.14.1 or below to be compromised and allow an attacker to execute

Understanding Apache Logging: How to View, Locate and Analyze Access & Error Logs

Apache - the technology that powers the web. I'm not sure if that is correct, but I think that we wouldn't see the world wide web in its current shape

Exception Handling in Java: How-to Tutorial with Examples & Best Practices

As developers, we would like our users to interact with applications that run smoothly and without issues. We want the libraries that we create to be widely adopted and successful.

Key Kubernetes Metrics and Resources to Monitor for Peak Cluster Performance

Monitoring is not easy. Period. In our guide to Kubernetes monitoring we explained how you need a different approach to monitoring Kubernetes than traditional VMs. Also, just in case you

SLF4J Tutorial: Loggers, Levels & How to Configure for Java Applications with Examples

Logging is a crucial part of the observability of your Java applications. Java logs combined with JVM metrics and traces give full observability into the application behavior and are invaluable

Logback Configuration Example: Tutorial on How to Use It for Logging in Java

Troubleshooting issues in your applications can be a complicated task requiring visibility into various components. In the worst-case scenario, to understand what is happening and why it is happening you

Java Logging Tutorial: Basic Concepts & Examples to Help You Log Efficiently

When it comes to troubleshooting Java application performance, JVM metrics are no longer enough. To fully understand the environment you also need Java logs and traces. Today, we're going to

Log4j2 Configuration Tutorial: Appenders, Levels, Examples

When it comes to troubleshooting application performance, the more information you have the better. Java logs combined with JVM metrics and traces give you full visibility into your Java applications.

Log4j Tutorial: How to Configure the Logger for Efficient Java Logging [Example]

Getting visibility into your application is crucial when running your code in production. What do we mean by visibility? Primarily things like application performance via metrics, application health, and availability,

Node.js Error Handling Made Easy: Best Practices On Just About Everything You Need to Know

Don't you hate it when you see an uncaughtException error pop up and crash your Node.js app? Yeah... I feel you. Can anything be worse? Oh yeah, sorry, unhandledRejection I

Solr Performance: Troubleshooting Solr Slow Queries Using Logs and Metrics

Let's say you get an alert that one or more queries is slow. Or that your users complain, whichever comes first 🙂  We've all been there… How do you find

Node.js Logging Tutorial: How to Check, Write & Debug [Best Practices]

Building SaaS products is hard. Making customers happy is even harder. I should know, I've built a start-up that failed! But, not everything is that bad. I learned a lot.

Understanding Logging Levels: What They Are & How To Use Them

Making sense out of logs is not an easy task. Log management solutions gather and accept data from multiple sources. Those sources can have different log structures, providing a different

What Is Log Analysis: Definition, Logging Use Cases & How-to Guide

Logs are one of the most valuable assets when it comes to IT system management and monitoring. As they record every action that took place on your network, logs provide

Java Logging Best Practices: 10+ Tips You Should Know to Get the Most Out of Your Logs

Having visibility into your Java application is crucial for understanding how it works right now, how it worked some time in the past and increasing your understanding of how it

What Are Linux Logs & How to Check Them: Logging Basics Explained with Examples

TL;DR note: if you want the bzip2 -9 version of this post, scroll down to the very last section for some quick pointers. If you want to learn a bit

Tutorial: Logging with journald

I'm sure you bumped into journald: it's what most distros use by default for system logging in Linux. Most applications running as a service will also log to the journal.

Where Are Docker Container Logs Stored?

Where are Docker container logs stored? There’s a short answer, and a long answer. The short answer, that will satisfy your needs in the vast majority of cases, is: /var/lib/docker/containers/<container_id>/<container_id>-json.log

Logging vs Monitoring: How are They Different & Why You Need Both

By Chris Tozzi Logging or monitoring? If you deploy and manage an application, these are the two key techniques available to you for helping to ensure that the application meets

Logstash Tutorial: Getting Started Guide for Shipping Logs

Looking to learn about Logstash as quickly as possible? This Logstash Tutorial is for you: we’ll install Logstash and push some Apache logs to Elasticsearch in less than 5 minutes.

14 Monitoring and Logging Best Practices and Standards for Monitoring

When managing cloud-native applications, it’s essential to have end-to-end visibility into what’s happening at any given time. This is especially true because of the distributed and dynamic nature of cloud-native

Monitoring Linux Audit Logs with auditd and Auditbeat

The Linux Audit framework is a kernel feature (paired with userspace tools) that can log system calls. For example, opening a file, killing a process or creating a network connection.

Centralized AWS Lambda Logs with Kinesis and Serverless

The key to gaining serverless observability is sending all AWS Lambda Logs to a central location where you can later group, filter and make sense of them. Sematext is a full-stack

5 Benefits to Run Elastic Stack in the Cloud

By Chris Riley What is the Elastic Stack? Elasticsearch, Logstash, and Kibana — the trio better known as Elastic Stack (or ELK, if you prefer a term that is now

The New Version of Logagent Enriches Container Logs with Metadata and GeoIP

Logagent is a modern, open-source, light-weight data shipper with out of the box and extensible log parsing, on-disk buffering, secure transport and bulk indexing to Elasticsearch and Sematext Cloud. Its

Elasticsearch Ingest Node vs Logstash Performance

Unless you are using a very old version of Elasticsearch you’re able to define pipelines within Elasticsearch itself and have those pipelines process your data in the same way you’d

Recipe: How to integrate rsyslog with Kafka and Logstash

This recipe is similar to the previous rsyslog + Redis + Logstash one, except that we’ll use Kafka as a central buffer and connecting point instead of Redis. You’ll have

Handling Multiline Stack Traces with Logstash

Here at Sematext we use Java and rely on Elasticsearch for our hosted ELK logging SaaS, a lot. We like them so much that we regularly share our logging experience

Logtrail + Kibana = Retro Terminal Experience

Logtrail is a popular Kibana plugin that makes Kibana look like Papertrail. Sematext’s Logtrail view in Kibana gives you the terminal-like experience of exploring logs without the pain of needing

Docker Log Driver Alternatives

"Why does the 'docker logs' command fail?", is one of our frequently asked questions. The answer is simple and mentioned in the Docker documentation: "The docker logs command is not available for drivers

GDPR: Top 5 Logging Best Practices

The rather broad definition of personal data in the GDPR requires paying special attention to log data. GDPR and personal data in web server logs is a popular topic in

Elasticsearch online training classes in 2018

Since we’ve been working with Elasticsearch on a daily basis for more than 7 years now and know so much about it, we like helping others improve their knowledge of

Top 10 Docker Logging Gotchas

Docker changed not only how applications are deployed, it also changed the workflow for log management. Instead of writing logs to files, containers write logs to the console (stdout/stderr) and

Embracing Kubernetes Successfully

Kubernetes is a really hot topic at the moment. All major cloud providers adopted it as a solution for deploying cloud native apps. Just a few weeks ago, AWS at

How to ship Kibana Server Logs to Elasticsearch

When dealing with log centralization in your organization you have to start with something. Often times people start by collecting logs for the most crucial pieces of software, and frequently

Logagent Meets Apache Kafka

This is a guest post from Filippo Balicchia. Filippo contributed Logagent plugins for Apache Kafka the details of which he is sharing in this post.  Filippo is a software engineer

How to Detect Malicious Traffic in Your Server Logs: Sematext + Access Watch

Do you know what portion of your traffic comes from bots? Do you know which bots are good and which ones are bad? Do you know what the bad bots

Introducing Sematext Cloud

While some are hallucinating about building walls, we at Sematext are looking to knock them down. One of the Sematext differentiators is that it’s not just a monitoring solution or

Introducing Instant Templates and Mappings

Logsene, our hosted ELK stack now part of Sematext Cloud, exposes the Elasticsearch API. As a Logsene user you may or may not be aware of this.  If you have

Metrics & Logs – Together or Separate?

Every modern organization depends on software, and where there is software there are metrics and logs and, one would hope, monitoring of these two critically important types of data. Operations

S3 Archiving & Longer Retention for Logs

Need to have your recent logs searchable for your DevOps team(s) to troubleshoot issues, but also need to archive old logs for compliance reasons?   Many organizations need that and

Navigation, Discoverability, and Accessibility

We’ve restructured the UI …. bigly :).   Everything you need can be quickly accessed via tabs on the left.  Account section is exposed, and Team-related functionality (invitations to apps

Tighter Metrics & Logs Integration

One of the Sematext differentiators is that it’s not just a monitoring solution or just log management solution.  It’s monitoring AND log management in a single solution.  This single pane

[VIDEO] Elasticsearch for logs and metrics: A deep dive @O’Reilly Velocity

Since we work with Elasticsearch and Solr on a daily basis and know so much about them we like helping others improve their knowledge of these technologies, not only through

Network Security with Bro (now Zeek) and Elasticsearch

Intrusion detection systems generate highly valuable logs with network usage details and alerts. They collect vast amounts of data and typically store them in structures with a large number of

Exploring Windows Kernel with Fibratus and Sematext

This is a guest post by Nedim Šabić, developer of Fibratus, a tool for exploration and tracing of the Windows kernel.  Unlike Linux / UNIX environments which provide a plethora of open

Elasticsearch for logs and metrics: A deep dive – Velocity 2016, O’REILLY CONFERENCES

We are known worldwide for our Elasticsearch, ELK stack and Solr consulting services, and we are always happy to help others improve their skills in these technologies, not only through

Tuning Solr & Pipeline for Logs – Video & Slides

Not everyone uses Splunk or ELK stack for logs. A few weeks ago, at the Lucene/Solr Revolution conference in Boston, we gave a talk about using Solr for logging, along

How to Ship Heroku Logs to Logsene / Managed ELK Stack

Update: We made it much easier to ship Heroku logs. Please read 5 Minute Recipe: Heroku Log Drain Setup for more details.  Heroku is a cloud platform based on a managed

Sending your Windows Event Logs to Sematext using NxLog and Logstash

There are a lot of sources of logs these days. Some may come from mobile devices, some from your Linux servers used to host data, while other can be related to your

How to forward CloudTrail (or other logs from AWS S3) to Logsene

This recipe shows how to send CloudTrail logs (which are .gz logs that AWS puts in a certain S3 bucket) to a Logsene application, but should apply to any kinds

Using Grafana with Elasticsearch for Log Analytics

Grafana is an open-source alternative to Kibana. Grafana is best known as a visualization / dashboarding tool focused on graphing metrics from various data sources, such as InfluxDB. Even though

Presentation: Log Analysis with Elasticsearch

Fresh from the Velocity NYC conference is the latest presentation from Sematext engineers Rafal Kuć and Radu Gheorghe — “From zero to production hero: Log Analysis with Elasticsearch.” The talk

Recipe: Apache Logs + rsyslog (parsing) + Elasticsearch

More than two years ago we posted a recipe on how to centralize syslog in Elasticsearch in order to search and analyze them with Kibana, all by using only rsyslog.

Recipe: rsyslog + Redis + Logstash

OK, so you want to hook up rsyslog with Logstash. If you don't remember why you want that, let me give you a few hints: Logstash can do lots of

Introducing Logsene CLI

[Note: this post was written by Sematext engineer Marko Bonaći] In vino veritas, right?  During a recent team gathering in Kraków, Poland, and after several yummy bottles of țuică, vișinată,