Log management

Logagent is a modern, open-source, light-weight data shipper with out of the box and extensible log parsing, on-disk buffering, secure transport and bulk indexing to Elasticsearch and Sematext Cloud. Its low memory footprint and low CPU overhead make it suitable for […]

Starting from Elasticsearch 5.0, you’re able to define pipelines within it that process your data, in the same way you’d normally do it with something like Logstash. We decided to take it for a spin and see how this new […]

This recipe is similar to the previous rsyslog + Redis + Logstash one, except that we’ll use Kafka as a central buffer and connecting point instead of Redis. You’ll have more of the same advantages: rsyslog is light and crazy-fast, […]

Here at Sematext we use Java and rely on Logsene, our hosted ELK logging SaaS, a lot. We like them so much that we regularly share our logging experience with everyone and help others with logging, especially, ELK stack. Centralized […]

When it comes to centralizing logs to Elasticsearch, the first log shipper that comes to mind is Logstash. People hear about it even if it’s not clear what it does: – Bob: I’m looking to aggregate logs – Alice: you […]

The Linux Audit framework is a kernel feature (paired with userspace tools) that can log system calls. For example, opening a file, killing a process or creating a network connection. These audit logs can be used to monitor systems for […]

The rather broad definition of personal data in the GDPR requires paying special attention to log data. GDPR and personal data in web server logs is a popular topic in many GDPR fora. For example, IP addresses or cookies might […]

Centralizing logs to Elasticsearch? Of course, the first log shipper that comes to mind is Logstash. When you get into it, you realize centralizing logs often implies a bunch of things, and Logstash isn’t the only log shipper that fits […]

Do you know what portion of your traffic comes from bots? Do you know which bots are good and which ones are bad? Do you know what the bad bots are up to? We all know Googlebot and consider it […]