With 10+ years of experience in building a centralized logging solution as well as being ELK consultants for hundreds of companies, we can design, optimize and troubleshoot logging setups through all steps:
- Log generation: The log format and initial destination options have important trade-offs. We're here to help you implement the right one for your needs: here's an example analysis for syslog.
- Log processing pipeline: Should you use Logstash, Fluentd or something else? How do we parse and enrich logs efficiently? What about Ingest nodes? We can help you find the right answer for these questions (and many more) as they apply to your use-case.
- Make Elasticsearch/OpenSearch perform and scale: We've done this literally since both projects were born through consulting, production support and training, as well as Sematext Logs.
- Visualize logs in Kibana/OpenSearch Dashboards: From best practices in plotting metrics to writing useful alert rules.
- Review and optimize Elasticsearch/OpenSearch setup: from merge policy to dynamic templates to garbage collection tuning and OS-level knobs and everything in between.
- Optimize the cluster layout: from dividing data into multiple streams to lifecycle policies to the number of nodes and tiers to the hardware profile of each node.
- Design the logging pipeline: where to do parsing and enriching, how many layers should there be, which log shippers and custom components, etc.
- Upgrade the ELK Stack or OpenSearch.
- Review and optimize the logging pipeline: logging in JSON, grammar vs regex-based parsing, where to buffer, how to batch and deal with backpressure, etc.
- Help build visualizations: build components for Kibana and OpenSearch Dashboards, best practices in building and using dashboards.
- Migrate to Elasticsearch/OpenSearch from other systems.
- Migrate from Elasticsearch to OpenSearch or vice versa.