Sematext Cloud comes with predefined dashboards for every Monitoring App. A huge time saver, because you don’t need to spend hours creating your monitoring or logging dashboards. However, the predefined reports might not be optimal for your particular observability needs. Especially for Logs Apps, you might find that Sematext’s native reports are very handy as you won’t need to switch to Kibana to have a quick look at your report. All reports can be scheduled and delivered via e-mail (click to email icon in the report actions bar).
We recently introduced several features that let you customize and extend Sematext App reports:
- Modify the existing App reports e.g. by moving charts via drag & drop or remove a chart via the top-left menu in the chart.
- Add or remove widgets in existing reports
- Add or change the displayed filters
- Add a new report page with your favorite metrics charts and filters
- Clone a report
- Organize reports in “folders” by adding groups for reports
- Change the order of groups and reports in the left sidebar using the handle on the left side of each group or report in the left sidebar
Note: All changes to App reports are visible to all App users with access to that App after hitting the “save” button. Don’t be afraid of playing around – you can always reset the App to the original state using the “Reset all reports” menu item, next to the App name in the left navigation bar.
Yandex ClickHouse is an open source column-oriented database management system capable of real-time generation of analytical data reports using SQL queries.
ClickHouse allows you to not only lower your TCO (Total Cost of Ownership) but also allows you to benefit from unmatched performance and scalability.
The setup, tuning and operations of ClickHouse require deep insights into the performance metrics such as locks, replication status, merge operations, cache usage and many more…
The new ClickHouse integration supports over 70 different ClickHouse metrics for system, queries, merge tree, replication, replicas, mark cache, R/W buffers, dictionaries, locks, distributed engine and Zookeeper errors & wait times.
ClickHouse key metrics in Sematext Cloud
How to use it
Because ClickHouse heavily relies on ZooKeeper we recommend setting up ZooKeeper monitoring and log collection for related logs. By doing this you can benefit from combined dashboards thus having your favorite ZooKeeper and ClickHouse metrics together with related logs for faster troubleshooting. Save time, gain visibility: the benefits are endless.
You asked, we listened. The new Sematext Monitoring Agent includes many features that will make your infrastructure and application monitoring a whole lot easier.
The recently opened-sourced Sematext Monitoring Agent is fully pluggable, making it possible for you to collect metrics from a number of additional sources. The new agent supports the latest Java versions and has updates for many integrations such as Apache Kafka, introduces support for ClickHouse monitoring, etc. In addition, we changed how the agent talks to Sematext. Instead of using a custom protocol, we’ve adopted the popular Influx Line Protocol for sending metrics and all their metadata via HTTPS. The old proprietary format will continue to work for a while to ensure backward compatibility with older agent versions.
You can find all details about the release in Open-sourcing Sematext Monitoring Agent.
The agent update is super simple: just use your package manager to update the spm-client package.
Kibana 6.x comes with a number of improvements including:
- New colors to improve accessibility
- Vega interactive visualizations
- Ability to apply Vega filters to dashboard context
- Visualization of other and missing terms buckets
- Lab visualizations
- Improved workflow for inspecting data
- Dashboard input controls
- And more!
You can find the Kibana button on the menu […] next to the App Switcher in the logs view.
Start Kibana 6.4 in Sematext Cloud and explore new features and visualizations. Enjoy!
The new Containers view provides you with enhanced visibility into all your containers and their hosts, drawing your attention to the ones that need it the most.
The main view is very much like “top for containers” – applied across all of your containers and their hosts. You can sort containers by memory or CPU usage to find the hottest containers, or simply display the top N containers. The detail view of each container displays container metrics in real-time.
In containers table, the user can see basic information and metrics about each server and by clicking on table row user can expand it to get more details about the container. Thus, for each container user can view metrics, review information about containers grouped by tags like host, image name, pod name …
By using the heatmap, user can get an overview of a large number of containers at once and see if selected metric has a high value on any of the monitored containers or host.
Sematext Logs Elasticsearch API Extra Security | July 2018
The Sematext API for log ingestion log search has always been compatible with the Elasticsearch API. On top of that, the Sematext API has always had additional security in place, for example:
- HTTPS encryption on all Sematext data receivers
- The API used the App tokens as index name, with the ability to generate multiple read-only or write-only tokens in the App settings. For example, somebody who needs to set up a log shipper really requires only the write-only token and doesn’t need to know the read-capable token, which is needed only if you want to query logs via the API.
To further secure access to our customers’ data we now require not just the App token, but also the API key for log searches via API. Importantly, note that you do not have to change any of your log shipping.
This change affects only access via read APIs, not write APIs. You can get your API key from https://apps.sematext.com/ui/account/api and, as you can see there, it can be regenerated at any time should it ever get compromised.
The following example shows a search request with curl using the new authentication header:
curl 'https://logsene-search.sematext.com:443/<APP TOKEN HERE>/_search?size=0&pretty' -H 'Authorization: apiKey <API KEY HERE>'
The API also supports HTTP Basic Authentication. Therefore third-party tools using Elasticsearch client libraries that use Basic Authentication with Elasticsearch can authenticate as well. The username is then the “apiKey” literal, and the password is the actual API key. Here is another example using curl with basic authentication:
curl -v -u apiKey:<API KEY HERE> 'https://logsene-search.sematext.com/<APP TOKEN HERE>/_search?size=0&pretty'
Servers view is giving overview over host servers used by your apps. User can use GroupBy and FilterBy
filters to decide which servers or group of servers he wants to monitor. There two main components for
monitoring: table-view and heat-map.
In servers table user can see basic informations and metrics about each server and by clicking on table
row user can expands it to get more details about server. Thus, for each server user can view metrics,
review informations about docker containers on that host and review metrics and informations for apps on
By using heat-map user can get overview over large number of servers at once and see if selected metric
has high value on any of monitored servers.
If you work with monitoring or monitoring tools much, you’ve probably seen the phrase “correlating”
here and there. Correlation is what can help you find metrics with similar pattern and see dependencies
between metrics even if you don’t suspect they are in dependency relation. When troubleshooting,
automatic metrics correlation is the easiest way to find root cause.
Automatic metrics correlation in Sematext Cloud work pretty straight forward: you select a metrics of
interested, select time range and Correlation mechanism will find all metrics which have similar behaviour
in the same time range. Similar spikes, similar trends, similar waves.
Dealing with log files or extracting data from various data sources is a daily task in the IT
administration. Adding logtrail to Kibana gives users the “terminal experience”,
that experience of tailing and filtering log files like in the good old terminal that tons of
people are used to and that many of us still love – typically black background, new logs appearing
at the bottom of the screen instead of the top screen, simply way to filter logs and “jump”
to various parts of the log stream.
To summarise this plugin enables you and your team to view, analyze and search log events from a
centralised clean & developer friendly interface.
Log in to see for yourself and check it out….. or read our blog post to find out about a few more improvements we’ve made.