Do you run Elasticsearch, Solr, Zookeeper, Apache Kafka, or any other App in Kubernetes, Docker Enterprise or Swarm?

Then you know that it could be challenging to set up monitoring agents to scale with your dynamic applications. Some challenges are configuring and discovering dynamic IP addresses, port numbers, DNS resolutions, JMX access options or database credentials.

If you’ve ever wondered how to use Sematext for monitoring services running in containers, we’ve just made that incredibly simple. We’ve completely revamped the agent instructions to make them easier to follow with copy-paste-execute commands for all popular Sematext monitoring integrations. Instructions for running services in plain containers, Kubernetes, or Docker Swarm are all available.

The new autodiscovery feature in the Sematext Agent automates the deployment of application monitoring agents completely.

The first time you set up monitoring for an App, you add labels or annotations to your App containers with the monitoring token and any other optional properties for configuring the Agent.

The Sematext UI displays the copy/paste instructions for your App containers and Sematext Agent covering:

• Docker
• Docker Enterprise / Swarm
• Kubernetes / OpenShift
• Helm

You can now use Sematext Cloud for user experience monitoring as well. New Experience Apps collect data directly from your website or webapp visitors and give you a complete picture of their experience and satisfaction. Monitoring your infrastructure and processes that run on your side gives you only half the story about how users perceive the performance of your website or single page app when they access it through their browser or mobile device.

Do you use your website to sell products or services? Website visitors are more likely to leave if your site is performing poorly.

Do you have a SaaS product? Your web app performance will directly impact your customers’ satisfaction, and therefore the adoption of your service.

Maybe your customers from North America are happy with the website performance, but customers from EU or Asia might have a completely different experience.

Browsers, network connectivity, operating systems, geographical location all influence your website performance and by getting the measurements directly from your visitors you can be sure that you are not missing any data you need to make your website perform better.

Experience Apps have support for both websites and single-page apps built with React, Angular, Ember.js, Vue.js and other frameworks. We track page loads, HTTP requests, user profiles, and sessions, and very soon crash reports as well. You can read the announcement or the documentation for more information, as well as start using Experience in Sematext Cloud US or in
Sematext Cloud EU.

For more information, take a look at the Sematext Experience demo and dive into some of the features we offer in our new real user monitoring.

Monitoring integrations come with a predefined set of charts and dashboards. Great if you want to get started quickly and don’t really know which metrics you should be monitoring or don’t want to spend time building charts and dashboards from scratch. Now, in addition to these out of the box charts and dashboards you can also craft completely custom charts for monitoring your applications using all available metrics. These charts can be added to dashboards to visualize your infrastructure and application behavior.

Custom charts give you full control over how you want to visualize your data. Some data series are easier to analyze when presented as bar charts, others are better rendered as line charts. You can also customize colors, line styles, stacking, and more. It’s all to make your monitoring easier. You decide which presentation method gives you the most power. Ship metrics to Sematext with input format compatible with Influx and build your own dashboards with any metrics by its name.

Imagine you want to correlate CPU and memory usage. The easiest way to see the correlation is to draw CPU and memory usage on the same chart/ CPU usage is expressed in percentages while memory is measured in bytes. No worries – just configure two Y-axes units and assign CPU usage to one and memory usage to the other Y-axis, as shown in this short video:

Do you want to group data series by hostname and sum them up to get total memory usage in your cluster? Just pick the tag to group by and the aggregation function. Do you want to see memory usage of each host separately? That’s just as easy: pick a grouping tag and no aggregation function (‘all separately’).

Custom charts are a beta feature for now. Over the coming weeks you’ll be able to:

• Perform math operations on your data. For example, you’ll be able to create a data series derived from two or more metrics.
• Use filter variables, so all charts on the dashboard can be controlled interactively with a single click without the need to change each chart configuration separately.
• Use the query editor if you prefer typing over clicking to create your charts.
• Building charts that combine logs counts and metrics on a single chart.
• …

Core infrastructure monitoring with Sematext just got better. How?
The new agent collects container metrics and events just like the old agent, but it also collects Kubernetes metrics and events. As a result, you can now see a number of Kubernetes dashboards out of the box.

You can now identify processes that use the most resources, whether CPU or memory, across your entire infrastructure. Forget about ssh-ing to machines and running top. In Sematext you can see the hungriest processes right now, but also their behavior over time. Processes running in containers are also monitored, giving you visibility inside containers as well.

The new inventory monitoring captures information about installed packages, their versions, operating system information, Linux distributions, and many other useful bits. Package install and uninstall events details are also captured, so you can always trace who changes what, where, and when. Using Sematext you now have a place where you can identify outdated packages, packages with vulnerabilities, groups of servers with identical sets of packages as well as those machines that you thought have the same setup but in fact don’t, and more.

Network monitoring was rewritten to utilize eBPF whenever possible instead of relying on network packets. This new implementation has almost no overhead.

The agent was rewritten in Go, shedding the need for the JVM, resulting in smaller CPU and memory footprint.

All of the above-mentioned reports are a part of the new type of Monitoring App we are calling Infra App. You can have any number of Infra Apps in your Sematext account, though most people will have one Infra App per monitored environment. For example, you might create one Infra App for monitoring all of your production applications and infrastructure and another for monitoring your staging environment. Learn more about infrastructure monitoring.

If installing the Sematext Agent on Kubernetes manually felt too old fashioned and if Sematext Agent Helm Chart was not good enough, you’ll be pleased to hear that Sematext Agent is now available as a Kubernetes Operator. Sematext Operator is available on Github at https://github.com/sematext/sematext-operator. You can also find it certified in the RedHat’s OpenShift Container Platform via OperatorHub.io. Learn more in Kubernetes agent docs.

Do you know how many different Linux distros you are running in production? Do you know which ones you are running and their versions? How can you tell which version of openssh is installed on your servers? Or maybe you are running several different versions and you can’t really easily check that? Do you know if you are running any software with known vulnerabilities?

With Sematext’s new Inventory Monitoring you can now answer such questions with ease. The new infrastructure Inventory gives you information about your operating systems, Linux distributions, hardware specs, installed packages, log files, and more. The agent also detects package installation and removal events. Your event stream shows which packages were installed, removed, or updated, by who, where, and when. All inventory data is collected by the Sematext Agent and is part of core infrastructure monitoring. To start seeing your inventory information in Sematext be sure you are running a recent version of Sematext Agent. For more info check out inventory monitoring documentation and the official announcement.

You can now use Sematext for real-time process monitoring. With process monitoring you gain real-time visibility into which processes use the most CPU or memory, whether they are running inside containers, in cloud instances, your own VMs, or bare-metal servers. To start collecting information about processes make sure you are using a recent Sematext Agent. For more information see process monitoring documentation and the official announcement.

You can now control during which times of the day and which days of the week alert rules are active or inactive. By default an alert rule is active all day, every day. The new alert scheduling lets you pick, for each day of the week, one or more time slots when the alert rule is active. Each alert rule can have its own schedule. Each day of the week can have different times when an alert rule is active. You can have different schedule Monday to Friday vs. Weekend, for example. The alert schedule can be modified at any time and it comes into effect immediately. Have a look!

Sematext Cloud comes with predefined dashboards for every Monitoring App. A huge time saver, because you don’t need to spend hours creating your monitoring or logging dashboards. However, the predefined reports might not be optimal for your particular observability needs. Especially for Logs Apps, you might find that Sematext’s native reports are very handy as you won’t need to switch to Kibana to have a quick look at your report. All reports can be scheduled and delivered via e-mail (click to email icon in the report actions bar).

We recently introduced several features that let you customize and extend Sematext App reports:

• Modify the existing App reports e.g. by moving charts via drag & drop or remove a chart via the top-left menu in the chart.
• Add or remove widgets in existing reports
• Add or change the displayed filters
• Add a new report page with your favorite metrics charts and filters
• Clone a report
• Organize reports in “folders” by adding groups for reports
• Change the order of groups and reports in the left sidebar using the handle on the left side of each group or report in the left sidebar

Yandex ClickHouse is an open source column-oriented database management system capable of real-time generation of analytical data reports using SQL queries.

ClickHouse allows you to not only lower your TCO (Total Cost of Ownership) but also allows you to benefit from unmatched performance and scalability.

With the release of the new Sematext Monitoring Agent, you can now monitor ClickHouse DB clusters.

The setup, tuning and operations of ClickHouse require deep insights into the performance metrics such as locks, replication status, merge operations, cache usage and many more…

The new ClickHouse integration supports over 70 different ClickHouse metrics for system, queries, merge tree, replication, replicas, mark cache, R/W buffers, dictionaries, locks, distributed engine and Zookeeper errors & wait times.

ClickHouse key metrics in Sematext Cloud
How to use it

Tips

You asked, we listened. The new Sematext Monitoring Agent includes many features that will make your infrastructure and application monitoring a whole lot easier.

The recently opened-sourced Sematext Monitoring Agent is fully pluggable, making it possible for you to collect metrics from a number of additional sources. The new agent supports the latest Java versions and has updates for many integrations such as Apache Kafka, introduces support for ClickHouse monitoring, etc. In addition, we changed how the agent talks to Sematext. Instead of using a custom protocol, we’ve adopted the popular Influx Line Protocol for sending metrics and all their metadata via HTTPS. The old proprietary format will continue to work for a while to ensure backward compatibility with older agent versions.

You can find all details about the release in Open-sourcing Sematext Monitoring Agent.

If you are an existing Sematext Cloud user, we encourage you to log in and update your agent to start benefiting from the lower agent footprint and flexible configuration options.

The agent update is super simple: just use your package manager to update the spm-client package.

Kibana 6.x comes with a number of improvements including:

• New colors to improve accessibility
• Vega interactive visualizations
• Ability to apply Vega filters to dashboard context
• Visualization of other and missing terms buckets
• Lab visualizations
• Improved workflow for inspecting data
• Dashboard input controls
• And more!

You can find the Kibana button on the menu […] next to the App Switcher in the logs view.

Start Kibana 6.4 in Sematext Cloud and explore new features and visualizations. Enjoy!

The new Containers view provides you with enhanced visibility into all your containers and their hosts, drawing your attention to the ones that need it the most.

The main view is very much like “top for containers” – applied across all of your containers and their hosts. You can sort containers by memory or CPU usage to find the hottest containers, or simply display the top N containers. The detail view of each container displays container metrics in real-time.

In containers table, the user can see basic information and metrics about each server and by clicking on table row user can expand it to get more details about the container. Thus, for each container user can view metrics, review information about containers grouped by tags like host, image name, pod name …

By using the heatmap, user can get an overview of a large number of containers at once and see if selected metric has a high value on any of the monitored containers or host.

Want to find out more about the new container view?
Read our blog post.
Can’t wait to try the new containers view? Here is the direct link for
Sematext Cloud USA or
Sematext Cloud Europe.

Sematext Logs Elasticsearch API Extra Security | July 2018

The Sematext API for log ingestion log search has always been compatible with the Elasticsearch API. On top of that, the Sematext API has always had additional security in place, for example:

• HTTPS encryption on all Sematext data receivers
• The API used the App tokens as index name, with the ability to generate multiple read-only or write-only tokens in the App settings. For example, somebody who needs to set up a log shipper really requires only the write-only token and doesn’t need to know the read-capable token, which is needed only if you want to query logs via the API.

To further secure access to our customers’ data we now require not just the App token, but also the API key for log searches via API. Importantly, note that you do not have to change any of your log shipping.

This change affects only access via read APIs, not write APIs. You can get your API key from https://apps.sematext.com/ui/account/api and, as you can see there, it can be regenerated at any time should it ever get compromised.

The following example shows a search request with curl using the new authentication header:

curl 'https://logsene-search.sematext.com:443/<APP TOKEN HERE>/_search?size=0&pretty' -H 'Authorization: apiKey <API KEY HERE>'

The API also supports HTTP Basic Authentication. Therefore third-party tools using Elasticsearch client libraries that use Basic Authentication with Elasticsearch can authenticate as well. The username is then the “apiKey” literal, and the password is the actual API key. Here is another example using curl with basic authentication:

curl -v -u apiKey:<API KEY HERE> 'https://logsene-search.sematext.com/<APP TOKEN HERE>/_search?size=0&pretty'

Servers view is giving overview over host servers used by your apps. User can use GroupBy and FilterBy
filters to decide which servers or group of servers he wants to monitor. There two main components for
monitoring: table-view and heat-map.

In servers table user can see basic informations and metrics about each server and by clicking on table
row user can expands it to get more details about server. Thus, for each server user can view metrics,
review informations about docker containers on that host and review metrics and informations for apps on
that host.

By using heat-map user can get overview over large number of servers at once and see if selected metric
has high value on any of monitored servers.

If you work with monitoring or monitoring tools much, you’ve probably seen the phrase “correlating”
here and there. Correlation is what can help you find metrics with similar pattern and see dependencies
between metrics even if you don’t suspect they are in dependency relation. When troubleshooting,
automatic metrics correlation is the easiest way to find root cause.

Automatic metrics correlation in Sematext Cloud work pretty straight forward: you select a metrics of
interested, select time range and Correlation mechanism will find all metrics which have similar behaviour
in the same time range. Similar spikes, similar trends, similar waves.

Dealing with log files or extracting data from various data sources is a daily task in the IT
administration. Adding logtrail to Kibana gives users the “terminal experience”,
that experience of tailing and filtering log files like in the good old terminal that tons of
people are used to and that many of us still love – typically black background, new logs appearing
at the bottom of the screen instead of the top screen, simply way to filter logs and “jump”
to various parts of the log stream.

To summarise this plugin enables you and your team to view, analyze and search log events from a
centralised clean & developer friendly interface.

Log in to see for yourself and check it out….. or read our blog post to find out about a few more improvements we’ve made.

Sematext has been offering monitoring and log management for a while now, but in the most recent release
we’ve further unified it, making it really simple for people to go through the well-established DevOps path:
Alert Notification ⇒ Metrics Charts ⇒ Log Inspection for details

Moreover, we’ve made it dead-simple to see metrics and logs in the single view and visually correlate between the two.  End result? Much faster troubleshooting ⇒ much more efficient root cause analysis ⇒ much shorter MTTR ⇒ happier DevOps & happier users!

For more details about this and the benefits this functionality enables for your team
check out our blog post
or read more about monitoring and
logging with Sematext Cloud.

We’ve restructured the UI.
Everything you need can be quickly accessed via tabs on the left.  Account section is exposed, and Team-related functionality
(invitations to apps or to account, user roles, etc.) is in its own Teams section.
Dashboards are in top-left.
Metrics, Logs, and Events are right there next to each other. AppMap, NetMap, and a top-like view of all your Servers is under
Infrastructure. Jumping to a specific Metrics or Logs App or a Dashboard can be done quickly from the searchable pulldown in top-right.  
If you have access to multiple accounts you can switch between them much like you can switch between multiple Google accounts.

Log in to see for yourself and check it out… or
read our blog post
to find out about a few more improvements we’ve made.

Sematext Cloud
UI looks and behaves differently. Indeed,
we rewrote the whoooooole front-end using modern front-end technologies like React.js and Redux.
We’ve switched to Chart.JS for charts, and contributed a few PRs to it along the way. We’ve made it possible
to show/hide chart legends, change their positioning, to easily add/remove individual timeseries from metrics charts, and so on.
Dashboards have been completely rewritten and are much, much faster and lighter than before.
You can easily create a Dashboard that contains charts from multiple different Metrics or Logs apps. You can also add logs themselves
to a Dashboard, or event charts, or components with custom Markdown in them.  

For more details about this check out our blog
post.

When you’re troubleshooting and inspecting logs what do you do? You search and filter logs to narrow things down, to find log
messages that will help you understand what, how, why, or when something happened. The key part here is that you search, and when
you search what you typically see are only logs that matched what you searched for (think grep).
Log Context in Sematext Cloud is much like that.  
Once you’ve found a log event of interest a single click will expose its context.  
It’s like grep -C NUM, but applicable across all your infrastructure, not just one log file at a time.

For more details on log event context check out our blog post or see it for yourself.

Need to have your recent logs searchable for your DevOps team(s) to troubleshoot issues, but also need to
archive old logs for compliance reasons? Many organizations need that and Sematext Cloud makes that simple.
You can certainly simply pick a longer data retention option along with your plan and daily log volume. Doing that makes all
your logs searchable for a long time. If, however, you simply need to store your older logs, so that you can retrieve them
later on, but do not need to search them, you can use S3 Archiving option provided by Sematext Cloud.
All you need to provide are Access Key ID, your Secret Access Key, and the name of the S3 bucket where you want your
logs to be archived.

For more details on log event context check out our blog post.

Alert Rules and related Notification Hooks (aka Integrations) sharing option was updated. Now each Alert Rule and each Notification
Hook is tied to account under which it was created. This way any member of your team will still see all Alert Rules created
under the common shared account. Each team member will be able to use and create new Alert Rules and Notification Hooks, which will
also be visible to other members of the team (but editable only by the shared account owner, or users with admin role). Such Alert
Rules and Hooks will not be visible to App guests though, so your team’s work in the common shared account will not be affected by them.

For more details on this new sharing option and its benefits over plain App sharing
check out our blog post or
see it for yourself.

We’ve changed how Alert Rules & Notification Hooks are handled in Sematext Cloud.
They now live at Account level, not App level, which makes their management much simpler, more intuitive, much more suitable
for modern teams, as described in Alert
Rules & Hook Notification Sharing. The new “Account Default” switch makes it easy to quickly change where alert
notifications are sent for all Alert Rules that are set up to use default account notifications.  Very handy for agile teams
who may change their alerting channel preferences over time, have lots of Alert Rules, and would go crazy if they had
to modify each of them.

See it for yourself.

You can now hook up Sematext Cloud Alerts to a number of different Notification Hooks: email,
any custom WebHook, Slack, PagerDuty, HipChat, VictorOps, OpsGenie, BigPanda, Pushover – for Android, iPhone, iPad, and Desktop (Pebble,
Android Wear, and Apple Watch, too!), Nagios – yes, people still use it, Zappier – and from there you can connect Sematext Alerts to a
number of other services. Setting any of these notification hooks takes only seconds.  You only need to configure them once and then,
because notification hooks and alerts are set
at account-level, you can reference them when creating alert rules.

We’ve refreshed our backend, or one of them at least. We’ve updated Elasticsearch to 5.x version. Along with it, we’ve
updated the integrated version of Kibana to which you can easily switch from the “Native UI” if you are a Kibana aficionado. From there
you can create all kinds of charts and dashboards. We’ve also made sure Kibana URLs are shareable, so you can share them
with your teammates. If they have access to the Logsene App whose Kibana URL you are sharing they, too, will be able to access
the shared charts, dashboards, etc.

For more details on this check out our blog post or
see it for yourself by logging in and going to one of your Logsene apps.

Unlike logs, which tend to be high volume, events are a little more infrequent, a little more special.  
Events are things like alerts, or deploys, or builds, or… Ultimately though, it is up to you and your team to choose what
to push into Sematext as an event. In this release we’ve surfaced Events all the way to the top, so they now stand shoulder
to shoulder to metrics and logs.  
You can view logs in their own event stream-like view and you can search them of course.  Importantly, however, you can
also correlate events with metrics, as shown below, much like you can correlate logs and metrics. Why is
correlating events with metrics valuable? Think about being able to correlate a bad build or not quite fully tested release
deployment with a change in metric values. When it’s sudden and things break right away you don’t necessarily need any
tools to point out the correlation – you’ve got fire you need to put out and the correlation is obvious.

For such things, it is handy to be able to look at metrics and events and see the correlation. Sounds good?  
Log in to see for yourself and check it out!

Logsene, our hosted ELK stack now part of Sematext Cloud,
exposes the Elasticsearch API. If you have experience with Elasticsearch though, if you’re aware of its rich ecosystem of tools, you know how
valuable this is. When you send your logs to a Logsene app their structure (their mapping in Elasticsearch parlance) is automatically
detected. Log event fields are created and their types are deduced from the received data. The mapping and the index template are also
created. Under the hood, Logsene maintains a sequence of indices for each Logsene app. Every so often a new index is added and its mapping
is based on the index template for that Logsene app.

For more details on this check out our blog post
or see it for yourself.

Up until now, Sematext Cloud users had to first provide their credit card details under their
account and then manually assign that credit card to each monitoring or logging App they wanted to use with a paid plan. It turns out
this is a bit of a pain when you have a lot of Apps, so we’ve done away with that.  From now on, the payment method defined at the
account level is automatically applied to all Apps with a paid plan. This simplifies management, especially in cases when users
have many Apps (think about credit card expiring and having to assign a new card to each of the 57 Apps you have!).

For more details on this check out our blog post.

We’ve built and open-sourced an Android SDK for shipping logs
from mobile apps to Sematext Cloud. This library not only lets you send your Android app logs
for troubleshooting purposes, but it also lets you collect app usage for analytics purposes! In other words, shipping logs and
usage events from an Android app will give you the ability to both troubleshoot issues with the app, as well as let you gain insights
about your app’s usage, adoption, etc. The library sends data in batches to preserve battery (every 60s), or if there are
more than 10 events queued up.

For more info, check out our blog post and
see Sematext Logsene Android project on Github.

Want to ship logs and other events from your iOS apps? We’ve open-sourced the iOS SDK for
shipping logs from iOS apps to
Sematext Cloud. This library not only lets you send your iOS app logs for
troubleshooting purposes, but it also lets you collect app usage for analytics purposes!  In other words, shipping
logs and usage events from an iOS app will give you the ability to both troubleshoot issues with the app, as well
as let you gain insights about  your app’s usage, adoption, etc. If you don’t see the events in the dashboard
immediately, note that this library sends data in batches to preserve the battery (every 60s), or if there are
more than 10 messages queued up.

For more info, check out our blog post
and see Sematext Logsene iOS project on Github.