When we first released Heartbeat Alerts years ago, we made it possible to get alerted when a monitored service running on a host stopped sending metrics to Sematext. This works well in the static world of virtual machines and bare-metal servers. However, it isn’t suitable for the dynamic world of containers.
We have now added the ability to create Heartbeat Alerts based on any combination of tags. Meaning, Heartbeat Alerts can be tied to specific services regardless of where they are running. You can specify tags to include or exclude, as well as tags to group by, then create Heartbeat Alerts for services matching your criteria. This way, even if a containerized service is moved from one host to another, it will not trigger false alerts.
To use tag-based Heartbeat Alerts you’ll want to update the Sematext Agent to the latest version. It includes tags from the Sematext Common Schema that tag-based alerts use:
os.hostcontainer.idcontainer.image.namecontainer.image.tagcontainer.name
Using these tags you can do things like:
- Filtering by and grouping by container.image.name to get alerted when all containers with the specified image name are gone.
- Grouping by
container.idto get alerted when that particular container dies. Yes, this could lead to a lot of alert noise, so use this sparingly and wisely. - Grouping by
os.hostandcontainer.image.nameto get alerted when a host stops running all containers with some image name (container may have moved to another host)
We have also improved the Heartbeat Alerts preview, which now shows all “heartbeats” – one for each combination of selected tags – in a tooltip when you hover, as shown in the screenshot below.
Read more about creating Heartbeat alerts here.