Live OpenSearch Online Training starting on October 12! See all classes


RancherOS Monitoring and Logging Support

RancherOS is one of the few “container only” operating systems and it evolved into an excellent orchestration tool for containers, competing e.g. with CoreOS. It supports several types of schedulers such as its own “Cattle” scheduler, as well as Kubernetes, Docker Swarm, and Mesos. A unique feature of RancherOS is its GUI for container orchestration based on templates for application stacks. Rancher Labs maintains a catalog for such templates and has integrated a community catalog, which includes Sematext Docker Agent for the collection of metrics, events and logs from all RancherOS cluster nodes and containers.

Monitoring all RancherOS nodes can be done several different ways, depending on which orchestration tool you use:

  • Deployment via rancher-compose for the whole RancherOS cluster using the Cattle scheduler
  • Deployment via the GUI (rancher server) using the Community Catalog entry (available for the Cattle scheduler)
  • Deployment as Kubernetes DaemonSet via kubectl for the Kubernetes scheduler
  • Deployment as Swarm global service using Swarm scheduler

This post provides the walk-throughs for all these deployment/orchestration options, with the common goal of collecting metrics, logs and events from each RancherOS node and all containers.

rancheros-sematext-docker-agent

Sematext Logging and Monitoring on RancherOS

Setup via Sematext Catalog Entry

When you run the Rancher server user interface, simply search in the community catalog for “sematext”, “monitoring” or “logs” and select “Sematext Docker Agent”.

Rancher-Catalog-monitoring-search

Sematext Docker Agent in RancherOS Community Catalog

Choose “View Details”, and in the “Configuration Options” enter the SPM and Logsene App tokens. You can obtain these from https://apps.sematext.com, where you can sign up and create your SPM and Logsene apps. If your Rancher cluster runs behind a firewall, you might need to specify the proxy URL in the HTTPS_PROXY or HTTP_PROXY environment variable.

Sematext-catalog-options

If you’d like to collect all logs, just press “Launch” without specifying any filter for containers or images.

You can find all details about the template on RancherOS blog New in Rancher Community Catalog: Monitoring and Logging by Sematext.

Setup via rancher-compose command

If you prefer rancher-compose over GUIs and use the Cattle scheduler then rancher-compose is the right tool to deploy Sematext Docker Agent. The following configuration will activate Sematext Docker Agent on every node in the RancherOS cluster. You’ll need to replace the SPM/Logsene App tokens, of course:

# sematext/docker-compose.yml
sematext-docker-agent:  
 image: 'sematext/sematext-agent-docker:latest'
 environment:
   - LOGSENE_TOKEN=YOUR_LOGSENE_TOKEN
   - SPM_TOKEN=YOUR_SPM_TOKEN 
 restart: always
 volumes:
   - /var/run/docker.sock:/var/run/docker.sock
 labels:
   io.rancher.scheduler.global: 'true'

Store the configuration as sematext/docker-compose.yml and activate the service:

rancher-compose up -f sematext/docker-compose.yml -d

Setup via Kubernetes DaemonSet

If you already deployed Kubernetes on RancherOS the Cattle scheduler should not be activated and you should use Kubernetes DaemonSet to deploy Sematext Docker Agent to all cluster nodes:

  1. Create a sematext-agent.yml file with the DaemonSet definition (just replace SPM/Logsene App tokens):
    apiVersion: extensions/v1beta1
    kind: DaemonSet
    metadata:
     name: sematext-agent
    spec:
     template:
       metadata:
         labels:
           app: sematext-agent
       spec:
         nodeSelector: {}
         dnsPolicy: "ClusterFirst"
         restartPolicy: "Always"
         containers:
         - name: sematext-agent
           image: sematext/sematext-agent-docker:latest
           imagePullPolicy: "Always"
           env:
           - name: SPM_TOKEN
             value: "YOUR_SPM_TOKEN"
           - name: LOGSENE_TOKEN
             value: "YOUR_LOGSENE_TOKEN"
           - name: KUBERNETES
             value: "1"
           volumeMounts:
             - mountPath: /var/run/docker.sock
               name: docker-sock
             - mountPath: /etc/localtime
               name: localtime
           securityContext:
             privileged: true
         volumes:
           - name: docker-sock
             hostPath:
               path: /var/run/docker.sock
           - name: localtime
             hostPath:
               path: /etc/localtime
    
  1. Activate the DaemonSet in the Kubernetes cluster:
    kubectl create -f sematext-agent.yml

Setup via Swarm global services

Similar to Kubernetes on RancherOS the Cattle scheduler is deactivated for Swarm as well. Thus, you can deploy Sematext Docker Agent as a global service on Swarm (>Docker engine 1.12). Connect you Docker client to RancherOS Swarm API endpoint and run the following global service definition with your SPM/Logsene App tokens. This will add Sematext Docker Agent to each Swarm node as soon it get launched.

docker service create --mode global --name sematext-agent-docker \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
-e SPM_TOKEN=YOUR_SPM_TOKEN -e LOGSENE_TOKEN=YOUR_LOGSENE_TOKEN \
sematext/sematext-agent-docker

Setup via Mesos Marathon

The following configuration will activate Sematext Docker Agent on every node in the Mesos cluster. Please note that you have to specify the number of Mesos nodes (instances), SPM App Token, and Logsene App Token. Example call to the Marathon API:

curl -XPOST -H "Content-type: application/json" http://your_marathon_server:8080/v2/apps -d '
{
"container": {
"type": "DOCKER",
"docker": {
"image": "sematext/sematext-agent-docker",
"privileged": true,
},
"volumes": [
{
"containerPath": "/var/run/docker.sock",
"hostPath": "/var/run/docker.sock",
"mode": "RW"
}
],
"network": "BRIDGE"
},
"env": {
"LOGSENE_TOKEN": "YOUR_LOGSENE_TOKEN",
"SPM_TOKEN": "YOUR_SPM_TOKEN"
},
"id": "sematext-agent-docker",
"instances": 1,
"cpus": 0.5,
"mem": 300,
"constraints": [["hostname","UNIQUE"]]
}

Summary

 

RancherOS provides many options and plays well with the most common orchestration tools.

Users can pick their preferred solution together with Sematext Docker Agent for the collection of metrics, events and logs. We have seen users moving from one orchestration platform to another when they hit some issues and having the flexibility to do so easily is extremely valuable in the rapidly changing container world.

As we hope you can see, RancherOS provides this flexibility, while Sematext Docker Agent support for all leading orchestration platforms ensures continuous visibility into Docker operational intelligence – metrics, events, and logs, regardless of which orchestration platform you use.

octi-sda

SIGN UP – FREE TRIAL

Start Free Trial