Definition: What Is Quality of Service (QoS) in Kubernetes?
Quality of Service (QoS) is a classification criterion in Kubernetes that determines how to schedule and manage Pods in the ecosystem. QoS prioritizes different applications based on their resource requirements. Kubernetes goes beyond container orchestration to process an application’s resources management and scheduled runtime. It allows you to specify resources, e.g., CPU/memory for your apps, in terms of requests and limits. The Pod YAML file indicates all the resources allocated to each Pod. QoS in Kubernetes ensures the node has enough resources to run the Pod.
Kubernetes QoS evaluates the CPU and memory configuration of the Pod. Kubernetes schedules a QoS class that determines how to schedule and evict Pods. For example, if resources in the node are scarce, Kubernetes uses the QoS class to determine which pod to evict first. When deployed, the user schedules resource requests and sets limits. These parameters determine the QoS class level assigned to each Pod.
Resource Types: Requests and Limits
When you deploy a pod, Kubernetes assigns a QoS class to the pod. Quality of Service determines the right class based on the requests and limit parameters of the CPU and memory resources.
A request is the amount of resource allocated to the container, while a limit is the maximum amount of resource the system allows the container to use. The difference between the requests and limits lies in scheduling. The Kube-scheduler usually calculates a QoS class according to the value of requests. With memory, the Kube-scheduler only uses the set memory to make judgments. The type of request and limit enforced determines whether the resource will be compressible or incompressible.
Kubernetes measures CPU resources in (v)Core equivalents and fractions. Meaning 0.5 is equivalent to half a core or in milicpu, e.g., 500m meaning half of the computing power of a CPU. Memory resources, on the other hand, are measured in bytes. You can specify them as decimals with one of SI suffixes.
Compressible Resource vs Incompressible Resource Guarantees
Resource requests and limits have different implications on the QoS. Resources can be classified into two different types, compressible and incompressible.
Compressible Resource
In Kubernetes, the CPU is a compressible resource. This means Kubernetes won’t evict the Pod even if your app exceeds the CPU limits. The worst that can happen is for Kubernetes to throttle your container, affecting the application’s performance.
If the limit is not specified, the Pods can use the excess CPU available.
Incompressible Resource
Kubernetes QoS considers memory as an incompressible resource. When a Pod runs out of memory, the kernel evicts it. Kubernetes only supports available memory.
The Pods only get the amount of memory set for them in requests; if they exceed their requests, Kubernetes kills them. This can happen if another pod with more containers requires more memory. But if they consume less, the kernel won’t kill them.
Kubernetes QoS class closely monitors how to manage and distribute resources. In cases where the system runs out of resources, Kubernetes could kill the containers considered less important to the application.
Kubernetes QoS Classes
Kubernetes categorizes Pods into three QoS classes: Guaranteed, Burstable, and BestEffort. The type of class determines the order of priority for eviction.
Guaranteed QoS Class
Kubernetes considers Pods classified as Guaranteed as a top priority. It won’t evict them until they exceed their limits. A Pod with a Guaranteed class has the following characteristics:
- All containers in the Pod have a memory limit and request.
- All containers in the Pod have a memory limit equal to the memory request.
- All containers in the Pod have a CPU limit and a CPU request.
- All containers in the Pod have a CPU limit equal to the CPU request.
For example, if the memory limit is 300MiB, the memory request should also be 300MiB. While the CPU limit is 800miliCPU, it should equal the CPU request. If a container is only assigned a resource limit without the request, Kubernetes will automatically match an equal number to the request value.
Software that runs full-time, such as database servers and real-time applications, is often assigned the Guaranteed QoS class.
The resources section in the YAML file will look like so:
resources:
limits:
memory: "300Mi"
cpu: "800m"
requests:
memory: "300Mi"
cpu: "800m"
Burstable QoS Class
Kubernetes assigns the Burstable class to a Pod when a container in the pod has more resource limit than the request value. A pod in this category will have the following characteristics:
- The Pod has not met the criteria for Guaranteed QoS class.
- A container in the Pod has an unequal memory or CPU request or limit
For example, if a container in the Pod has a memory limit of 300MiB and a memory request of 100MiB, Kubernetes will classify it as Burstable. If the node runs out of resources, this Pod is the second priority for eviction.
Common use cases for this class include websites and email systems that do not use many resources.
The resource section on the YAML file will look like so:
resources:
limits:
memory: "300Mi"
cpu: "800m"
requests:
memory: "100Mi"
cpu: "600m"
BestEffort QoS Class
Pods characterized in the QoS class of BestEffort, have containers with no resource limits or requests set. In the configuration file, the resources are not allocated for either memory or CPU. BestEffort Pods use resources available in the node.
Kubernetes considers such Pods as low priority. It will evict them first in case resources are scarce in the node. Kubernetes assigns BestEffort class to applications that can experience downtimes, like test systems and non-critical applications like video processing.
The YAML file will not have a resource section. It will look like so:
apiVersion: v1
kind: Pod
metadata:
name: qos-demo-3
namespace: qos-example
spec:
containers:
- name: qos-demo-3-ctr
image: nginx
Why Are QoS Classes Important in Kubernetes?
Kubernetes QoS classes are important when monitoring host resources. In cases where the host’s resources are limited or depleted, kubelet will conduct eviction of some pods. Eviction frees resources for Pods that are of high priority to run effectively. It also prevents the scheduling of new pods to the depleted host.
During the eviction, kubelet selects the Pods to evict in order of QoS class. Pods classified as BestEffort will be first evicted. Secondly, Pods in the Burstable category with limited resources will be next. Last to evict are the Pods classified as Guaranteed. Kubernetes evicts Pods in the Guaranteed class once they exceed their limits or the host schedules to delete them.
You can set an eviction mode for the Pods, also known as Soft eviction, or allow Kubernetes to start Hard eviction once it reaches a set threshold.
[product_banner type=”kubernetes”]Effortlessly monitor your entire Kubernetes infrastructure in minutes.[/product_banner]
How to Configure Quality of Service for Pods?
To configure QoS for Pods in Kubernetes, you need to set up a cluster and at least two nodes. Next, follow the following instructions:
1. Create a Namespace
You need a namespace to isolate the resources you create in the project from the rest of the cluster. Create a namespace by running the following command:
kubectl create namespace qos-example
2. Create a Pod With a QoS Class
Kubernetes categorizes all Pods in QoS classes. To create one for the Guaranteed class create a folder named examples that has a folder named pods. The pods folder should have another folder in it named qos. Then create a YAML file named qos-pods.yaml with the following configuration:
apiVersion: v1
kind: Pod
metadata:
name: qos-demo
namespace: qos-example
spec:
containers:
- name: qos-demo-ctr
image: nginx
resources:
limits:
memory: "300Mi"
cpu: "800m"
requests:
memory: "300Mi"
cpu: "800m"
Then run the following command to create the Pod:
kubectl apply -f https://k8s.io/examples/pods/qos/qos-pod.yaml --namespace=qos-example
To see details about the Pod including it’s QoS class, run the following command:
kubectl get pod qos-demo --namespace=qos-example --output=yaml
containerStatuses:
- containerID: containerd://91687c7c9949d7c32d6c604dda3b3e34c66cf59adf6f38dca341b2e720b0351e
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:b8f2383a95879e1ae064940d9a200f67a6c79e710ed82ac42263397367e7cc4e
lastState: {}
name: qos-demo-ctr
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2023-01-13T08:27:57Z"
hostIP: 172.30.2.2
phase: Running
podIP: 192.168.1.3
podIPs:
- ip: 192.168.1.3
qosClass: Guaranteed
startTime: "2023-01-13T08:27:50Z"
You should see the Pod details, and it’s QoS class as Guaranteed. This is because you set its memory and CPU resources with the same values for limits and requests. Use the YAML file to determine the QoS class of your Pods depending on the resources you allocate to each.
You can create pods for each of the other classes by changing the value of the requests and limits. For example, to create a pod for the Burstable QoS class, allocate more resource limits than the number of requests. Kubernetes will automatically classify it in the Burstable class.
….
3. Delete a Pod
To delete a Pod, run the following command:
kubectl delete pod qos-demo --namespace=qos-example
If successful, Kubernetes will notify you that it deleted the Pod.
Kubernetes Monitoring with Sematext
To ensure proper Kubernetes QoS, you need a reliable monitoring and alerting solution that can detect performance anomalies and notify you in real time. Sematext Monitoring is a performance monitoring solution with Kubernetes monitoring capabilities, which provides valuable insights into the health of your cluster and its nodes, and monitors pods, deployments, and services.
Sematext comes with pre-built and customizable dashboards that give you access to all the Kubernetes metrics needed for efficient Kubernetes QoS monitoring. These include container metrics like CPU, memory, and disk I/O, as well as network utilization statistics.
You can set up alert rules for each of these metrics and use the tool’s anomaly detection feature to receive notifications whenever issues arise within your Kubernetes environment. Sematext offers multiple notification channels, including email, Slack, and PagerDuty, ensuring that you can intervene immediately after issues are found before they escalate.
Start your 14-day free trial or watch the video below to learn more about how easy is to use Sematext for Kubernetes monitoring.