Throttling Pods in Kubernetes
Jun 08, 2023 • 0 Minute Read
If you've been learning about Kubernetes, chances are you're familiar with the concept of pods. Pods are an essential component in Kubernetes. That's why the "Pod of Minerva" (an interactive guide used in this Certified Kubernetes Administrator course), is standing for the guide to the pod.
You might notice from the screenshot above, there is more than just one pod running on this node. That's because flannel and kube-proxy will be running in the cluster as well, in the kube-system namespace. And that's where the total amount of requests comes from. From the screenshot above, the flannel pod is requesting 100m of CPU and the pod we just scheduled is requesting 800m of CPU. That gives us our total of 900m (90%) listed at the bottom.
To explicitly state the maximum amount of CPU and memory a pod can use, we can set limits in the pod YAML as well. Here's the same YAML from the first pod, but we've added limits:
The Pod
Pods are scheduled to machines or VMs, in which the scheduler (the control plane component) attempts to fit the pod on a VM, given it has enough CPU and memory to sustain the needs of the pod. For those of us who want to utilize all of the resources of our VMs, you may be asking "How can I tell Kubernetes to fit pods on my VM so that it's utilizing all the resources it has?" After all, most of us don't have the ability to create an infinite amount of VMs for our pods to live on and utilizing every square inch of the resources that we've paid for is not only cost-effective, but allows us to ensure our pods are receiving adequate resources to run, in turn, allowing our app to perform better.In this short tutorial, you'll learn how to use requests and limits for your pods in order to better organize how your pods are scheduled across your cluster. I aim to convince you that adding this to every pod definition is a great idea that offers you peace of mind that will add value to your application running on Kubernetes.Requests and Limits
When creating a pod, you can specify the amount of CPU and memory a container needs, called the requests. You can also set a limit on the amount of CPU and memory a container should use, which is called limits. Setting the requests and limits becomes very important when considering the JVM heap size for Java applications. Most likely, if not set correctly, the JVM process will be killed by the systems OOM killer.Let's go ahead and create a pod with just requests and I'll explain what this is doing for the pod. First, create the following pod YAML that will specify the image to use and the request amounts. If you don't have access to your own cluster, get started with Linux Academy's free community edition (for limited access).apiVersion: v1kind: Podmetadata:name: pod1spec:containers:- image: busyboxcommand: ["dd", "if=/dev/zero", "of=/dev/null"]name: pod1resources:requests:cpu: 800mmemory: 20MiGo ahead and run the following command to create the pod:
kubectl create -f pod.yamlTo see this pod and which node it's running on, enter the following command:
kubectl get pods -o wideTo get more information about the node and its utilized resources, enter the following command:
kubectl describe node [node_name]Note: If you don't know the node name, type the following to get it:
kubectl get nodesAt the bottom of the output from the kubectl describe node command, you will notice the requests and limits, similar to this:
You might notice from the screenshot above, there is more than just one pod running on this node. That's because flannel and kube-proxy will be running in the cluster as well, in the kube-system namespace. And that's where the total amount of requests comes from. From the screenshot above, the flannel pod is requesting 100m of CPU and the pod we just scheduled is requesting 800m of CPU. That gives us our total of 900m (90%) listed at the bottom.
The Node CPU Resource
You'd think that because each of these pods has requested an amount, they will only use the requested amount and that's it. That is not true. Actually, both pods will most likely use all the remaining CPU if required. But, they don't just split it evenly. The remaining CPU is split in a 1 to 5 ratio. The first pod will get one-sixth of the CPU time and the other will get the remaining five-sixths.To explicitly state the maximum amount of CPU and memory a pod can use, we can set limits in the pod YAML as well. Here's the same YAML from the first pod, but we've added limits:
apiVersion: v1kind: Podmetadata:name: pod1spec:containers:- image: busyboxcommand: ["dd", "if=/dev/zero", "of=/dev/null"]name: pod1resources:requests:cpu: 800mmemory: 20Milimits:cpu: 800mmemory: 20MiTo apply these changes, you'll have to delete the existing pod and re-create it using the modified YAML. Use the following command to delete the pod:
kubectl delete pods pod1Then, you can create the pod with the new YAML using the following command:
kubectl create -f pod.yamlWe can check if the pod is running by entering the following command:
kubectl get pods -o wideNow that our pod is running, it has requested 800m CPU and 20Mi of memory, and because we set the limit of the same, it will use the requested amount and nothing more.
More on Pod CPU and Memory
This has been a short tutorial on setting CPU and memory requests and limits in Kubernetes. If you'd like to know more about scheduling pods in Kubernetes, check out the Cloud Native Certified Kubernetes Administrator (CKA) course on Linux Academy or check out these free resources below to learn more about Kubernetes: