Permanently Disable Swap for Kubernetes Cluster

 2 years ago
source link: https://brandonwillmott.com/2020/10/15/permanently-disable-swap-for-kubernetes-cluster/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Permanently Disable Swap for Kubernetes Cluster

I set up a few Kubernetes clusters preparing for the CKA exam and discovered that upon reboot, my control plane node didn’t work. Running any kubectl command I get:

The connection to the server was refused - did you specify the right host or port?

This is a common error message that can result from numerous issues but I started troubleshooting by seeing if the kubelet process was running with ps aux | grep kube. Nothing…no kubelet service running. Then I looked at /var/log/syslog and I found the problem — swap was back on!

Oct 15 06:23:13 master-02 kubelet[4657]: F1015 06:23:13.446790 4657 server.go:265] failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename#011#011#011#011Type#011#011Size#011Used#011Priority /dev/dm-1 partition#0112097148#0110#011-2]

It turns out even though I turned the swap off with swapoff -a, that’s only a temporary solution for disabling the swap. Upon reboot, it will be enabled due to it being referenced in /etc/fstab:

/dev/mapper/ubuntu--vg-swap_1 none swap sw 0 0

To disable permanently, you can run the following command:

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

This command will insert a comment on the line referencing swap in /etc/fstab but will not take effect immediately so a reboot will be necessary. If you want to avoid a reboot, you can also run sudo swapoff -a again and wait for the kubelet process to retry again or force the service to restart with sudo systemctl restart kubelet.

The topic of Kubernetes not supporting linux memory management techniques like paging to disk has puzzled many people and there’s a good discussion about it in a GitHub issue that was raised: Kubelet/Kubernetes should work with Swap Enabled. Suffice to say, swap support isn’t changing anytime soon.

This entry was posted in Kubernetes: 0-7 and tagged cka, fstab, k8s, kubectl, kubelet, kubernetes, swap, swapoff on October 15, 2020.

About Joyk

Aggregate valuable and interesting links.
Joyk means Joy of geeK