Kubernetes控制平面組件：Kubelet詳解（五）：切換docker運行時為containerd

云原生學習路線導航頁（持續更新中）

kubernetes學習系列快捷鏈接
- Kubernetes架構原則和對象設計（一）
- Kubernetes架構原則和對象設計（二）
- Kubernetes架構原則和對象設計（三）
- Kubernetes控制平面組件：etcd（一）
- Kubernetes控制平面組件：etcd（二）
- Kubernetes控制平面組件：API Server詳解（一）
- Kubernetes控制平面組件：API Server詳解（二）
- Kubernetes控制平面組件：調度器Scheduler（一）
- Kubernetes控制平面組件：調度器Scheduler（二）
- Kubernetes控制平面組件：Kubelet詳解（一）：API接口層介紹
- Kubernetes控制平面組件：Kubelet詳解（二）：核心功能層
- Kubernetes控制平面組件：Kubelet詳解（三）：CRI 容器運行時接口層
- Kubernetes控制平面組件：Kubelet詳解（四）：gRPC 與 CRI gRPC實現
- Kubernetes控制平面組件：Kubelet詳解（五）：切換docker運行時為containerd
- Kubernetes控制平面組件：Kubelet 之 Static 靜態 Pod

本文是kubernetes的控制面組件Kubelet系列第五篇文章，本篇詳細講解了如何將集群的容器運行時從docker切換到containerd的全過程

希望大家多多點贊關注評論收藏，作者會更有動力繼續編寫技術文章

1.Docker 與 containerd 差異

在這里插入圖片描述

在 Kubernetes控制平面組件：Kubelet詳解（三）：CRI 容器運行時接口層中，我們學了不同運行時的差異。
- 使用 dockershim 作為運行時，kubelet調用dockershim后，需要依次經過dockershim、docker daemon、containerd之后，才能調用到oci runtime，非常厚重，而且docker本身包含很多kubernetes用不到的功能組件（如storage、networking等），由于代碼內置在kubernetes中而不得不攜帶
- 后來谷歌帶頭為cr定義了cri規范之后，docker不得不將container和image的部分抽取出來成為獨立的containerd組件，直接使用containerd作為運行時，去除了kubelet調用 dockershim和docker daemon 的兩層，效率大大提升
使用kubeadm安裝測試集群時，我們通常會使用docker作為運行時，本文將講述如何將集群的運行時，從docker切到containerd

2.切換docker為containerd

官方文檔：https://kubernetes.io/zh-cn/docs/setup/production-environment/container-runtimes/#containerd

2.1.Stop service

因為要改他們的配置，所以先把kubelet、docker、containerd都停掉

systemctl stop kubelet
systemctl stop docker
systemctl stop containerd

2.2.Create containerd config folder

2.2.1.生成containerd配置文件

創建containerd的配置文件目錄，生成 containerd 的默認配置文件并寫入系統配置路徑
提示：如果已有自定義配置，直接運行會丟失原有配置，建議先備份
```
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
```
- containerd config default
  - 調用 containerd 工具生成默認配置的 TOML 格式內容
  - 輸出到標準輸出（stdout）
  - |（管道）：將前一個命令的輸出傳遞給后續命令
- sudo tee /etc/containerd/config.toml
  - tee：同時輸出到屏幕并寫入文件
  - /etc/containerd/config.toml：containerd 的主配置文件路徑

2.2.2.container配置文件示例

生成 containerd 配置文件內容示例如下：

disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2[cgroup]path = ""[debug]address = ""format = ""gid = 0level = ""uid = 0[grpc]address = "/run/containerd/containerd.sock"gid = 0max_recv_message_size = 16777216max_send_message_size = 16777216tcp_address = ""tcp_tls_ca = ""tcp_tls_cert = ""tcp_tls_key = ""uid = 0[metrics]address = ""grpc_histogram = false[plugins][plugins."io.containerd.gc.v1.scheduler"]deletion_threshold = 0mutation_threshold = 100pause_threshold = 0.02schedule_delay = "0s"startup_delay = "100ms"[plugins."io.containerd.grpc.v1.cri"]device_ownership_from_security_context = falsedisable_apparmor = falsedisable_cgroup = falsedisable_hugetlb_controller = truedisable_proc_mount = falsedisable_tcp_service = trueenable_selinux = falseenable_tls_streaming = falseenable_unprivileged_icmp = falseenable_unprivileged_ports = falseignore_image_defined_volumes = falsemax_concurrent_downloads = 3max_container_log_line_size = 16384netns_mounts_under_state_dir = falserestrict_oom_score_adj = falsesandbox_image = "k8s.gcr.io/pause:3.5"selinux_category_range = 1024stats_collect_period = 10stream_idle_timeout = "4h0m0s"stream_server_address = "127.0.0.1"stream_server_port = "0"systemd_cgroup = falsetolerate_missing_hugetlb_controller = trueunset_seccomp_profile = ""[plugins."io.containerd.grpc.v1.cri".cni]bin_dir = "/opt/cni/bin"conf_dir = "/etc/cni/net.d"conf_template = ""ip_pref = ""max_conf_num = 1[plugins."io.containerd.grpc.v1.cri".containerd]default_runtime_name = "runc"disable_snapshot_annotations = truediscard_unpacked_layers = falseignore_rdt_not_enabled_errors = falseno_pivot = falsesnapshotter = "overlayfs"[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = ""[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options][plugins."io.containerd.grpc.v1.cri".containerd.runtimes][plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = "io.containerd.runc.v2"[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]BinaryName = ""CriuImagePath = ""CriuPath = ""CriuWorkPath = ""IoGid = 0IoUid = 0NoNewKeyring = falseNoPivotRoot = falseRoot = ""ShimCgroup = ""SystemdCgroup = false[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]base_runtime_spec = ""cni_conf_dir = ""cni_max_conf_num = 0container_annotations = []pod_annotations = []privileged_without_host_devices = falseruntime_engine = ""runtime_path = ""runtime_root = ""runtime_type = ""[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options][plugins."io.containerd.grpc.v1.cri".image_decryption]key_model = "node"[plugins."io.containerd.grpc.v1.cri".registry]config_path = ""[plugins."io.containerd.grpc.v1.cri".registry.auths][plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.headers][plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]tls_cert_file = ""tls_key_file = ""[plugins."io.containerd.internal.v1.opt"]path = "/opt/containerd"[plugins."io.containerd.internal.v1.restart"]interval = "10s"[plugins."io.containerd.internal.v1.tracing"]sampling_ratio = 1.0service_name = "containerd"[plugins."io.containerd.metadata.v1.bolt"]content_sharing_policy = "shared"[plugins."io.containerd.monitor.v1.cgroups"]no_prometheus = false[plugins."io.containerd.runtime.v1.linux"]no_shim = falseruntime = "runc"runtime_root = ""shim = "containerd-shim"shim_debug = false[plugins."io.containerd.runtime.v2.task"]platforms = ["linux/amd64"]sched_core = false[plugins."io.containerd.service.v1.diff-service"]default = ["walking"][plugins."io.containerd.service.v1.tasks-service"]rdt_config_file = ""[plugins."io.containerd.snapshotter.v1.aufs"]root_path = ""[plugins."io.containerd.snapshotter.v1.btrfs"]root_path = ""[plugins."io.containerd.snapshotter.v1.devmapper"]async_remove = falsebase_image_size = ""discard_blocks = falsefs_options = ""fs_type = ""pool_name = ""root_path = ""[plugins."io.containerd.snapshotter.v1.native"]root_path = ""[plugins."io.containerd.snapshotter.v1.overlayfs"]root_path = ""upperdir_label = false[plugins."io.containerd.snapshotter.v1.zfs"]root_path = ""[plugins."io.containerd.tracing.processor.v1.otlp"]endpoint = ""insecure = falseprotocol = ""[proxy_plugins][stream_processors][stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]path = "ctd-decoder"returns = "application/vnd.oci.image.layer.v1.tar"[stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]path = "ctd-decoder"returns = "application/vnd.oci.image.layer.v1.tar+gzip"[timeouts]"io.containerd.timeout.bolt.open" = "0s""io.containerd.timeout.shim.cleanup" = "5s""io.containerd.timeout.shim.load" = "5s""io.containerd.timeout.shim.shutdown" = "3s""io.containerd.timeout.task.state" = "2s"[ttrpc]address = ""gid = 0uid = 0

2.3.Update default config

上面生成了默認配置/etc/containerd/config.toml，其中有幾個值得關注的值

vi /etc/containerd/config.toml
sed -i s#k8s.gcr.io/pause:3.5#registry.aliyuncs.com/google_containers/pause:3.5#g /etc/containerd/config.toml
sed -i s#'SystemdCgroup = false'#'SystemdCgroup = true'#g /etc/containerd/config.toml

文件中有個配置sandbox_image，值為 k8s.gcr.io/pause:3.5，即容器sandbox的默認使用鏡像，k8s.gcr.io/pause:3.5 國內拉取不到，可以根據需要改成阿里云鏡像 registry.aliyuncs.com/google_containers/pause:3.5
還有個配置 SystemdCgroup，cgroup的driver，默認是false，需要開啟

2.4.Edit kubelet config and add extra args

containerd配置文件設置好了之后，還需要修改kubelet的配置，讓kubelet知道接下來運行時將會使用containerd

# 先通過查看kubelet服務，找到kubelet的服務配置文件路徑
[root@VM-226-235-tencentos ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node AgentLoaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)Drop-In: /usr/lib/systemd/system/kubelet.service.d└─10-kubeadm.confActive: inactive (dead) since Sat 2025-05-17 23:22:46 CST; 42min agoDocs: https://kubernetes.io/docs/# 編輯配置文件，添加或修改一條 包含 KUBELET_EXTRA_ARGS 的 Environment，表示額外的配置：--container-runtime=remote, --container-runtime-endpoint 為 containerd 的 sock
# 這個socket就是grpc發送的位置
vi /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.5"

2.5.Restart

都編輯好了之后，進行重啟。發現沒有再啟動docker，因為運行時里用不到了

systemctl daemon-reload
systemctl restart containerd
systemctl restart kubelet

2.6.Config crictl to set correct endpoint

創建crictl的文件，指定 containerd 的 sock

cat <<EOF | sudo tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
EOF

2.7.使用crictl操作容器和鏡像

# 獲取 pod sandbox 容器
[root@VM-226-235-tencentos ~]# crictl pods
POD ID              CREATED             STATE               NAME                                           NAMESPACE           ATTEMPT             RUNTIME
8f5a4d8d532af       20 minutes ago      Ready               nginx-sts-2                                    default             0                   (default)
f45055e2ebb27       21 minutes ago      Ready               nginx-deployment-585449566-rz58v               default             0                   (default)
606b6cec9e655       21 minutes ago      Ready               my-release-1-etcd-0                            default             0                   (default)