K8s 鏡像緩存管理 kube-fledged 認知

寫在前面

  • 博文內容為K8s 鏡像緩存管理 kube-fledged 認知
  • 內容涉及：
    • kube-fledged 簡單介紹
    • 部署以及基本使用
  • 理解不足小伙伴幫忙指正

不必太糾結于當下，也不必太憂慮未來，當你經歷過一些事情的時候，眼前的風景已經和從前不一樣了。——村上春樹

簡單介紹

我們知道 k8s 上的容器調度需要在調度的節點行拉取當前容器的鏡像，在一些特殊場景中，

  • 需要快速啟動和/或擴展的應用程序。例如，由于數據量激增，執行實時數據處理的應用程序需要快速擴展。
  • 鏡像比較龐大，涉及多個版本，節點存儲有限，需要動態清理不需要的鏡像
  • 無服務器函數通常需要在幾分之一秒內立即對傳入事件和啟動容器做出反應。
  • 在邊緣設備上運行的 IoT 應用程序，需要容忍邊緣設備和鏡像鏡像倉庫之間的間歇性網絡連接。
  • 如果需要從專用倉庫中拉取鏡像，并且無法授予每個人從此鏡像倉庫拉取鏡像的訪問權限，則可以在群集的節點上提供鏡像。
  • 如果集群管理員或操作員需要對應用程序進行升級，并希望事先驗證是否可以成功拉取新鏡像。

kube-fledged 是一個 kubernetes operator，用于直接在 Kubernetes 集群的 worker 節點上創建和管理容器鏡像緩存。它允許用戶定義鏡像列表以及這些鏡像應緩存到哪些工作節點上（即拉取）。因此，應用程序 Pod 幾乎可以立即啟動，因為不需要從鏡像倉庫中提取鏡像。

kube-fledged 提供了 CRUD API 來管理鏡像緩存的生命周期，并支持多個可配置的參數，可以根據自己的需要自定義功能。

Kubernetes 具有內置的鏡像垃圾回收機制。節點中的 kubelet 會定期檢查磁盤使用率是否達到特定閾值（可通過標志進行配置）。一旦達到這個閾值，kubelet 會自動刪除節點中所有未使用的鏡像。

需要在建議的解決方案中實現自動和定期刷新機制。如果鏡像緩存中的鏡像被 kubelet 的 gc 刪除，下一個刷新周期會將已刪除的鏡像拉入鏡像緩存中。這可確保鏡像緩存是最新的。

設計流程

https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png

部署 kube-fledged

Helm 方式部署

──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$mkdir  kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$cd kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$export KUBEFLEDGED_NAMESPACE=kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE}
namespace/kube-fledged created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/
"kubefledged-charts" has been added to your repositories
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubefledged-charts" chart repository
...Successfully got an update from the "kubescape" chart repository
...Successfully got an update from the "rancher-stable" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "openkruise" chart repository
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "botkube" chart repository
Update Complete. ?Happy Helming!?

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait

實際部署中發現，由于網絡問題，chart 無法下載，所以通過 make deploy-using-yaml 使用 yaml 方式部署

Yaml 文件部署

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$git clone https://github.com/senthilrch/kube-fledged.git
正克隆到 'kube-fledged'...
remote: Enumerating objects: 10613, done.
remote: Counting objects: 100% (1501/1501), done.
remote: Compressing objects: 100% (629/629), done.
remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112
接收對象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done.
處理 delta 中: 100% (4431/4431), done.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$ls
kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$cd kube-fledged/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml

第一次部署，發現鏡像拉不下來

┌──[root@vms100.liruilongs.github.io]-[~]
└─$kubectl get all -n kube-fledged
NAME                                               READY   STATUS                  RESTARTS         AGE
pod/kube-fledged-controller-df69f6565-drrqg        0/1     CrashLoopBackOff        35 (5h59m ago)   21h
pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2   0/1     Init:CrashLoopBackOff   35 (5h58m ago)   21h
pod/kubefledged-controller-55f848cc67-7f4rl        1/1     Running                 0                21h
pod/kubefledged-webhook-server-597dbf4ff5-l8fbh    0/1     Init:CrashLoopBackOff   34 (6h ago)      21hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   21h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   21hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       0/1     1            0           21h
deployment.apps/kube-fledged-webhook-server   0/1     1            0           21h
deployment.apps/kubefledged-controller        0/1     1            0           21h
deployment.apps/kubefledged-webhook-server    0/1     1            0           21hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         0       21h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         0       21h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         0       21h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         0       21h
┌──[root@vms100.liruilongs.github.io]-[~]
└─$

這里我們找一下要拉取的鏡像

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat *.yaml | grep image:- image: senthilrch/kubefledged-controller:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0

單獨拉取一些，當前使用 ansible 在所有工作節點批量操作

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml

其他相關的鏡像都拉取一下

操作完成之后容器狀態全部正常

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl -n kube-fledged get all
NAME                                               READY   STATUS    RESTARTS   AGE
pod/kube-fledged-controller-df69f6565-wdb4g        1/1     Running   0          13h
pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp   1/1     Running   0          13h
pod/kubefledged-controller-55f848cc67-klxlm        1/1     Running   0          13h
pod/kubefledged-webhook-server-597dbf4ff5-ktbsh    1/1     Running   0          13hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   36h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   36hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       1/1     1            1           36h
deployment.apps/kube-fledged-webhook-server   1/1     1            1           36h
deployment.apps/kubefledged-controller        1/1     1            1           36h
deployment.apps/kubefledged-webhook-server    1/1     1            1           36hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         1       36h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         1       36h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         1       36h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         1       36h

驗證是否安裝成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get pods -n kube-fledged -l app=kubefledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-klxlm       1/1     Running   0          16h
kubefledged-webhook-server-597dbf4ff5-ktbsh   1/1     Running   0          16h
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.

使用 kubefledged

創建鏡像緩存對象

根據 Demo 文件，創建鏡像緩存對象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$cd deploy/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- ghcr.io/jitesoft/nginx:1.23.1# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- us.gcr.io/k8s-artifacts-prod/cassandra:v7- us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0nodeSelector:tier: backend# Specifies a list of image pull secrets to pull images from private repositories into the cacheimagePullSecrets:- name: myregistrykey

官方的 Demo 中對應的 鏡像拉取不下來，所以換一下

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7
Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

為了測試選擇器標簽的使用,我們找一個節點的標簽單獨做鏡像緩存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get nodes  --show-labels

同時我們直接從公有倉庫拉取鏡像，所以不需要 imagePullSecrets 對象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml

修改后的 yaml 文件

  • 添加了一個所有節點的 liruilong/my-busybox:latest 鏡像緩存
  • 添加了一個 kubernetes.io/hostname: vms105.liruilongs.github.io 對應標簽選擇器的 liruilong/hikvision-sdk-config-ftp:latest 鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

直接創建報錯了

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io")
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

解決辦法，刪除對應的對象，重新創建

我在當前項目的一個 issues 下面找到了解決辦法 https://github.com/senthilrch/kube-fledged/issues/76

看起來這是因為 Webhook CA 是硬編碼的，但是當 webhook 服務器啟動時，會生成一個新的 CA 捆綁包并更新 webhook 配置。當發生另一個部署時，將重新應用原始 CA 捆綁包，并且 Webhook 請求開始失敗，直到再次重新啟動 Webhook 組件以修補捆綁包init-server

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make remove-kubefledged-and-operator
# Remove kubefledged
kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml
error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2"
ensure CRDs are installed first

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
namespace/kube-fledged created
kubectl apply -f deploy/kubefledged-crd.yaml
customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged
....................
kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch
Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available...
deployment "kubefledged-webhook-server" successfully rolled out
kubectl get pods -n kube-fledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-76c4v       1/1     Running   0          112s
kubefledged-webhook-server-597dbf4ff5-56h6z   1/1     Running   0          66s

重新創建緩存對象，創建成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
imagecache.kubefledged.io/imagecache1 created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
NAME          AGE
imagecache1   10s
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

查看當前被納管的鏡像緩存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl get imagecaches imagecache1 -n kube-fledged -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 83,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20169836","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:06:47Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheRefresh","startTime": "2024-03-02T01:05:33Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$

通過 ansible 來驗證

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | CHANGED | rc=0 >>
liruilong/hikvision-sdk-config-ftp                                          latest            a02cd03b4342   4 months ago    830MB
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

開啟自動刷新

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=
imagecache.kubefledged.io/imagecache1 annotated
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

添加鏡像緩存

添加一個新的鏡像緩存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 92,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175233","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest","liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:43:32Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:40:34Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

通過 ansible 確認

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.102 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest                                      17dbd4002a8c   5 years ago     170MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

刪除鏡像緩存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 94,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175766","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"message": "Image cache is being updated. Please view the status after some time","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:48:03Z","status": "Processing"}
}

通過 Ansible 確認，可以看到無論是 mastere 上的節點還是 work 的節點，對應的鏡像緩存都被清理

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

這里需要注意如果清除所有的鏡像緩存，那么需要把 images 下的數組 寫成 “”.

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 98,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20176849","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": [""]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:52:16Z","message": "All cached images succesfully deleted from respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:51:47Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

如果通過下面的方式刪除，直接注釋調對應的標簽

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster#- images:#- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

那么會報下面的錯

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists
You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.

博文部分內容參考

? 文中涉及參考鏈接內容版權歸原作者所有，如有侵權請告知,如果你認可它不要吝嗇星星哦 😃

https://github.com/senthilrch/kube-fledged

? 2018-2024 liruilonger@gmail.com, All rights reserved. 保持署名-非商用-相同方式共享(CC BY-NC-SA 4.0)

本文來自互聯網用戶投稿，該文觀點僅代表作者本人，不代表本站立場。本站僅提供信息存儲空間服務，不擁有所有權，不承擔相關法律責任。
如若轉載，請注明出處：http://www.pswp.cn/news/716417.shtml
繁體地址，請注明出處：http://hk.pswp.cn/news/716417.shtml
英文地址，請注明出處：http://en.pswp.cn/news/716417.shtml

如若內容造成侵權/違法違規/事實不符，請聯系多彩編程網進行投訴反饋email:809451989@qq.com，一經查實，立即刪除！

相關文章

springboot236基于springboot在線課程管理系統的設計與實現

springboot236基于springboot在線課程管理系統的設計與實現

基于SpringBoot在線課程管理系統的設計與實現 摘要 本文首先介紹了在線課程管理系統的現狀及開發背景&#xff0c;然后論述了系統的設計目標、系統需求、總體設計方案以及系統的詳細設計和實現&#xff0c;最后對在線課程管理系統進行了系統檢測并提出了還需要改進的問題。本系…
閱讀更多...
Spring Data Redis 使用方式

Spring Data Redis 使用方式

在Java中操作Redis 1. 在Java中操作Redis1.1 Redis的Java客戶端1.2 Spring Data Redis使用方式1.2.1 介紹1.2.1 環境搭建1.2.3 操作常見類型數據 1. 在Java中操作Redis 1.1 Redis的Java客戶端 Redis 的 Java 客戶端很多&#xff0c;常用的幾種&#xff1a; JedisLettuceSpri…
閱讀更多...
關于Axios接口請求超時處理與重試的方法教程

關于Axios接口請求超時處理與重試的方法教程

在前端開發中&#xff0c;使用Axios作為HTTP客戶端庫進行接口請求是非常常見的做法。然而&#xff0c;在實際開發中&#xff0c;我們經常會遇到網絡不穩定或服務器響應緩慢導致接口請求超時的情況。為了提高用戶體驗和程序的穩定性&#xff0c;我們需要實現接口請求超時的處理與…
閱讀更多...
UnityAPI的學習——Transform類

UnityAPI的學習——Transform類

Transform類繼承自Component類&#xff0c;并實現了IEnumberable接口。Transform是GameObject必須擁有得一個組件&#xff0c;用來管理所在GameObject對象的坐標位置、選擇角度、和大小縮放。 Transform實現了IEnumberable接口&#xff0c;因此可以在程序中使用foreach()方法快…
閱讀更多...
echarts vue 動畫效果的水球圖、波浪圖教程

echarts vue 動畫效果的水球圖、波浪圖教程

1、安裝插件 前提是已經安裝了echarts&#xff08;我的版本是4.2.1&#xff09; npm install echarts-liquidfill --save 我安裝了3.1.0版本的&#xff0c;結果運行時報錯"TypeError: wave.ensureState is not a function" 原因&#xff1a;echarts版本和echarts-l…
閱讀更多...
miniconda3徹底刪除虛擬環境

miniconda3徹底刪除虛擬環境

退出虛擬環境&#xff1a;確保您不在要刪除的虛擬環境中。如果在&#xff0c;使用命令 conda deactivate 來退出當前激活的虛擬環境。查看虛擬環境列表&#xff1a;運行命令 conda env list 或 conda info -e 來查看所有存在的虛擬環境及其路徑。刪除虛擬環境&#xff1a;使用命…
閱讀更多...
在VMware中安裝CentOS 7并配置Docker

在VMware中安裝CentOS 7并配置Docker

VMware安裝CentOS 7 一、介紹 該文章介紹如何使用啟動U盤在虛擬機里面安裝系統&#xff0c;虛擬機版本為VMware Workstation 16 pro&#xff0c;Linux版本為CentOS Linux release 7.9.2009 (Core)。 二、安裝 1、創建虛擬機 點擊創建新的虛擬機 選擇典型就可以了&#xf…
閱讀更多...
前綴和算法題（區間次方和、小藍平衡和、大石頭的搬運工、最大數組和）

前綴和算法題（區間次方和、小藍平衡和、大石頭的搬運工、最大數組和）

一、前綴和的原理和特點 prefix表示前綴和&#xff0c;前綴和由一個用戶輸入的數組生成。對于一個數組a[]&#xff08;下標從1開始&#xff09;&#xff0c;我們定義一個前綴和數組prefix[]&#xff0c;滿足&#xff1a; prefix有一個重要的特性&#xff0c;可以用于快速生成p…
閱讀更多...
WordPress建站入門教程：如何安裝本地WordPress網站運行環境？

WordPress建站入門教程：如何安裝本地WordPress網站運行環境？

有些站長想要搭建WordPress網站&#xff0c;又擔心自己玩不轉&#xff0c;白白浪費購買域名和主機空間的費用。像這種情況&#xff0c;最好的做法就是在自己電腦上安裝一個WordPress網站運行環境&#xff0c;然后在本地電腦搭建WordPress&#xff0c;等熟悉掌握后再考慮購買域名…
閱讀更多...
設計模式學習筆記——抽象工廠模式

設計模式學習筆記——抽象工廠模式

設計模式&#xff08;創建型&#xff09;—— 抽象工廠模式 在工廠模式中&#xff0c;我們為每一個類都設計了一個工廠&#xff0c;以此來獲取該類的對象&#xff0c;但缺點就是一旦類多了&#xff0c;工廠就多了&#xff0c;這時候我們可以考慮這些類間是否有關聯&#xff0c…
閱讀更多...
多輸入多輸出 | MATLAB實現GWO-Elman灰狼優化循環神經網絡多輸入多輸出預測

多輸入多輸出 | MATLAB實現GWO-Elman灰狼優化循環神經網絡多輸入多輸出預測

多輸入多輸出 | MATLAB實現GWO-Elman灰狼優化循環神經網絡多輸入多輸出預測 目錄 多輸入多輸出 | MATLAB實現GWO-Elman灰狼優化循環神經網絡多輸入多輸出預測預測效果基本介紹程序設計往期精彩參考資料 預測效果 基本介紹 Matlab實現GWO-Elman灰狼優化循環神經網絡多輸入多輸出…
閱讀更多...
kernel bypass 是什么？

kernel bypass 是什么？

文章目錄 一、kernel bypass 是什么二、Kernel Bypass技術優缺點三、Kernel Bypass技術應用領域四、Kernel Bypass的實現方式 一、kernel bypass 是什么 Kernel Bypass是一種技術&#xff0c;旨在通過繞過操作系統核來提高網絡數據包處理的性能和降低延遲。它的主要優點是高性能…
閱讀更多...
[LeetBook]【學習日記】有序鏈表合并

[LeetBook]【學習日記】有序鏈表合并

21. 合并兩個有序鏈表 將兩個升序鏈表合并為一個新的 升序 鏈表并返回。新鏈表是通過拼接給定的兩個鏈表的所有節點組成的。 示例 1&#xff1a; 輸入&#xff1a;l1 [1,2,4], l2 [1,3,4] 輸出&#xff1a;[1,1,2,3,4,4] 示例 2&#xff1a; 輸入&#xff1a;l1 [], l2 [] …
閱讀更多...
如何在電腦上中恢復已刪除的視頻

如何在電腦上中恢復已刪除的視頻

您可以在電腦中恢復已刪除的視頻&#xff0c;無需任何繁瑣的工作。您所需要做的就是閱讀本文&#xff0c;了解恢復已刪除視頻的最佳方法。 一次錯誤的點擊可能會奪走您以視頻形式存儲的寶貴記憶。嗯&#xff0c;有些視頻不適合刪除&#xff0c;您希望永遠保留它們。失去這些寶…
閱讀更多...
如何使用Docker搭建StackEdit編輯器并結合內網穿透實現遠程辦公

如何使用Docker搭建StackEdit編輯器并結合內網穿透實現遠程辦公

文章目錄 前言1. ubuntu安裝VNC2. 設置vnc開機啟動3. windows 安裝VNC viewer連接工具4. 內網穿透4.1 安裝cpolar【支持使用一鍵腳本命令安裝】4.2 創建隧道映射4.3 測試公網遠程訪問 5. 配置固定TCP地址5.1 保留一個固定的公網TCP端口地址5.2 配置固定公網TCP端口地址5.3 測試…
閱讀更多...
優選算法|【雙指針】|1089.復寫零

優選算法|【雙指針】|1089.復寫零

目錄 題目描述 題目解析 算法原理講解 代碼 題目描述 1089. 復寫零 給你一個長度固定的整數數組 arr &#xff0c;請你將該數組中出現的每個零都復寫一遍&#xff0c;并將其余的元素向右平移。 注意&#xff1a;請不要在超過該數組長度的位置寫入元素。請對輸入的數組 就…
閱讀更多...
LeetCode受限條件下可到達節點的數目

LeetCode受限條件下可到達節點的數目

題目描述 現有一棵由 n 個節點組成的無向樹&#xff0c;節點編號從 0 到 n - 1 &#xff0c;共有 n - 1 條邊。 給你一個二維整數數組 edges &#xff0c;長度為 n - 1 &#xff0c;其中 edges[i] [ai, bi] 表示樹中節點 ai 和 bi 之間存在一條邊。另給你一個整數數組 restr…
閱讀更多...
OJ：移除鏈表元素

OJ：移除鏈表元素

203. 移除鏈表元素 - 力扣&#xff08;LeetCode&#xff09; 思路&#xff1a;這個題可以直接在原鏈表上進行修改&#xff0c;但是修改鏈表的指向是有點麻煩的&#xff0c;所以我們給兩個指針&#xff0c;phead和ptail,這是新鏈表的兩個指針&#xff0c;再給一個指針pcur來遍歷…
閱讀更多...
Java和JavaScript區別

Java和JavaScript區別

1. Java和javaScript都是面向對象語言 2. 他兩除了名字相似之外沒有任何關系 3. Java是一種真正的面向對象語言&#xff0c;不管開發什么程序都要設計對象&#xff1b;而JavaScript是種腳本語言&#xff0c;主要實現前端頁面的交互&#xff0c;比如驗證表單&#xff0c;彈窗提…
閱讀更多...
Sqli-labs靶場第12關詳解[Sqli-labs-less-12]

Sqli-labs靶場第12關詳解[Sqli-labs-less-12]

Sqli-labs-Less-12 #手工注入 post傳參了 根據題目看&#xff0c;像一個登錄頁面&#xff0c;嘗試使用布爾型盲注測試能否登錄網站 1. Username輸入a a" 測試是否會有報錯&#xff0c;burp抓包 報錯&#xff1a;syntax to use near "a"") and passw…
閱讀更多...
最新文章

Copyright @ 2022~2023