參考 文章：Helm部署Harbor_helm harbor_風向決定發型丶的博客-CSDN博客

安裝好后使用 nerd

containerd對接harbor_containerd 容器 insecure-registries 配置_檸是檸檬的檬的博客-CSDN博客?

推送鏡像??Containerd 對接私有鏡像倉庫 Harbor - 知乎?

接下來我們來測試下如何在 containerd 中使用 Harbor 鏡像倉庫。

首先我們需要將私有鏡像倉庫配置到 containerd 中去，修改 containerd 的配置文件?/etc/containerd/config.toml：

      [plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".tls]insecure_skip_verify = true[plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".auth]username = "admin"password = "Harbor12345"[plugins."io.containerd.grpc.v1.cri".registry.headers][plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".registry.mirrors."core.harbor.domain"]endpoint = ["https://core.harbor.domain"]

在?plugins."io.containerd.grpc.v1.cri".registry.configs?下面添加對應?core.harbor.domain?的配置信息，insecure_skip_verify = true?表示跳過安全校驗，然后通過?plugins."io.containerd.grpc.v1.cri".registry.configs."core.harbor.domain".auth?配置 Harbor 鏡像倉庫的用戶名和密碼。

配置完成后重啟 containerd：

$ systemctl restart containerd

現在我們使用?nerdctl?來進行登錄：

$ nerdctl login -u admin core.harbor.domain
Enter Password:
ERRO[0004] failed to call tryLoginWithRegHost            error="failed to call rh.Client.Do: Get \"https://harbor.k8s.local/v2/\": x509: certificate signed by unknown authority" i=0
FATA[0004] failed to call rh.Client.Do: Get "https://harbor.k8s.local/v2/": x509: certificate signed by unknown authority
[root@master1 ~]#

可以看到還是會報證書相關的錯誤，只需要添加一個?--insecure-registry?參數即可解決該問題：

$ nerdctl login -u admin --insecure-registry core.harbor.domain
Enter Password:
WARN[0004] skipping verifying HTTPS certs for "harbor.k8s.local"
WARNING: Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded

制作一個es鏡像：

在?Releases · medcl/elasticsearch-analysis-ik · GitHub?下載 es 的ik插件

unzip xx -d ik

FROM docker.elastic.co/elasticsearch/elasticsearch:8.8.2
# ik 下載地址 https://github.com/medcl/elasticsearch-analysis-ik/releases
COPY ik /usr/share/elasticsearch/plugins/ik
# COPY pinyin /usr/share/elasticsearch/plugins/pinyin

然后將該鏡像重新 tag 成 Harbor 上的鏡像地址：

harbor新增一個項目 es

?

nerdctl tag elasticsearch:latest core.harbor.domain/elasticsearch:latest

再執行 push 命令即可將鏡像推送到 Harbor 上：

nerdctl -n k8s.io  push --insecure-registry core.harbor.domain/es/elasticsearch:latest

$ nerdctl push --insecure-registry harbor.k8s.local/library/busybox:1.35.0
INFO[0000] pushing as a reduced-platform image (application/vnd.docker.distribution.manifest.list.v2+json, sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726)
WARN[0000] skipping verifying HTTPS certs for "harbor.k8s.local"
index-sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:8cde9b8065696b65d7b7ffaefbab0262d47a5a9852bfd849799559d296d2e0cd: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 6.9 s                                                                    total:  2.2 Ki (333.0 B/s)

推送完成后，我們就可以在 Portal 頁面上看到這個鏡像的信息了：

鏡像 push 成功，同樣可以測試下 pull：

$ nerdctl rmi harbor.k8s.local/library/busybox:1.35.0
Untagged: harbor.k8s.local/library/busybox:1.35.0@sha256:8c40df61d40166f5791f44b3d90b77b4c7f59ed39a992fd9046886d3126ffa68
Deleted: sha256:cf4ac4fc01444f1324571ceb0d4f175604a8341119d9bb42bc4b2cb431a7f3a5
$ nerdctl rmi busybox:1.35.0
Untagged: docker.io/library/busybox:1.35.0@sha256:8c40df61d40166f5791f44b3d90b77b4c7f59ed39a992fd9046886d3126ffa68
Deleted: sha256:cf4ac4fc01444f1324571ceb0d4f175604a8341119d9bb42bc4b2cb431a7f3a5
$ nerdctl pull --insecure-registry  harbor.k8s.local/library/busybox:1.35.0
WARN[0000] skipping verifying HTTPS certs for "harbor.k8s.local"
harbor.k8s.local/library/busybox:1.35.0:                                          resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:8cde9b8065696b65d7b7ffaefbab0262d47a5a9852bfd849799559d296d2e0cd: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:fc0cda0e09ab32c72c61d272bb409da4e2f73165c7bf584226880c9b85438e63:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.7 s                                                                    total:  2.2 Ki (3.2 KiB/s)$ nerdctl images
REPOSITORY                          TAG       IMAGE ID        CREATED           PLATFORM       SIZE         BLOB SIZE
harbor.k8s.local/library/busybox    1.35.0    29fe0126b13c    17 seconds ago    linux/amd64    1.2 MiB      757.7 KiB

但是上面我們也可以看到單獨使用 containerd 比如通過 nerdctl 或者 ctr 命令訪問 Harbor 鏡像倉庫的時候即使跳過證書校驗或者配置上 CA 證書也是會出現證書錯誤的，這個時候我們需要去跳過證書校驗或者指定證書路徑才行。

# 解決辦法1.指定 -k 參數跳過證書校驗。
$ ctr images pull --user admin:Harbor12345 -k harbor.k8s.local/library/busybox:1.35.0# 解決辦法2.指定CA證書、Harbor 相關證書文件路徑。
$ ctr images pull --user admin:Harbor12345 --tlscacert ca.crt harbor.k8s.local/library/busybox:1.35.0

但是如果直接使用?ctrctl?則是有效的：

$ crictl pull harbor.k8s.local/library/busybox@sha256:29fe0126b13c3ea2641ca42c450fa69583d212dbd9b7b623814977b5b0945726
Image is up to date for sha256:d8c0f97fc6a6ac400e43342e67d06222b27cecdb076cbf8a87f3a2a25effe81c

如果想要在 Kubernetes 中去使用那么就需要將 Harbor 的認證信息以 Secret 的形式添加到集群中去：

$ kubectl create secret docker-registry harbor-auth --docker-server=https://harbor.k8s.local --docker-username=admin --docker-password=Harbor12345 --docker-email=info@ydzs.io -n default

然后我們使用上面的私有鏡像倉庫來創建一個 Pod：

# test-harbor.yaml
apiVersion: v1
kind: Pod
metadata:name: harbor-registry-test
spec:containers:- name: testimage: harbor.k8s.local/library/busybox:1.35.0args:- sleep- "3600"imagePullSecrets:- name: harbor-auth

創建后可以查看該 Pod 是否能正常獲取鏡像：

$ kubectl describe pod harbor-registry-test
Name:         harbor-registry-test
Namespace:    default
Priority:     0
Node:         node1/192.168.0.107
Start Time:   Thu, 07 Jul 2022 18:52:39 +0800
# ......
Events:Type    Reason     Age   From               Message----    ------     ----  ----               -------Normal  Scheduled  10s   default-scheduler  Successfully assigned default/harbor-registry-test to node1Normal  Pulling    10s   kubelet            Pulling image "harbor.k8s.local/library/busybox:1.35.0"Normal  Pulled     5s    kubelet            Successfully pulled image "harbor.k8s.local/library/busybox:1.35.0" in 4.670528883sNormal  Created    5s    kubelet            Created container testNormal  Started    5s    kubelet            Started container test

到這里證明上面我們的私有鏡像倉庫搭建成功了，大家可以嘗試去創建一個私有的項目，然后創建一個新的用戶，使用這個用戶來進行 pull/push 鏡像，Harbor 還具有其他的一些功能，比如鏡像復制，Helm Chart 包托管等等，大家可以自行測試，感受下 Harbor 和官方自帶的 registry 倉庫的差別。

?

?