需求：
1.創建4個pv，一個pv一個minio-pod。使用sts動態分配pvc(根據存儲類找到pv)。----持久化
2.暴露minio的9001端口。（nodeport）----管理界面
鏡像：minio/minio:RELEASE.2023-03-20T20-16-18Z--->換國內源
說明：本環境僅測試，兩個minio的pod放在master節點，在master加了污點容忍，其他兩個miniopod放在工作節點，上生產環境需要調整，官方建議：4個minio需要4塊硬盤，2個完好可讀，3個完好可讀寫，可以掛一塊磁盤到pv目錄。
159m? 2個minio+/data/minio-pv-0,1
160n? ?1個minio+/data/minio-pv-2
161n? ?1個minio+/data/minio-pv-3
159m# mkdir /data/minio-pv-{0..1} && chmod 777 -R /data/minio-pv-{0..1}
160n#mkdir /data/minio-pv-2 && chmod 777 -R /data/minio-pv-2
161n#mkdir /data/minio-pv-3?&& chmod 777 -R /data/minio-pv-3
#159master上操作
mkdir minio-kind && cd minio-kind
vim minio-ns.yaml? #官方建議獨立分配minio命名空間
apiVersion: v1
kind: Namespace
metadata:name: miniovim minio-pv.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: minio-storage #創建類
provisioner: kubernetes.io/no-provisioner
---
apiVersion: v1
kind: PersistentVolume
metadata:name: minio-pv-0labels:type: local
spec:capacity:storage: 1Gi #生產環境需要調整擴大accessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: minio-storagelocal:path: /data/minio-pv-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- 159m #kubectl get nodes查看node名稱
---
apiVersion: v1
kind: PersistentVolume
metadata:name: minio-pv-1labels:type: local
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: minio-storagelocal:path: /data/minio-pv-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- 159m
---
apiVersion: v1
kind: PersistentVolume
metadata:name: minio-pv-2labels:type: local
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: minio-storagelocal:path: /data/minio-pv-2nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- 160n
---
apiVersion: v1
kind: PersistentVolume
metadata:name: minio-pv-3labels:type: local
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: minio-storagelocal:path: /data/minio-pv-3nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- 161n
vim minio-sts.yaml? #svc,secret一起寫了，注意要保證集群的coredns正常工作
apiVersion: apps/v1
kind: StatefulSet
metadata:name: minionamespace: minio
spec:serviceName: "minio"replicas: 4 # 集群節點數量selector:matchLabels:app: miniotemplate:metadata:labels:app: miniospec:tolerations:- key: "node-role.kubernetes.io/control-plane"operator: "Exists"effect: "NoSchedule"containers:- name: minio#image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/quay.io/minio/minio:latestimage: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/minio/minio:RELEASE.2023-03-20T20-16-18Zargs:- server- http://minio-0.minio:9000/data- http://minio-1.minio:9000/data- http://minio-2.minio:9000/data- http://minio-3.minio:9000/data- --console-address- ":9001" # 控制臺地址參數- --address- ":9000" # API 地址參數ports:- containerPort: 9000name: minio- containerPort: 9001name: minio-consoleenv:- name: MINIO_ROOT_PASSWORDvalueFrom:secretKeyRef:name: minio-secretkey: MINIO_ROOT_PASSWORD- name: MINIO_ROOT_USERvalueFrom:secretKeyRef:name: minio-secretkey: MINIO_ROOT_USER#- name: MINIO_BROWSER#value: "on"volumeMounts:- name: minio-datamountPath: /datavolumeClaimTemplates:- metadata:name: minio-dataspec:accessModes: [ "ReadWriteOnce" ]storageClassName: minio-storage # 使用 minio-storage 存儲類resources:requests:storage: 1Gi # 每個節點的存儲大小
---
apiVersion: v1
kind: Service
metadata:name: minionamespace: minio
spec:#clusterIP: Noneports:- port: 9000targetPort: 9000protocol: TCPname: minionodePort: 30001- port: 9001targetPort: 9001protocol: TCPname: minio-consolenodePort: 30002selector:app: miniotype: NodePort
---
apiVersion: v1
kind: Secret
metadata:name: minio-secretnamespace: minio
type: Opaque
data: MINIO_ROOT_USER: cXdlYXNkMTE= #用戶名base64，這里的參數根據版本不同可能需要調整為accessMINIO_ROOT_PASSWORD: cXdlYXNkMTE= #密碼base64
running起來了過一會再用mc(linux系統需要安裝)連接測試，初始化時間較長
#測試連接
kubectl?port-forward?svc/minio?9000:9000?-n?minio?#臨時測試
mc?alias?set?myminio?http://127.0.0.1:9000?用戶名?密碼#防火墻開放30001和30002端口，提供外部訪問(nodeport)
ufw allow 30001/TCP
ufw allow 30002/TCP
#訪問管理頁面
10.10.10.159:30002?
報錯：
ERROR Unable to initialize backend: format.json file: expected format-type: fs, found: xl
delete -f .資源后，需要清理干凈pv殘留的數據，隱藏目錄需要指定刪除，不然會一直引用之前的數據。這里刪除的是我的pv環境目錄，根據場景刪除。
rm -fr /data/minio-pv-0/.minio.sys/
rm -fr /data/minio-pv-1/.minio.sys/
rm -fr /data/minio-pv-2/.minio.sys/
rm -fr /data/minio-pv-3/.minio.sys/
