目錄
一、整體架構
二、部署MySQL主從
三、部署Redis哨兵
四、部署WordPress
五、注意事項
一、整體架構
? ? ? ? 本項目為在一主三從的Kubernetes集群上部署WordPress博客。因為WordPress部分容器版本自行集成Apache和PHP服務,因此在Kubernetes上部署WordPress只需提供MySQL存儲數據和Redis緩存Session會話即可。Kubernetes集群基礎架構如下:
#k8s-master01 ? ? ? ?172.29.7.10
#k8s-node01 ? ? ? ? ? 172.29.7.11
#k8s-node02 ? ? ? ? ? 172.29.7.12
#k8s-node03 ? ? ? ? ? 172.29.7.13
#nfs-server ? ? ? ? ? ? 172.29.7.20
#參考往期博客部署NFS-CSI-Driver使Kubernetes集群能調用NFS本地服務器
root@k8s-master01:~# cat /etc/hosts 127.0.0.1 localhost 172.29.7.10 k8s-master01.wlm.com k8s-master01 kubeapi.wlm.com kubeapi 172.29.7.11 k8s-node01.wlm.com k8s-node01 172.29.7.12 k8s-node02.wlm.com k8s-node02 172.29.7.13 k8s-node03.wlm.com k8s-node03 root@k8s-master01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master01 Ready control-plane 17d v1.29.10 k8s-node01 Ready <none> 17d v1.29.10 k8s-node02 Ready <none> 17d v1.29.10 k8s-node03 Ready <none> 17d v1.29.10
二、部署MySQL主從
? ? ? ? 為了確保數據的高可用和容災性部署MySQL主從為WordPress提供服務。MySQL為有狀態服務應該使用Operator或Statefulset進行部署,此處涉及到主從復制配置復雜,因此簡化使用Deployment進行部署。
#創建項目存放位置
root@k8s-master01:~# mkdir -pv wordpress/mysql/
root@k8s-master01:~# cd wordpress/mysql/#創建PVC實現持久化
root@k8s-master01:~/wordpress/mysql# cat mysql-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: mysql-pvc-masternamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi --- apiVersion: v1 kind: PersistentVolumeClaim metadata:name: mysql-pvc-slavenamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi
#創建Secret存放MySQL密碼(密碼為wlm123使用base64加密后如下)
root@k8s-master01:~/wordpress/mysql# kubectl create secret generic mysql-secret --from-literal='root-password'='wlm123' --from-literal='db-password'='wlm123' --dry-run=client -o yaml > mysql-secret.yaml root@k8s-master01:~/wordpress/mysql# cat mysql-secret.yaml apiVersion: v1 data:db-password: d2xtMTIzroot-password: d2xtMTIz kind: Secret metadata:namespace: wordpressname: mysql-secret
#創建ConfigMap為MySQL提供配置文件,同時實現主從復制
?root@k8s-master01:~/wordpress/mysql# cat mysql-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: mysql-confignamespace: wordpress data:my.cnf: |[mysqld]server-id=1log_bin=/var/lib/mysql/mysql-bin.logbind-address = 0.0.0.0character-set-server = utf8mb4default_authentication_plugin=mysql_native_password[client]default-character-set = utf8mb4 root@k8s-master01:~/wordpress/mysql# cat mysql-config-slave.yaml apiVersion: v1 kind: ConfigMap metadata:name: mysql-config-slavenamespace: wordpress data:my.cnf: |[mysqld]server-id=22log_bin=/var/lib/mysql/mysql-bin.logbind-address = 0.0.0.0character-set-server = utf8mb4default_authentication_plugin=mysql_native_passwordread-only=ONrelay_log=relay-logrelay_log_index=relay-log.index[client]default-character-set = utf8mb4
#可選操作,創建Service Account賦予MySQL服務名稱空間管理員身份,確保其有權限讀取Secret
root@k8s-master01:~/wordpress/mysql# cat mysql-sc.yaml apiVersion: v1 kind: ServiceAccount metadata:creationTimestamp: nullname: mysql-scnamespace: wordpress root@k8s-master01:~/wordpress/mysql# cat rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata:namespace: wordpressname: secret-reader rules: - apiGroups: ["*"]resources: ["*"]verbs: ["*"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata:namespace: wordpressname: read-secrets subjects: - kind: ServiceAccountname: mysql-scnamespace: wordpress roleRef:kind: Rolename: secret-readerapiGroup: rbac.authorization.k8s.io
#部署MySQL應用
root@k8s-master01:~/wordpress/mysql# cat mysql-deploy.yaml --- apiVersion: apps/v1 kind: Deployment metadata:name: mysql-masternamespace: wordpress spec:replicas: 1selector:matchLabels:app: mysqlrole: mastertemplate:metadata:labels:app: mysqlrole: masterspec:serviceAccountName: mysql-sccontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: root-password- name: MYSQL_DATABASEvalue: "wp"- name: MYSQL_USERvalue: "wpuser"- name: MYSQL_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: db-passwordports:- containerPort: 3306volumeMounts:- name: mysql-datamountPath: /var/lib/mysql- name: mysql-configmountPath: /etc/mysql/conf.dvolumes:- name: mysql-datapersistentVolumeClaim:claimName: mysql-pvc-master- name: mysql-configconfigMap:name: mysql-config --- apiVersion: v1 kind: Service metadata:name: mysql-masternamespace: wordpress spec:ports:- port: 3306targetPort: 3306selector:app: mysqlrole: master root@k8s-master01:~/wordpress/mysql# cat mysql-deploy-slave.yaml --- apiVersion: apps/v1 kind: Deployment metadata:name: mysql-slavenamespace: wordpress spec:replicas: 1selector:matchLabels:app: mysqlrole: slavetemplate:metadata:labels:app: mysqlrole: slavespec:serviceAccountName: mysql-sccontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: root-password- name: MYSQL_DATABASEvalue: "wp"- name: MYSQL_USERvalue: "wpuser"- name: MYSQL_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: db-passwordports:- containerPort: 3306volumeMounts:- name: mysql-datamountPath: /var/lib/mysql- name: mysql-config-slavemountPath: /etc/mysql/conf.dvolumes:- name: mysql-datapersistentVolumeClaim:claimName: mysql-pvc-slave- name: mysql-config-slaveconfigMap:name: mysql-config-slave --- apiVersion: v1 kind: Service metadata:name: mysql-slavenamespace: wordpress spec:ports:- port: 3306targetPort: 3306selector:app: mysqlrole: slave
#進入MySQL創建復制賬戶并進行主從配置,確保IO線程和SQL線程正常運行
root@k8s-master01:~/wordpress/mysql# kubectl apply -f . root@k8s-master01:~/wordpress/mysql# kubectl exec -it mysql-master-7ff66bfb86-bvlfg -n wordpress -- mysql -uroot -pwlm123 mysql> create user 'replica'@'%' identified by 'wlm123'; mysql> grant replication slave on *.* to 'replica'@'%'; mysql> flush privileges; mysql> show master status\G; *************************** 1. row ***************************File: mysql-bin.000003Position: 1538Binlog_Do_DB:Binlog_Ignore_DB: Executed_Gtid_Set: 1 row in set (0.00 sec) root@k8s-master01:~/wordpress/mysql# kubectl exec -it?mysql-slave-58cf89f5dd-nh4sz -n wordpress -- mysql -uroot -pwlm123 mysql> CHANGE MASTER TO-> MASTER_HOST='mysql-master',-> MASTER_USER='replica',-> MASTER_PASSWORD='wlm123',-> MASTER_LOG_FILE='mysql-bin.000003',-> MASTER_LOG_POS=1538; mysql> START SLAVE; mysql> show slave status\G; *************************** 1. row ***************************Slave_IO_State: Waiting for master to send eventMaster_Host: mysql-masterMaster_User: replicaMaster_Port: 3306Master_Port: 3306Connect_Retry: 60Master_Log_File: mysql-bin.000003Read_Master_Log_Pos: 1538Relay_Log_File: relay-log.000002Relay_Log_Pos: 320Relay_Master_Log_File: mysql-bin.000003Slave_IO_Running: YesSlave_SQL_Running: Yes
三、部署Redis哨兵
? ? ? ? 為了確保Redis故障自動轉移,使用Satefulset配置Redis一主兩從三哨兵。
#創建項目目錄
root@k8s-master01:~# mkdir ?wordpress/redis/
root@k8s-master01:~# cd wordpress/redis/#創建PVC實現持久化
root@k8s-master01:~/wordpress/redis# cat redis-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: redis-pvcnamespace: wordpress spec:accessModes:- ReadWriteManyresources:requests:storage: 10GistorageClassName: nfs-csi
#創建ConfigMap為Redis提供配置文件
root@k8s-master01:~/wordpress/redis# cat redis-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: redis-confignamespace: wordpress data:redis.conf: |bind 0.0.0.0masterauth 123456requirepass 123456port 6379protected-mode noslave-read-only yes root@k8s-master01:~/wordpress/redis# cat redis-sen-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: redis-sentinel-confignamespace: wordpress data:sentinel.conf: |port 26379sentinel monitor mymaster redis-0.redis-svc.wordpress.svc.cluster.local 6379 2sentinel down-after-milliseconds mymaster 5000sentinel failover-timeout mymaster 10000sentinel auth-pass mymaster 123456sentinel resolve-hostnames yes
#部署HeadLess Service使Redis各Pod可通過名稱相互解析IP地址
root@k8s-master01:~/wordpress/redis# cat redis-svc.yaml --- apiVersion: v1 kind: Service metadata:name: redis-svcnamespace: wordpress spec:clusterIP: Noneports:- port: 6379targetPort: 6379selector:app: redis --- apiVersion: v1 kind: Service metadata:name: redis-sentinelnamespace: wordpress spec:clusterIP: Noneports:- port: 26379targetPort: 26379selector:app: redis-sentinel
#部署redis應用
root@k8s-master01:~/wordpress/redis# cat redis.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: redisnamespace: wordpress spec:serviceName: redis-svcreplicas: 3selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:6.2-alpineports:- containerPort: 6379volumeMounts:- name: redis-datamountPath: /data- name: redis-configmountPath: /usr/local/etc/rediscommand:- sh- -c- |cp /usr/local/etc/redis/redis.conf /data/redis.conf && \redis-server /data/redis.confvolumes:- name: redis-configconfigMap:name: redis-config- name: redis-datapersistentVolumeClaim:claimName: redis-pvc root@k8s-master01:~/wordpress/redis# cat redis-sen.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: redis-sentinelnamespace: wordpress spec:serviceName: redis-sentinelreplicas: 3selector:matchLabels:app: redis-sentineltemplate:metadata:labels:app: redis-sentinelspec:containers:- name: redis-sentinelimage: redis:6.2-alpineports:- containerPort: 26379volumeMounts:- name: redis-sentinel-configmountPath: /usr/local/etc/redisreadOnly: true- name: redis-sen-datamountPath: /datacommand:- sh- -c- |cp /usr/local/etc/redis/sentinel.conf /data/sentinel.conf && \redis-sentinel /data/sentinel.confvolumes:- name: redis-sentinel-configconfigMap:name: redis-sentinel-config- name: redis-sen-datapersistentVolumeClaim:claimName: redis-pvc
#手動配置Redis的主從復制,進入哨兵Pod確保哨兵狀態正常
root@k8s-master01:~/wordpress/redis# kubectl apply -f?redis-pvc.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f?redis-config.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f?redis-sen-config.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f?redis-svc.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f?redis.yaml root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-1 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> replicaof redis-0.redis-svc.wordpress.svc.cluster.local 6379 OK 127.0.0.1:6379> exit /data # exit root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-2 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> replicaof redis-0.redis-svc.wordpress.svc.cluster.local 6379 OK root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-0 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> info replication # Replication role:master connected_slaves:2 slave0:ip=10.244.3.38,port=6379,state=online,offset=70,lag=1 slave1:ip=10.244.2.38,port=6379,state=online,offset=70,lag=1 master_failover_state:no-failover master_replid:09a96f14ebbcf02e358bd675d853dc5797743af5 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:70 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:70 root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-sentinel-0 -n wordpress ?-- sh /data # redis-cli ?-p 26379 127.0.0.1:26379> info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=10.244.1.61:6379,slaves=2,sentinels=3
四、部署WordPress
????????WordPress為無狀態應用使用Deployment即可完成部署,可根據業務情況進行控制副本數量。
#創建項目目錄
root@k8s-master01:~# mkdir ?wordpress/deploy/
root@k8s-master01:~# cd wordpress/deploy/#創建PVC實現持久化并部署應用
root@k8s-master01:~/wordpress/deploy# cat pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: wordpress-pvcnamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi root@k8s-master01:~/wordpress/deploy# cat wordpress.yaml apiVersion: apps/v1 kind: Deployment metadata:name: wordpressnamespace: wordpress spec:replicas: 2selector:matchLabels:app: wordpresstemplate:metadata:labels:app: wordpressspec:containers:- name: wordpressimage: wordpress:6.7-php8.1-apacheenv:- name: WORDPRESS_DB_HOSTvalue: "mysql-master:3306"- name: WORDPRESS_DB_USERvalue: "wpuser"- name: WORDPRESS_DB_PASSWORDvalue: "wlm123"- name: WORDPRESS_DB_NAMEvalue: "wp"- name: WORDPRESS_REDIS_HOSTvalue: "redis-svc:6379"ports:- containerPort: 80volumeMounts:- name: wordpress-storagemountPath: /var/www/htmlvolumes:- name: wordpress-storagepersistentVolumeClaim:claimName: wordpress-pvc --- apiVersion: v1 kind: Service metadata:name: wordpressnamespace: wordpress spec:type: LoadBalancerselector:app: wordpressports:- protocol: TCPport: 80targetPort: 80 root@k8s-master01:~/wordpress/deploy# kubectl get svc -n wordpress NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-master ClusterIP 10.104.10.121 <none> 3306/TCP 16d mysql-slave ClusterIP 10.104.11.245 <none> 3306/TCP 16d redis-sentinel ClusterIP None <none> 26379/TCP 3d1h redis-svc ClusterIP None <none> 6379/TCP 3d1h wordpress LoadBalancer 10.101.6.62 172.29.7.52 80:31292/TCP 15d
#通過LoadBalance暴露的External-IP訪問Web頁面進行配置
#為WordPress配置Ingress服務,方便外部流量管理
root@k8s-master01:~/wordpress/deploy# kubectl create ingress wordpress --rule="wordpress.wlm.com/*"=wordpress:80 --class=nginx ?-n wordpress --dry-run=client -o yaml > ingress-wordpress.yaml root@k8s-master01:~/wordpress/deploy# cat ingress-wordpress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata:creationTimestamp: nullname: wordpressnamespace: wordpress spec:ingressClassName: nginxrules:- host: wordpress.wlm.comhttp:paths:- backend:service:name: wordpressport:number: 80path: /pathType: Prefix status:loadBalancer: {} root@k8s-master01:~/wordpress/deploy# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller LoadBalancer 10.107.247.232 172.29.7.51 80:31631/TCP,443:32678/TCP 16d ingress-nginx-controller-admission ClusterIP 10.106.45.66 <none> 443/TCP 16d
#通過客戶端修改本地hosts文件使域名指向Ingress的IP進行博客訪問
五、注意事項
????????MySQL主從需要確保各容器內配置的server-id唯一,若擴容MySQL從服務需要手動修改配置確保server-id唯一性。
????????MySQL配置文件需要修改默認認證插件至mysql_native_password避免密碼驗證失敗。
????????Redis使用6.2版本時需要再Sentinel配置文件內加參數sentinel resolve-hostnames yes,否則無法解析pod名稱,即使添加無頭服務也不能解析,使用kubectl log pod會有相關提示,在reids-0內使用nslookup命令可驗證無頭服務已正常生效。
????????Redis哨兵模式需要修改配置文件,Configmap使用只讀掛載時,需要用命令把配置文件復制到持久化的data目錄,使用sentinel命令指定data目錄下的配置文件