一、環境準備與基礎網絡配置
1.1 節點角色與網絡規劃
節點角色 | 主機名 | 所屬網段 | IP 地址 | 網關 | 核心功能 |
---|---|---|---|---|---|
Web 服務器 | web1 | 10.1.8.0/24 | 10.1.8.11 | 10.1.8.10(后期調整為 10.1.8.20) | 部署 Nginx/HTTPD,提供 Web 服務 |
Web 服務器 | web2 | 10.1.8.0/24 | 10.1.8.12 | 10.1.8.10(后期調整為 10.1.8.20) | 部署 Nginx/HTTPD,提供 Web 服務 |
Web 服務器 | web3 | 10.1.8.0/24 | 10.1.8.13 | 10.1.8.10(后期調整為 10.1.8.20) | 部署 Nginx/HTTPD,提供 Web 服務 |
負載均衡器(初期) | lb | 10.1.1.0/24 | 10.1.1.10 | 無需配置 | 部署 LVS,實現負載均衡 |
客戶端 | client1 | 10.1.8.0/24 | 10.1.8.21 | 10.1.8.10(后期調整為 10.1.8.20) | 測試 Web 服務訪問 |
客戶端 | client2 | 10.1.1.0/24 | 10.1.1.21 | 10.1.1.10(后期調整為 10.1.1.20) | 測試負載均衡與高可用 |
路由器 | router | 多網段(10.1.8.0/24、10.1.1.0/24、10.1.2.0/24) | 10.1.8.20、10.1.1.20、10.1.2.20 | - | 實現多網段互通 |
NFS 服務器 | nfs | 10.1.2.0/24 | 10.1.2.100 | 10.1.2.20 | 提供共享存儲,存儲 Web 靜態資源 |
高可用負載均衡器 | ha1 | 10.1.8.0/24 | 10.1.8.14 | 10.1.8.20 | 部署 HAProxy+Keepalived,主負載均衡節點 |
高可用負載均衡器 | ha2 | 10.1.8.0/24 |
1.2 基礎網絡配置(全節點)
1.2.1 10.1.8.0/24 網段節點(web1、web2、web3、client1)
web 服務器
[root@web1-3 ~]#yum install -y nginxsystemctl enable nginx --nowecho Welcome to $(hostname) > /usr/share/nginx/html/index.html
lvs 服務器
[root@lvs ~]# yum install -y ipvsadm# systemctl enable ipvsadm# 等ipvs規則配置完成后再啟動ipvsadm服務
client服務器
# 10.1.8.0/24 網段網關為10.1.8.10 ? #web123 client1nmcli connection modify ens33 ipv4.gateway 10.1.8.10nmcli connection up ens33 ??# 10.1.1.0/24 網段網關為10.1.1.10 #client2 lb不用配置nmcli connection modify ens33 ipv4.gateway 10.1.1.10nmcli connection up ens33 ?
二、Web 服務部署(web1、web2、web3 節點)
配置 web
[root@web1-3 ~]#?# 部署 webyum install -y nginxecho Welcome to $(hostname) > /usr/share/nginx/html/index.html systemctl enable nginx.service --now??#驗證 Web 服務可用性[root@client1 ~ 10:56:20]# curl 10.1.8.11Welcome to web1.laoma.cloud[root@client1 ~ 10:57:49]# curl 10.1.8.12Welcome to web2.laoma.cloud[root@client1 ~ 10:57:53]# curl 10.1.8.13Welcome to web3.laoma.cloud??
三、LVS 負載均衡部署(lb 節點)
配置 LVS
[root@lb ~ 10:55:16]# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf[root@lb ~ 10:59:15]# sysctl -pnet.ipv4.ip_forward = 1??[root@lb ~ 10:59:21]# yum install -y ipvsadm[root@lb ~ 10:59:38]# touch /etc/sysconfig/ipvsadm[root@lb ~ 10:59:38]# systemctl enable ipvsadm --now?# 創建輪詢負載[root@lb ~ 10:59:38]# ipvsadm -A -t 10.1.1.10:80 -s rr[root@lb ~ 11:00:03]# ipvsadm -a -t 10.1.1.10:80 -r 10.1.8.11 -m[root@lb ~ 11:00:03]# ipvsadm -a -t 10.1.1.10:80 -r 10.1.8.12 -m[root@lb ~ 11:00:03]# ipvsadm -a -t 10.1.1.10:80 -r 10.1.8.13 -m[root@lb ~ 11:00:03]# ipvsadm-save -n > /etc/sysconfig/ipvsadm??# 核實配置是否生效[root@lb ~ 11:00:04]# ipvsadm -Ln[root@lb ~ 11:00:59]# for i in {1..90};do curl -s 10.1.1.10 ;done|sort|uniq -c30 Welcome to web1.laoma.cloud30 Welcome to web2.laoma.cloud30 Welcome to web3.laoma.cloud?
負載均衡模式更改為加權輪詢
[root@lb ~ 11:01:12]# ipvsadm -E -t 10.1.1.10:80 -s wrr[root@lb ~ 11:01:33]# ipvsadm -e -t 10.1.1.10:80 -r 10.1.8.12 -m -w 2[root@lb ~ 11:01:33]# ipvsadm -e -t 10.1.1.10:80 -r 10.1.8.13 -m -w 3[root@lb ~ 11:01:34]# ipvsadm -Ln?訪問驗證[root@lb ~ 11:01:46]# for i in {1..90};do curl -s 10.1.1.10 ;done|sort|uniq -c15 Welcome to web1.laoma.cloud30 Welcome to web2.laoma.cloud45 Welcome to web3.laoma.cloud??
思考
此時client1是否可以通過10.1.1.10訪問后端服務器?
答:不能訪問
client1 發出去數據包經過10.1.1.10的ipvs模塊處理,而后端web收到數據包后根據來源地址10.1.8.21進行回復,也就是直接返回client1 導致數據包沒有返回LVS處理
如果不能,需要如何配置才能實現訪問?
[root@web1 ~ 10:57:32]# nmcli connection modify ens33 ipv4.routes '10.1.8.21 255.255.255.255 10.1.8.10'[root@web1 ~ 11:02:32]# nmcli connection up ens33?[root@web2 ~ 10:57:32]# nmcli connection modify ens33 ipv4.routes '10.1.8.21 255.255.255.255 10.1.8.10'[root@web2 ~ 11:02:32]# nmcli connection up ens33?[root@web3 ~ 10:57:32]# nmcli connection modify ens33 ipv4.routes '10.1.8.21 255.255.255.255 10.1.8.10'[root@web3 ~ 11:02:32]# nmcli connection up ens33?
#清空之前實驗[root@lb ~ 11:02:09]# > /etc/sysconfig/ipvsadm[root@lb ~ 11:16:16]# systemctl restart ipvsadm.service ?進入虛擬機把lb的第二張網卡拔掉?#web1 2 3 client1 執行# 10.1.8.0/24 網段網關為10.1.8.20nmcli connection modify ens33 ipv4.gateway 10.1.8.20nmcli connection up ens33?#client2 執行# 10.1.1.0/24 網段網關為10.1.1.20nmcli connection modify ens33 ipv4.gateway 10.1.1.20nmcli connection up ens33
四、LVS-DR 模式部署(全節點適配)
4.1 虛擬網卡(Dummy)配置
Web 節點(web1、web2、web3)配置
# 增加虛擬網卡,子網掩碼一定要32位 [root@client2 ~ 11:20:31]# nmcli connection add type dummy ifname dummy con-name dummy ipv4.method manual ipv4.addresses 10.1.8.100/32 連接 "dummy" (cafa29cd-6424-4356-9dc0-edc6b044be44) 已成功添加。 [root@client2 ~ 11:32:10]# nmcli connection up dummy[root@web1 ~ 11:20:31]# nmcli connection add type dummy ifname dummy con-name dummy ipv4.method manual ipv4.addresses 10.1.8.100/32 連接 "dummy" (c1d840b5-f6f9-4aa9-9688-2318a45628e1) 已成功添加。 [root@web1 ~ 11:34:09]# nmcli connection up dummy[root@web2 ~ 11:20:31]# nmcli connection add type dummy ifname dummy con-name dummy ipv4.method manual ipv4.addresses 10.1.8.100/32 連接 "dummy" (c1d840b5-f6f9-4aa9-9688-2318a45628e1) 已成功添加。 [root@web2 ~ 11:34:09]# nmcli connection up dummy[root@web3 ~ 11:20:31]# nmcli connection add type dummy ifname dummy con-name dummy ipv4.method manual ipv4.addresses 10.1.8.100/32 連接 "dummy" (c1d840b5-f6f9-4aa9-9688-2318a45628e1) 已成功添加。 [root@web3 ~ 11:34:09]# nmcli connection up dummy[root@client2 ~ 11:32:11]# nmcli connection delete dummy 成功刪除連接 "dummy" (cafa29cd-6424-4356-9dc0-edc6b044be44)。 [root@client2 ~ 11:35:56]# nmcli c NAME UUID TYPE DEVICE ens33 555eece5-af4c-45ae-bab9-c07e68d0e649 ethernet ens33 # LVS 節點(lb)配置 [root@web1 ~ 11:33:40]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.11/24 fe80::20c:29ff:feb2:fcae/64 dummy0 DOWN dummy UNKNOWN 10.1.8.100/32 fe80::6f18:c0cb:74d0:ea0d/64 [root@web2 ~ 11:34:04]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.12/24 fe80::20c:29ff:fefe:b2c/64 dummy0 DOWN dummy UNKNOWN 10.1.8.100/32 fe80::f4aa:4d23:ac32:7129/64[root@web3 ~ 11:36:41]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.13/24 fe80::20c:29ff:fe1d:4a9c/64 dummy0 DOWN dummy UNKNOWN 10.1.8.100/32 fe80::30ca:ef21:2f8:fe5c/64
4.2 LVS-DR 模式規則配置(lb 節點)
[root@lb ~ 11:36:58]# nmcli connection add type dummy ifname dummy con-name dummy ipv4.method manual ipv4.addresses 10.1.8.100/32 連接 "dummy" (8cdb619b-460e-4b83-afe3-5f855a601d4d) 已成功添加。 [root@lb ~ 11:40:46]# nmcli connection up dummy 連接已成功激活(D-Bus 活動路徑:/org/freedesktop/NetworkManager/ActiveConnection/4) [root@lb ~ 11:40:48]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.10/24 fe80::20c:29ff:fe55:d621/64 dummy0 DOWN dummy UNKNOWN 10.1.8.100/32 fe80::d941:efae:a684:17ea/64 [root@lb ~ 11:40:59]# ipvsadm -A -t 10.1.8.100:80 -s rr [root@lb ~ 11:42:26]# ipvsadm -a -t 10.1.8.100:80 -r 10.1.8.11:80 [root@lb ~ 11:42:26]# ipvsadm -a -t 10.1.8.100:80 -r 10.1.8.12:80 [root@lb ~ 11:42:26]# ipvsadm -a -t 10.1.8.100:80 -r 10.1.8.13:80 [root@lb ~ 11:42:26]# ipvsadm-save -n > /etc/sysconfig/ipvsadm [root@lb ~ 11:42:27]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.10:80 wrr-> 10.1.8.11:80 Masq 1 0 0 -> 10.1.8.12:80 Masq 2 0 0 -> 10.1.8.13:80 Masq 3 0 0 TCP 10.1.8.100:80 rr-> 10.1.8.11:80 Route 1 0 0 -> 10.1.8.12:80 Route 1 0 0 -> 10.1.8.13:80 Route 1 0 0 [root@lb ~ 11:42:59]# ipvsadm -D -t 10.1.1.10:80 [root@lb ~ 11:43:20]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.8.100:80 rr-> 10.1.8.11:80 Route 1 0 0 -> 10.1.8.12:80 Route 1 0 0 -> 10.1.8.13:80 Route 1 0 0 #DR 模式可用性測試(client1、client2 節點) [root@client2 ~ 11:36:58]# curl http://10.1.8.100 Welcome to web3.laoma.cloud [root@client2 ~ 11:44:04]# curl http://10.1.8.100 Welcome to web2.laoma.cloud [root@client2 ~ 11:44:06]# curl http://10.1.8.100 Welcome to web1.laoma.cloud[root@client1 ~ 11:36:58]# curl http://10.1.8.100 Welcome to web3.laoma.cloud [root@client1 ~ 11:45:11]# curl http://10.1.8.100 Welcome to web2.laoma.cloud [root@client1 ~ 11:45:12]# curl http://10.1.8.100 Welcome to web1.laoma.cloud[root@client1 ~ 11:45:12]# for i in {1..90};do curl -s 10.1.8.100 ;done|sort|uniq Welcome to web1.laoma.cloud Welcome to web2.laoma.cloud Welcome to web3.laoma.cloud[root@client2 ~ 11:46:31]# for i in {1..90};do curl -s 10.1.8.100 ;done|sort|uniq -c30 Welcome to web1.laoma.cloud30 Welcome to web2.laoma.cloud30 Welcome to web3.laoma.cloud
五、Keepalived 高可用部署(web1、web2 節點,后期遷移至 ha1、ha2)
5.1 初期 Web 節點高可用配置(web1 為主、web2 為備)
[root@web1 ~ 14:02:06]# nmcli connection delete dummy 成功刪除連接 "dummy" (6a249f96-28ab-41c5-8f22-e9f0f3e395bc)。[root@lb ~ 14:05:28]#nmcli connection delete dummy reboot
配置 web2,作為備節點
[root@web2 ~]# yum install -y keepalived cp /etc/keepalived/keepalived.conf{,.ori} vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {router_id web2 } #vim vrrp_instance nginx {state BACKUPinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} }[root@web2 ~ 14:17:54]# systemctl enable keepalived.service --now[root@web2 ~ 14:18:09]# systemctl restart keepalived.service [root@web2 ~ 14:19:28]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.12/24 10.1.8.100/24 fe80::20c:29ff:fefe:b2c/64 [root@client1 ~ 14:05:28]# while true;do curl -s http://10.1.8.100/;sleep 1;done Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud ......[root@client2 ~ 14:05:27]# while true;do curl -s http://10.1.8.100/;sleep 1;done Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud .....
5.1.2 主節點配置(web1 節點)
[root@web2 ~ 14:19:38]# scp /etc/keepalived/keepalived.conf web1:/etc/keepalived/keepalived.conf Warning: Permanently added 'web1,10.1.8.11' (ECDSA) to the list of known hosts. keepalived.conf 100% 320 998.8KB/s 00:00 # 雙 VIP 高可用配置(web1、web2 節點) # web1 節點配置(主 10.1.8.100,備 10.1.8.200) [root@web1 ~ 14:16:44]# vim /etc/keepalived/keepalived.conf [root@web1 ~ 14:36:08]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalivedglobal_defs {router_id web1 }vrrp_instance web {state MASTERinterface ens33virtual_router_id 51priority 200advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} }[root@web1 ~ 14:36:19]# systemctl restart keepalived.service #client鏈接顯示 ....... Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud ...... [root@web2 ~ 14:34:11]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.12/24 fe80::20c:29ff:fefe:b2c/64 [root@web1 ~ 14:38:42]# systemctl stop keepalived.service #client鏈接顯示 ....... Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud ......
#web2 節點配置(備 10.1.8.100,主 10.1.8.200) [root@web1 ~ 14:49:14]# vim /etc/keepalived/keepalived.conf [root@web1 ~ 14:49:18]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalivedglobal_defs {router_id web1 }vrrp_instance web_1 {state MASTERinterface ens33virtual_router_id 51priority 200advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} } vrrp_instance web_2 {state BACKUPinterface ens33virtual_router_id 52priority 100advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.200/24} }#web2 [root@web2 ~ 14:37:35]# vim /etc/keepalived/keepalived.conf [root@web2 ~ 14:50:03]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalivedglobal_defs {router_id web2 }vrrp_instance web_1 {state BACKUPinterface ens33virtual_router_id 52priority 100advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} }vrrp_instance web_2 {state MASTERinterface ens33virtual_router_id 51priority 200advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.200/24} }[root@web1 ~ 14:51:38]# systemctl restart keepalived.service [root@web1 ~ 14:52:10]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.11/24 10.1.8.100/24 fe80::20c:29ff:feb2:fcae/64 [root@web2 ~ 14:51:12]# systemctl restart keepalived.service [root@web2 ~ 14:52:16]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.12/24 10.1.8.200/24 fe80::20c:29ff:fefe:b2c/64 #驗證 高可用切換測試(client1 節點) [root@client1 ~ 14:40:09]# while true;do curl -s http://10.1.8.100/;sleep 1;done Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud Welcome to web1.laoma.cloud ^C [root@client1 ~ 14:53:26]# while true;do curl -s http://10.1.8.200/;sleep 1;done Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud Welcome to web2.laoma.cloud ^C#斷開web1,查看web2 [root@web1 ~ 14:52:26]# systemctl stop keepalived.service [root@web2 ~ 14:55:19]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.12/24 10.1.8.200/24 10.1.8.100/24 fe80::20c:29ff:fefe:b2c/64
[root@web1 ~ 14:55:07]# nmcli c NAME UUID TYPE DEVICE 有線連接 1 57b39c8d-d270-3ce9-95e4-48b5823381a6 ethernet ens36 ens33 555eece5-af4c-45ae-bab9-c07e68d0e649 ethernet ens33 [root@web1 ~ 15:05:28]# nmcli connection modify 有線連接\ 1 ipv4.method manual ipv4.addresses 10.1.2.11/24 connection.id ens36;nmcli connection up ens36[root@web2 ~ 15:07:10]# nmcli connection modify 有線連接\ 1 ipv4.method manual ipv4.addresses 10.1.2.12/24 connection.id ens36;nmcli connection up ens36[root@web3 ~ 15:07:31]# nmcli connection modify 有線連接\ 1 ipv4.method manual ipv4.addresses 10.1.2.13/24 connection.id ens36;nmcli connection up ens36[root@web1 ~ 15:07:04]# systemctl stop keepalived [root@web2 ~ 15:07:04]# systemctl stop keepalived [root@web3 ~ 15:07:04]# systemctl stop keepalived [root@lb ~ 14:05:28]# init 0[root@web 1 2 3 ~ 15:13:37]# ip -br a
用cluster-tpl 克隆三臺虛擬機nfs(修改網卡為vm (僅主機2)) ha1 ha2(不變)#nfs (2.100)為2.0網絡段 router 主機添加網卡(一共三個)添加網卡(設置vm (2僅主機))
# **nfs**虛擬機界面執行 hostnamectl set-hostname nfs.lyk.cloud nmcli c nmcli connection modify ens33 ipv4.addresses 10.1.2.100/24 ipv4.gateway 10.1.2.20 nmcli connection up ens33 bash ping 1.1.1.1#Xshell執行 [root@router ~ 15:26:51]# nmcli c NAME UUID TYPE DEVICE ens33 555eece5-af4c-45ae-bab9-c07e68d0e649 ethernet ens33 ens36 c4a81250-34ce-3a67-a3ea-bacfb0289b97 ethernet ens36 有線連接 1 e7758ba8-0110-3e86-883c-8aa1bbdf1f2b ethernet -- [root@router ~ 15:27:05]# nmcli connection modify 有線連接\ 1 ipv4.method manual ipv4.addresses 10.1.2.20/24 connection.id ens37 [root@router ~ 15:29:39]# nmcli connection up ens37 連接已成功激活(D-Bus 活動路徑:/org/freedesktop/NetworkManager/ActiveConnection/11)[root@router ~ 15:31:07]# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 ens33 UP 10.1.8.20/24 fe80::20c:29ff:fe62:b97a/64 ens36 UP 10.1.1.20/24 fe80::f529:7e26:4c51:56cc/64 ens37 UP 10.1.2.20/24 fe80::1334:fcc3:8f69:5e5a/64 [root@nfs ~ 15:28:08]# ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=2 ttl=127 time=144 ms 64 bytes from 1.1.1.1: icmp_seq=3 ttl=127 time=186 ms 64 bytes from 1.1.1.1: icmp_seq=4 ttl=127 time=390 ms 64 bytes from 1.1.1.1: icmp_seq=5 ttl=127 time=145 ms ......
hostnamectl set-hostname ha1.lyk.cloud nmcli connection modify ens33 ipv4.addresses 10.1.8.14 ipv4.gateway 10.1.8.20 nmcli connection up ens33 bashhostnamectl set-hostname ha2.lyk.cloud nmcli connection modify ens33 ipv4.addresses 10.1.8.15 ipv4.gateway 10.1.8.20 nmcli connection up ens33 bash
[root@client2 ~ 15:52:26]# vim /etc/hosts [root@client2 ~ 15:52:35]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6########### cluster ################ 10.1.8.100 www.laoma.cloud www 10.1.8.10 lb.laoma.cloud lb 10.1.8.11 web1.laoma.cloud web1 10.1.8.12 web2.laoma.cloud web2 10.1.8.13 web3.laoma.cloud web3 10.1.8.14 ha1.laoma.cloud ha1 10.1.8.15 ha2.laoma.cloud ha2 10.1.8.20 router.laoma.cloud router 10.1.8.21 client1.laoma.cloud client1 10.1.1.21 client2.laoma.cloud client2#client2推送到其他節點 [root@client2 ~ 15:59:47]# for host in 10.1.8.1{1..5} 10.1.8.20 10.1.8.21 ; do scp /etc/hosts $host:/etc/hosts; done
六、NFS 共享存儲部署(nfs 節點與 web 節點)
6.1 NFS 服務器配置(nfs 節點)
6.1.1 安裝 NFS 服務并創建共享目錄
web三個加網卡,vm (僅主機2)
# 安裝軟件 web1 2 3 nfs yum install -y nfs-utils#再次添加一行nfs 再次推送 [root@client2 ~ 16:01:05]# vim /etc/hosts [root@client2 ~ 16:00:37]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6########### cluster ################ 10.1.8.100 www.laoma.cloud www 10.1.8.10 lb.laoma.cloud lb 10.1.8.11 web1.laoma.cloud web1 10.1.8.12 web2.laoma.cloud web2 10.1.8.13 web3.laoma.cloud web3 10.1.8.14 ha1.laoma.cloud ha1 10.1.8.15 ha2.laoma.cloud ha2 10.1.8.20 router.laoma.cloud router 10.1.8.21 client1.laoma.cloud client1 10.1.1.21 client2.laoma.cloud client2 10.1.2.100 nfs.laoma.cloud nfs
安裝 NFS 服務并創建共享目錄
[root@nfs ~ 16:19:02]# mkdir /var/www/html/ -p [root@nfs ~ 16:19:55]# echo Welcome to www.lyk.cloud > /var/www/html/index.html [root@nfs ~ 16:20:01]# echo '/var/www 10.1.2.0/24(rw,sync)' >> /etc/exports [root@nfs ~ 16:27:46]# systemctl restart nfs-server [root@nfs ~ 16:20:06]# systemctl enable nfs-server.service --now [root@nfs ~ 16:21:12]# systemctl status nfs[root@web1 ~ 16:22:14]# systemctl disable nginx.service --now Removed symlink /etc/systemd/system/multi-user.target.wants/nginx.service. [root@web1 ~ 16:26:11]# systemctl start httpd[root@web1 ~ 16:22:14]# systemctl disable nginx.service --now Removed symlink /etc/systemd/system/multi-user.target.wants/nginx.service. [root@web1 ~ 16:26:11]# systemctl start httpd[root@web2 ~ 16:26:32]# yum install -y httpd [root@web2 ~ 16:26:54]# echo Welcome to $(hostname) > /var/www/html/index.html [root@web2 ~ 16:26:54]# systemctl enable httpd.service --now Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.[root@web3 ~ 16:27:03]# systemctl disable nginx --now Removed symlink /etc/systemd/system/multi-user.target.wants/nginx.service. [root@web3 ~ 16:27:09]# systemctl enable httpd.service --now#驗證 NFS 共享(web1 節點) [root@web1 ~ 16:28:00]# showmount -e nfs Export list for nfs: /var/www 10.1.2.0/24# Web 節點掛載 NFS(web1、web2、web3 節點) # 配置永久掛載(/etc/fstab) [root@web1 ~ 16:35:19]# vim /etc/fstab #最后一行添加 【】echo 'nfs.laoma.cloud:/var/www /var/www/ nfs defaults 0 0' >> /etc/fstab #或者 [root@web1 ~ 17:01:10]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Fri Aug 1 15:45:32 2025 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/centos-root / xfs defaults 0 0 UUID=b54b3764-2b2b-4a76-a0ec-83e308071ae5 /boot xfs defaults 0 0 /dev/mapper/centos-swap swap swap defaults 0 0 nfs.laoma.cloud:/var/www /var/www/ nfs defaults 0 0[root@web1 ~ 17:01:10]# mount /var/www/ [root@web1 ~ 17:02:20]# df -h /var/www/ 文件系統 容量 已用 可用 已用% 掛載點 nfs.laoma.cloud:/var/www 50G 1.6G 49G 4% /var/www#web2 3 [root@web2 ~ 16:26:54]# echo 'nfs.laoma.cloud:/var/www /var/www/ nfs defaults 0 0' >> /etc/fstab [root@web2 ~ 16:42:10]# mount /var/www/ [root@web2 ~ 17:10:11]# df -h /var/www/ 文件系統 容量 已用 可用 已用% 掛載點 nfs.laoma.cloud:/var/www 50G 1.6G 49G 4% /var/www[root@web3 ~ 16:26:54]# echo 'nfs.laoma.cloud:/var/www /var/www/ nfs defaults 0 0' >> /etc/fstab [root@web3 ~ 16:42:10]# mount /var/www/ [root@web3 ~ 17:10:11]# df -h /var/www/ 文件系統 容量 已用 可用 已用% 掛載點 nfs.laoma.cloud:/var/www 50G 1.6G 49G 4% /var/www#驗證 [root@client2 ~ 16:58:02]# curl 10.1.8.13 Welcome to www.lyk.cloud [root@client2 ~ 17:12:12]# curl 10.1.8.12 Welcome to www.lyk.cloud [root@client2 ~ 17:12:15]# curl 10.1.8.11 Welcome to www.lyk.cloud
七、HAProxy 負載均衡部署(ha1、ha2 節點)
7.1 HAProxy 基礎配置(ha1、ha2 節點)
[root@web1 ~ 17:09:54]# umount /var/www [root@web2 ~ 17:09:54]# umount /var/www [root@web3 ~ 17:09:54]# umount /var/www #驗證 [root@client2 ~ 17:12:17]# curl 10.1.8.11 Welcome to web1.laoma.cloud [root@client2 ~ 17:14:47]# curl 10.1.8.12 Welcome to web2.laoma.cloud [root@client2 ~ 17:14:50]# curl 10.1.8.13 Welcome to web3.laoma.cloud
7.1.1 安裝 HAProxy 并備份配置
[root@ha1-2 ~]# yum install -y haproxy # 備份 haproxy 配置文件 cp /etc/haproxy/haproxy.cfg{,.ori}# 修改 haproxy 配置文件,最后添加以下內容 echo ' ########### web 代理 ########### frontend http_frontbind *:80use_backend http_back backend http_backbalance roundrobinserver node1 10.1.8.11:80 checkserver node2 10.1.8.12:80 checkserver node3 10.1.8.13:80 check ' >> /etc/haproxy/haproxy.cfg# 啟用并啟動服務 systemctl enable haproxy.service --now#驗證 HAProxy 負載效果(client2 節點) [root@client2 ~ 17:14:51]# curl 10.1.8.14 Welcome to web1.laoma.cloud [root@client2 ~ 17:24:28]# curl 10.1.8.14 Welcome to web2.laoma.cloud [root@client2 ~ 17:24:30]# curl 10.1.8.14 Welcome to web3.laoma.cloud
7.2 HAProxy+Keepalived 高可用配置(ha1 為主、ha2 為備)
7.2.1 主節點配置(ha1 節點)
[root@ha1 ~ 17:25:35]# cp /etc/keepalived/keepalived.conf{,.bak} [root@ha1 ~ 17:25:35]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {router_id ha1 }vrrp_instance nginx {state MASTERinterface ens33virtual_router_id 51priority 110advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} }[root@ha1 ~ 17:26:07]# systemctl enable keepalived.service --now Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
7.2.2 備節點配置(ha2 節點)
yum install -y keepalived ipvsadm cp /etc/keepalived/keepalived.conf{,.bak} vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {router_id ha2 }vrrp_instance nginx {state BACKUPinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass lyk@123}virtual_ipaddress {10.1.8.100/24} }systemctl enable keepalived.service --now
八、集群功能與高可用測試
8.1 功能性測試(client2 節點)
[root@client1 ~ 17:30:02]# while true ;do curl -s www.laoma.cloud;sleep 1;done 1 2 3輪流 #好像沒效果 [root@ha1 ~ 17:26:12]# systemctl stop keepalived.service#停web3 [root@web3 ~ 17:20:45]# systemctl stop httpd.service Welcome to web1.laoma.cloud Welcome to web2.laoma.cloud...... 沒有web3了#web1 掛載 [root@web1 ~ 17:19:57]# mount -a Welcome to web2.laoma.cloud Welcome to www.lyk.cloud Welcome to web2.laoma.cloud .... web1變成www
#恢復web1 3 [root@web1 ~ 17:35:32]# umount /var/www [root@web3 ~ 17:34:32]# systemctl restart httpd.service
8.2 高可用切換測試
8.2.1 負載均衡主節點故障測試
[root@client2 ~ 17:29:31]# while true ;do curl -s http://10.1.8.100;sleep 1;done Welcome to web3.laoma.cloud Welcome to web2.laoma.cloud Welcome to web1.laoma.cloud Welcome to web3.laoma.cloud ......
測試1:停止 ha1 上 keepalived 服務。
[root@ha1 ~ 17:32:39]# systemctl stop keepalived.service[root@client2 ~ 17:29:31]# while true ;do curl -s http://10.1.8.100;sleep 1;done Welcome to web3.laoma.cloud Welcome to web2.laoma.cloud Welcome to web1.laoma.cloud Welcome to web3.laoma.cloud ...... 無影響
結果:客戶端無感知故障,正常訪問集群。
測試2:恢復 ha1 上 keepalived 服務。
[root@ha1 ~ 17:46:04]# systemctl start keepalived.service **結果**:客戶端無感知故障,正常訪問集群。
負載均衡測試
測試1:停止 web2 上 httpd.service ,監控客戶端訪問情況。
[root@web2 ~ 17:48:27]# systemctl stop httpd.service
結果:大概 15 秒,LVS 將 web2 從后端虛擬主機中剔除
測試2:啟動 web2 上 httpd.service,監控客戶端訪問情況。
[root@web2 ~]# systemctl start httpd.service
結果:大概 5 秒,LVS將web2加入后端虛擬主機中。