使用LVS的 NAT 模式實現 3 臺RS的輪詢訪問
- 1.配置 RS(NAT模式)
- 2. 配置 LVS 主機(僅主機、NAT模式)
- 2.1 配置僅主機網卡(192.168.66.150/24 VIP )
- 2.2 配置 NAT 網卡(192.168.88.6/24 DIP)
- 2.3 LVS 規則匹配
- 2.4 配置內核轉發參數
- 3. 配置客戶端IP(僅主機模式:192.168.66.100/24)
- 報錯糾錯
- curl報錯
- ipvs配置規則報錯
使用LVS的 NAT 模式實現 3 臺RS的輪詢訪問
節點規劃:
主機 | 角色 | 系統 | 網絡 | IP |
---|---|---|---|---|
client | client | redhat 9.5 | 僅主機 | 192.168.66.100/24 |
lvs | lvs | redhat 9.5 | 僅主機 NAT | 192.168.66.150/24 VIP 192.168.88.6/24 DIP |
nginx | rs1 | redhat 9.5 | NAT | 192.168.88.7/24 |
nginx | rs2 | redhat 9.5 | NAT | 192.168.88.17/24 |
nginx | rs3 | redhat 9.5 | NAT | 192.168.88.27/24 |
僅主機網段:192.168.66.0/24
NAT網段:192.168.88.0/24
1.配置 RS(NAT模式)
- 掛載安裝nginx
- echo首頁內容
echo $(hostname -I) > /usr/share/nginx/html/index.html
- 關閉防火墻、selinux
- 開啟 nginx 服務
- curl本機 IP 查看首頁內容
- RS1、RS2、RS3同樣操作:
[root@rs1 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs1 ~]# dnf install nginx -y
Updating Subscription Management repositories.
.....
Complete![root@rs1 ~]# echo $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs1 ~]# systemctl stop firewalld
[root@rs1 ~]# setenforce 0
[root@rs1 ~]# systemctl start nginx
[root@rs1 ~]# curl 192.168.88.7
192.168.88.7
2. 配置 LVS 主機(僅主機、NAT模式)
開啟主機之前,先編輯虛擬機,設置兩張網卡:
保險起見:防止不能上外網,開啟虛擬機后,先把 ipvsamd 軟件安裝上
[root@lvs ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@lvs ~]# dnf install ipvsadm -y
- ipvsadm軟件包安裝目錄
[root@lvs ~]# rpm -ql ipvsadm
/etc/sysconfig/ipvsadm-config #配置文件
/usr/lib/.build-id
/usr/lib/.build-id/0b
/usr/lib/.build-id/0b/d10d85dc0121855898c34f27a7730b50772fcc
/usr/lib/systemd/system/ipvsadm.service #服務啟動文件
/usr/sbin/ipvsadm #主程序
/usr/sbin/ipvsadm-restore #規則重載工具
/usr/sbin/ipvsadm-save #規則保存工具
/usr/share/doc/ipvsadm
/usr/share/doc/ipvsadm/MAINTAINERS
/usr/share/doc/ipvsadm/README
/usr/share/man/man8/ipvsadm-restore.8.gz
/usr/share/man/man8/ipvsadm-save.8.gz
/usr/share/man/man8/ipvsadm.8.gz
- 命令使用
#管理集群服務規則:ipvsadm -A|E virtual-service [-s scheduler] [-p [timeout]] [-M netmask] [--pe persistence_engine] [-b sched-flags]ipvsadm -D virtual-service #刪除ipvsadm -C #清空ipvsadm -R #重載,相當于ipvsadm-restoreipvsadm -S [-n] #保存,相當于ipvsadm-save
#管理集群中的RSipvsadm -a|e virtual-service -r server-address [options]ipvsadm -d virtual-service -r server-addressipvsadm -L|l [virtual-service] [options] #查看ipvsadm -Z [virtual-service] #清空計數器ipvsadm --set tcp tcpfin udpipvsadm --start-daemon {master|backup} [daemon-options]ipvsadm --stop-daemon {master|backup}ipvsadm -h
[root@lvs ~]# nmcli c show
NAME UUID TYPE DEVICE
Wired connection 1 90b171af-f007-36a1-aec7-477b813d9ccd ethernet ens224
ens160 d622d6da-1540-371d-8def-acd3db9bd38d ethernet ens160
lo b5d28f8d-076d-4e67-9fba-afa12cc2e571 loopback lo
# 修改連接名稱
[root@lvs ~]# nmcli c modify 'Wired connection 1' connection.id ens224
[root@lvs ~]# nmcli c show
NAME UUID TYPE DEVICE
ens224 90b171af-f007-36a1-aec7-477b813d9ccd ethernet ens224
ens160 d622d6da-1540-371d-8def-acd3db9bd38d ethernet ens160
lo b5d28f8d-076d-4e67-9fba-afa12cc2e571 loopback lo [root@lvs ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
#僅主機網卡
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:0c:29:26:e1:1b brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.66.128/24 brd 192.168.66.255 scope global dynamic noprefixroute ens160valid_lft 1673sec preferred_lft 1673secinet6 fe80::20c:29ff:fe26:e11b/64 scope link noprefixroute valid_lft forever preferred_lft forever
#NAT網卡
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:25 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.88.149/24 brd 192.168.88.255 scope global dynamic noprefixroute ens224valid_lft 1673sec preferred_lft 1673secinet6 fe80::54c2:3ed3:5085:89a0/64 scope link noprefixroute valid_lft forever preferred_lft forever
2.1 配置僅主機網卡(192.168.66.150/24 VIP )
[root@lvs ~]# nmcli c modify ens160 ipv4.addresses 192.168.66.150/24 ipv4.gateway 192.168.66.2 ipv4.method manual connection.autoconnect yes
[root@lvs ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@lvs ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:1b brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.66.150/24 brd 192.168.66.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe26:e11b/64 scope link noprefixroute valid_lft forever preferred_lft forever
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:25 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.88.149/24 brd 192.168.88.255 scope global dynamic noprefixroute ens224valid_lft 1272sec preferred_lft 1272secinet6 fe80::54c2:3ed3:5085:89a0/64 scope link noprefixroute valid_lft forever preferred_lft forever
2.2 配置 NAT 網卡(192.168.88.6/24 DIP)
[root@lvs ~]# nmcli c modify ens224 ipv4.addresses 192.168.88.6/24 ipv4.gateway 192.168.88.2 ipv4.method manual connection.autoconnect yes
[root@lvs ~]# nmcli c up ens224
[root@lvs ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:1b brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.66.150/24 brd 192.168.66.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe26:e11b/64 scope link noprefixroute valid_lft forever preferred_lft forever
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:25 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.88.6/24 brd 192.168.88.255 scope global noprefixroute ens224valid_lft forever preferred_lft foreverinet6 fe80::54c2:3ed3:5085:89a0/64 scope link noprefixroute valid_lft forever preferred_lft forever
2.3 LVS 規則匹配
- 安裝 ipvsadm(在配置網卡之前)
保險起見:防止不能上外網,先把 ipvsamd 軟件安裝上 - 啟動 ipvsadm(lvs 服務器中已可以訪問RS主機,客戶端暫不能通過訪問 VIP 訪問到后端的 RS 服務器)
ipvsadm-save > /etc/sysconfig/ipvsadm
systemctl start ipvsadm
- LVS規則匹配(LVS主機的僅主機VIP-IP,對應RS主機NAT-IP)
ipvsadm -A -t 192.168.66.150:80 -s rr
ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.7:80 -m -w 2
ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.17:80 -m -w 2
ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.27:80 -m -w 2
- 重啟ipvsadm服務
systemctl restart ipvsadm
# 啟動 ipvsadm
[root@lvs ~]# ipvsadm-save > /etc/sysconfig/ipvsadm
[root@lvs ~]# systemctl start ipvsadm
#lvs 服務器中已可以訪問RS主機,客戶端暫不能通過訪問 VIP 訪問到后端的 RS 服務器(原因是沒有做 LVS 規則匹配)
[root@lvs ~]# curl 192.168.88.7
192.168.88.7
[root@lvs ~]# curl 192.168.88.17
192.168.88.17
[root@lvs ~]# curl 192.168.88.27
192.168.88.27# LVS規則匹配
#配置一條規則
[root@lvs ~]# ipvsadm -A -t 192.168.66.150:80 -s rr
-A:指定規則
-t:指定虛擬的主機的地址
-s:算法 rr#為規則增加RS
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.7:80 -m -w 2
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.17:80 -m -w 2
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.27:80 -m -w 2
-a:虛擬的服務器地址
-r:真實的服務器地址
-m:NAT模式-Masq(不指定即為DR模式-route)
-W:權重root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.66.150:80 rr-> 192.168.88.7:80 Masq 2 0 0 -> 192.168.88.17:80 Masq 2 0 0 -> 192.168.88.27:80 Masq 2 0 0# 重啟ipvsadm
[root@lvs ~]# systemctl restart ipvsadm
重啟好后,可通過客戶端來測試訪問 VIP 能否訪問到后端的 RS 服務器,此時不報錯,但是沒有數據返回。原因是需要配置內核轉發參數
2.4 配置內核轉發參數
- 配置內核轉發參數
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
或者
vim /etc/sysctl.conf
net.ipv4.ip_forward=1
- 執行命令來生效。
sysctl -p
- 重啟ipvsadm服務
systemctl restart ipvsadm
# 配置內核轉發參數
[root@lvs ~]# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf# 執行命令來生效
[root@lvs ~]# sysctl -p
net.ipv4.ip_forward = 1# 重啟ipvsadm服務
[root@lvs ~]# systemctl restart ipvsadm
3. 配置客戶端IP(僅主機模式:192.168.66.100/24)
[root@client ~]# nmcli c modify ens160 ipv4.addresses 192.168.66.100/24 ipv4.gateway 192.168.66.2 connection.autoconnect yes
[root@client ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
[root@client ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:d8:fd:b7 brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.66.100/24 brd 192.168.66.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fed8:fdb7/64 scope link noprefixroute valid_lft forever preferred_lft forever
客戶端通過 VIP 成功 經過 LVS 主機訪問到 RS 主機
[root@client ~]# curl 192.168.66.150
192.168.88.7
[root@client ~]# curl 192.168.66.150
192.168.88.17
[root@client ~]# curl 192.168.66.150
192.168.88.27
報錯糾錯
curl報錯
- RS主機配置nginx后訪問本機IP報錯
- nignx服務未啟動
systemctl start nginx - 防火墻/selinux
setenforce 0
systemctl stop firewalld
- nignx服務未啟動
[root@rs1 ~]# curl 192.168.88.7
curl: (7) Failed to connect to 192.168.88.7 port 80: Connection refused
- 客戶端主機curl訪問IP報錯
[root@client ~]# curl 192.168.66.150
curl: (7) Failed to connect to 192.168.66.150 port 80: No route to host
#lvs防火墻未關
[root@lvs ~]# systemctl stop firewalld
新的報錯:
[root@client ~]# curl 192.168.66.150
curl: (7) Failed to connect to 192.168.66.150 port 80: Connection refused
ipvs配置規則報錯
- 在網絡配置中指定了一個無效的虛擬服務器地址和端口
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 1923168.88.7:80 -m -w 2
illegal real server address[:port] specified[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.7:80 -m -w 2
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.17:80 -m -w 2
[root@lvs ~]# ipvsadm -a -t 192.168.66.150:80 -r 192.168.88.27:80 -m -w 2