Centos7+Nginx+Keepalived實現Apache服務的高可用&負載均衡

今天是2017年的第一天，昨天也就是2016年的最后一天，我嘗試部署了Centos7+Nginx+Keepalived實現WEB服務的高可用負載均衡服務，終于在2017年的第一天前完成了，所以在此分享給有需要的朋友；說到負載均衡，其實在linux下有很多服務可以實現，比如nginx、haproxy、lvs等服務，當前我們在前面的文章有介紹過了，但是對于高可用服務，我們在linux下最常見也是應用最多的是Keepalived，對于這些綜合服務對于一個管理員來說是必須要會的，所以今天我們主要介紹的是Centos7+Nginx+Keepalived實現WEB(Apache)服務的高可用負載均衡。具體見下：

wKioL1hn81XCUUt2AAEAl01oBfs174.jpg-wh_50

環境介紹：

Hostname：A-S

IP：192.168.5.21

Role：Apache Web Service

Hostname：B-S

IP：192.168.5.22

Role：Apache Web Service

Hostname：BB-S

IP：192.168.5.53

Role：Nginx+Keepalived

Hostname：BB-S

IP：192.168.5.54

Role：Nginx+Keepalived

Virtual IP：192.168.5.88

我們需要首先安裝兩臺Apache Web服務

因為我們準備使用yum安裝服務，所以需要定義阿里云倉庫

cd?/etc/yum.repo
vim?epel.repo
添加以下內容
[epel]???name=aliyun?epel????baseurl=http://mirrors.aliyun.com/epel/7Server/x86_64/
gpgcheck=0

Image(28)

Image(29)

首先在192.168.5.21上安裝apache

yum?isntall?-y?httpd

Image(30)

然后定義顯示頁面

vim?/var/www/html/index.html
</html>
<!DOCTYPE?html>
<html>
<head>
<title>Welcome?to?Apache</title>
<style>body?{35em;margin:?0?auto;font-family:?Tahoma,?Verdana,?Arial,?sans-serif;}
</style>
<style?type="text/css">
h1{color:red}
h2{color:blue}
h3{color:green}
h4{color:yellow}
}
</style>
</head><body?bgcolor='#46A3FF'>
<h1>Welcome?to?A-S?Apache</h1>
<h2>HostName:A-S</h2>
<h3>IP:192.168.5.21</h3>
<h4>Service:Apache</h4>
<input?type=button??value="Refresh"?οnclick="window.location.href('http://192.168.5.21')">
</body>
</html>

Image(31)

Systemctl?start?httpd

clip_p_w_picpath002

然后添加默認的防火墻端口8o

Firewall-cmd?--zone=public?--add-port='80/tcp'?--permanent

clip_p_w_picpath004

或者vim?/etc/firewalld/zone/public.xml
添加一下格式
<port?portocal='tcp'?port='80'>

我們測試訪問

Image(33)

我們按照同上的方法在第二臺服務器上進行配置，安裝跳過

第二臺主機的配置：192.168.5.22? 主機名 B-S

安裝好httpd后，我們將a-s上的index拷貝到b-s服務器上

scp?index.html?root@192.168.5.22:/var/www/html/

然后修改index.html文件

</html>
<!DOCTYPE?html>
<html>
<head>
<title>Welcome?to?Apache</title>
<style>body?{35em;margin:?0?auto;font-family:?Tahoma,?Verdana,?Arial,?sans-serif;}
</style>
<style?type="text/css">
h1{color:red}
h2{color:blue}
h3{color:green}
h4{color:yellow}
}
</style>
</head><body?bgcolor='#CA8EFF'>
<h1>Welcome?to?B-S?Apache</h1>
<h2>HostName:B-S</h2>
<h3>IP:192.168.5.22</h3>
<h4>Service:Apache</h4>
<input?type=button??value="Refresh"?οnclick="window.location.href('http://192.168.5.22')">
</body>
</html>

Image(34)

測試訪問

Image(35)

接下來我們開始準備在兩臺前端服務器192.168.5.53、192.168.5.54上安裝nginx和keepalived；我們將keepalived和nginx安裝在同一臺機器上

我們首先在主備服務器上安裝nginx

我們首先要定于安裝源， 才可以使用yum安裝，我們在主備服務器上都配置

cd?/etc/yum.repo
vim?epel.repo
添加以下內容
[epel]???name=aliyun?epel????baseurl=http://mirrors.aliyun.com/epel/7Server/x86_64/
gpgcheck=0

Image(36)

定義好后，我們將定義的安裝源給另外一臺備服務器也拷貝一份

scp?/etc/yum.repo/epel.repo?root@192.168.5.54/etc/yum.repo/

Image(37)

指定源后，我們就可以開始安裝nginx了，首先在主服務器上

yum?install?-y?nginx

Image(38)

安裝后，我們首先查看默認的nginx配置

vim?/etc/nginx/nginx.conf
#?For?more?information?on?configuration,?see:
#???*?Official?English?Documentation:?http://nginx.org/en/docs/
#???*?Official?Russian?Documentation:?http://nginx.org/ru/docs/
user?nginx;
worker_processes?auto;
error_log?/var/log/nginx/error.log;
pid?/run/nginx.pid;
#?Load?dynamic?modules.?See?/usr/share/nginx/README.dynamic.
include?/usr/share/nginx/modules/*.conf;
events?{worker_connections?1024;
}
http?{log_format??main??'$remote_addr?-?$remote_user?[$time_local]?"$request"?''$status?$body_bytes_sent?"$http_referer"?''"$http_user_agent"?"$http_x_forwarded_for"';access_log??/var/log/nginx/access.log??main;sendfile????????????on;tcp_nopush??????????on;tcp_nodelay?????????on;keepalive_timeout???65;types_hash_max_size?2048;include?????????????/etc/nginx/mime.types;default_type????????application/octet-stream;#?Load?modular?configuration?files?from?the?/etc/nginx/conf.d?directory.#?See?http://nginx.org/en/docs/ngx_core_module.html#include#?for?more?information.include?/etc/nginx/conf.d/*.conf;server?{listen???????80?default_server;listen???????[::]:80?default_server;server_name??_;root?????????/usr/share/nginx/html;#?Load?configuration?files?for?the?default?server?block.include?/etc/nginx/default.d/*.conf;location?/?{}error_page?404?/404.html;location?=?/40x.html?{}error_page?500?502?503?504?/50x.html;location?=?/50x.html?{}}
#?Settings?for?a?TLS?enabled?server.
#
#????server?{
#????????listen???????443?ssl?http2?default_server;
#????????listen???????[::]:443?ssl?http2?default_server;
#????????server_name??_;
#????????root?????????/usr/share/nginx/html;
#
#????????ssl_certificate?"/etc/pki/nginx/server.crt";
#????????ssl_certificate_key?"/etc/pki/nginx/private/server.key";
#????????ssl_session_cache?shared:SSL:1m;
#????????ssl_session_timeout??10m;
#????????ssl_ciphers?HIGH:!aNULL:!MD5;
#????????ssl_prefer_server_ciphers?on;
#
#????????#?Load?configuration?files?for?the?default?server?block.
#????????include?/etc/nginx/default.d/*.conf;
#
#????????location?/?{
#????????}
#
#????????error_page?404?/404.html;
#????????????location?=?/40x.html?{
#????????}
#
#????????error_page?500?502?503?504?/50x.html;
#????????????location?=?/50x.html?{
#????????}
#????}
}

Image(39)

默認配置我們備份一份

cp?/etc/nginx/nginx.conf?/etc/nginx/nginx.conf.bak

Image(40)

接下來我們要配置nginx實現后端服務器的web的負載均衡;

在此我們使用的是nginx負載均衡的默認方式-輪詢

我們需要在http區域里面添加負載配置

upstream?real_server_pool?{#ip_hash;server??192.168.5.21:80?weight=1?max_fails=2?fail_timeout=30s;server??192.168.5.22:80?weight=1?max_fails=2?fail_timeout=30s;}#?添加一組真實的服務器地址池#?供proxy_pass和fastcgi_pass指令中使用的代理服務器#?后臺如果有動態應用的時候,ip_hash指令可以通過hash算法#?將客戶端請求定位到同一臺后端服務器上,解決session共享,#?但建議用動態應用做session共享#?server用于指定一個后端服務器的名稱和參數#?weight代表權,重默認為1,權重越高被分配的客戶端越多#?max_fails?指定時間內對后端請求失敗的次數#?fail_timeout?達到max_fails指定的失敗次數后暫停的時間#?down參數用來標記為離線,不參與負載均衡.在ip_hash下使用#?backup僅僅在非backup服務器宕機或繁忙的時候使用server?{listen???????192.168.5.53:80;?????????????#?監聽ip改為本地ipserver_name??localhost;#charset?koi8-r;#access_log??logs/host.access.log??main;location?/?{#root???html;#index??index.html?index.htm;proxy_next_upstream?http_502?http_504?error?timeout?invalid_header;proxy_pass?http://real_server_pool;proxy_set_header?Host?$host;proxy_set_header?X-Forwarded-For?$remote_addr;}

Image(41)

添加后的

#?For?more?information?on?configuration,?see:
#???*?Official?English?Documentation:?http://nginx.org/en/docs/
#???*?Official?Russian?Documentation:?http://nginx.org/ru/docs/
user?nginx;
worker_processes?auto;
error_log?/var/log/nginx/error.log;
pid?/run/nginx.pid;
#?Load?dynamic?modules.?See?/usr/share/nginx/README.dynamic.
include?/usr/share/nginx/modules/*.conf;
events?{worker_connections?1024;
}
http?{log_format??main??'$remote_addr?-?$remote_user?[$time_local]?"$request"?''$status?$body_bytes_sent?"$http_referer"?''"$http_user_agent"?"$http_x_forwarded_for"';access_log??/var/log/nginx/access.log??main;sendfile????????????on;tcp_nopush??????????on;tcp_nodelay?????????on;keepalive_timeout???65;types_hash_max_size?2048;include?????????????/etc/nginx/mime.types;default_type????????application/octet-stream;upstream?real_server_pool?{#ip_hash;server??192.168.5.21:80?weight=1?max_fails=2?fail_timeout=30s;server??192.168.5.22:80?weight=1?max_fails=2?fail_timeout=30s;}#?添加一組真實的服務器地址池#?供proxy_pass和fastcgi_pass指令中使用的代理服務器#?后臺如果有動態應用的時候,ip_hash指令可以通過hash算法#?將客戶端請求定位到同一臺后端服務器上,解決session共享,#?但建議用動態應用做session共享#?server用于指定一個后端服務器的名稱和參數#?weight代表權,重默認為1,權重越高被分配的客戶端越多#?max_fails?指定時間內對后端請求失敗的次數#?fail_timeout?達到max_fails指定的失敗次數后暫停的時間#?down參數用來標記為離線,不參與負載均衡.在ip_hash下使用#?backup僅僅在非backup服務器宕機或繁忙的時候使用#?Load?modular?configuration?files?from?the?/etc/nginx/conf.d?directory.#?See?http://nginx.org/en/docs/ngx_core_module.html#include#?for?more?information.include?/etc/nginx/conf.d/*.conf;server?{#??listen???????80?default_server;#??listen???????[::]:80?default_server;listen???????192.168.5.53:80;?????????????#?監聽ip改為本地ipserver_name??localhost;#?server_name??_;root?????????/usr/share/nginx/html;#?Load?configuration?files?for?the?default?server?block.include?/etc/nginx/default.d/*.conf;
#增加的location?/?{#root???html;#index??index.html?index.htm;proxy_next_upstream?http_502?http_504?error?timeout?invalid_header;proxy_pass?http://real_server_pool;proxy_set_header?Host?$host;proxy_set_header?X-Forwarded-For?$remote_addr;}error_page?404?/404.html;location?=?/40x.html?{}error_page?500?502?503?504?/50x.html;location?=?/50x.html?{}}
#?Settings?for?a?TLS?enabled?server.
#
#????server?{
#????????listen???????443?ssl?http2?default_server;
#????????listen???????[::]:443?ssl?http2?default_server;
#????????server_name??_;
#????????root?????????/usr/share/nginx/html;
#
#????????ssl_certificate?"/etc/pki/nginx/server.crt";
#????????ssl_certificate_key?"/etc/pki/nginx/private/server.key";
#????????ssl_session_cache?shared:SSL:1m;
#????????ssl_session_timeout??10m;
#????????ssl_ciphers?HIGH:!aNULL:!MD5;
#????????ssl_prefer_server_ciphers?on;
#
#????????#?Load?configuration?files?for?the?default?server?block.
#????????include?/etc/nginx/default.d/*.conf;
#
#????????location?/?{
#????????}
#
#????????error_page?404?/404.html;
#????????????location?=?/40x.html?{
#????????}
#
#????????error_page?500?502?503?504?/50x.html;
#????????????location?=?/50x.html?{
#????????}
#????}
}

接著我們重啟nginx

systemctl?enable?nginx
systemctl?start?nginx

Image(42)

接下來我們嘗試訪問一下，因為使用的是輪詢，所以訪問nginx會跳轉到不同的兩個頁面

Image(43)

Image(44)

我們在安裝keepalived前，先配置路由轉發

vim?/etc/sysctl.conf
net.ipv4.ip_forward?=?1?#?此參數改為1

Image(26)

sysctl?-p?#?使修改生效

Image(27)

接著我們在主服務器上安裝keepalived

yum?install?-y?keepalived

Image(45)

安裝完成后，我們備份一下keepalived的配置文件

cp?/etc/keepalived/keepalived.conf??/etc/keepalived/keepalived.conf.bak

Image(46)

接下來我們查看一下默認的keepalived默認配置

cat?/etc/keepalived/keepalived.conf
!?Configuration?File?for?keepalived
global_defs?{notification_email?{acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from?Alexandre.Cassen@firewall.locsmtp_server?192.168.200.1smtp_connect_timeout?30router_id?LVS_DEVEL
}
vrrp_instance?VI_1?{state?MASTERinterface?eth0virtual_router_id?51priority?100advert_int?1authentication?{auth_type?PASSauth_pass?1111}virtual_ipaddress?{192.168.200.16192.168.200.17192.168.200.18}
}
virtual_server?192.168.200.100?443?{delay_loop?6lb_algo?rrlb_kind?NATnat_mask?255.255.255.0persistence_timeout?50protocol?TCPreal_server?192.168.201.100?443?{weight?1SSL_GET?{url?{path?/digest?ff20ad2481f97b1754ef3e12ecd3a9cc}url?{path?/mrtg/digest?9b3a0c85a887a256d6939da88aabd8cd}connect_timeout?3nb_get_retry?3delay_before_retry?3}}
}
virtual_server?10.10.10.2?1358?{delay_loop?6lb_algo?rrlb_kind?NATpersistence_timeout?50protocol?TCPsorry_server?192.168.200.200?1358real_server?192.168.200.2?1358?{weight?1HTTP_GET?{url?{path?/testurl/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl2/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl3/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}connect_timeout?3nb_get_retry?3delay_before_retry?3}}real_server?192.168.200.3?1358?{weight?1HTTP_GET?{url?{path?/testurl/test.jspdigest?640205b7b0fc66c1ea91c463fac6334c}url?{path?/testurl2/test.jspdigest?640205b7b0fc66c1ea91c463fac6334c}connect_timeout?3nb_get_retry?3delay_before_retry?3}}
}
virtual_server?10.10.10.3?1358?{delay_loop?3lb_algo?rrlb_kind?NATnat_mask?255.255.255.0persistence_timeout?50protocol?TCPreal_server?192.168.200.4?1358?{weight?1HTTP_GET?{url?{path?/testurl/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl2/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl3/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}connect_timeout?3nb_get_retry?3delay_before_retry?3}}real_server?192.168.200.5?1358?{weight?1HTTP_GET?{url?{path?/testurl/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl2/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}url?{path?/testurl3/test.jspdigest?640205b7b0fc66c1ea91c463fac6334d}connect_timeout?3nb_get_retry?3delay_before_retry?3}}
}

接下來我們修改默認的keepalived配置

echo?>?/etc/keepalived/keepalived.conf

Image(47)

我們清空默認配置后，添加以下內容

vim?/etc/keepalived/keepalived.conf
global_defs?{notification_email?{gavin@ixmsoft.com?????????????#?定義通知郵箱,有多個可以換行添加??????
}notification_email_from?admin@ixmsoft.com#?定義發送郵件的郵箱smtp_server?mail.ixmsoft.com?????????????#?定義發件服務器smtp_connect_timeout?30??????????????????#?定義連接smtp服務器超時時間router_id?LVS_DEVEL
}
vrrp_instance?VI_1?{state?MASTER???????????????????#?標示主備,備機上改為BACKUPinterface???ens160???????????#?HA監測的端口virtual_router_id?51???????????#?主備的virtual_router_id的值必須相同priority?100???????????????????#?優先級,通常主要比備稍大advert_int?1???????????????????#?VRRP?Multicast?廣播周期秒數authentication?{???????????????#?定義認證auth_type?PASS?????????????#?認證方式auth_pass?1111?????????????#?認證口令字}virtual_ipaddress?{????????????#?定義vip192.168.5.88??????????????#?多個可換行添加,一行一個}
}
virtual_server?192.168.5.88?80?{delay_loop?6?????????????#?每隔?6?秒查詢?realserver?狀態lb_algo?rrlb_kind?NATnat_mask?255.255.255.0persistence_timeout?50???#?同一IP?的連接50秒內被分配到同一臺realserverprotocol?TCP?????????????#?用TCP監測realserver的狀態real_server?192.168.5.53?80?{?????????????#需要指向nginx負載的地址weight?3????????????????#?權重TCP_CHECK?{connect_timeout?10??#?10秒無響應超時nb_get_retry?3delay_before_retry?3connect_port?80}}real_server?192.168.5.54?80?{??????????????#需要指向nginx負載的地址weight?3TCP_CHECK?{connect_timeout?3delay_before_retry?3connect_port?80}}
}

我們介紹一下主要參數

global_defs
notification_email?：?keepalived在發生諸如切換操作時需要發送email通知地址，后面的?smtp_server?相比也都知道是郵件服務器地址。也可以通過其它方式報警，畢竟郵件不是實時通知的。
router_id?：?機器標識，通常可設為hostname。故障發生時，郵件通知會用到
vrrp_instance
state?：?指定instance(Initial)的初始狀態，就是說在配置好后，這臺服務器的初始狀態就是這里指定的，但這里指定的不算，還是得要通過競選通過優先級來確定。如果這里設置為MASTER，但如若他的優先級不及另外一臺，那么這臺在發送通告時，會發送自己的優先級，另外一臺發現優先級不如自己的高，那么他會就回搶占為MASTER
interface?：?實例綁定的網卡，因為在配置虛擬IP的時候必須是在已有的網卡上添加的
mcast_src_ip?：?發送多播數據包時的源IP地址，這里注意了，這里實際上就是在那個地址上發送VRRP通告，這個非常重要，一定要選擇穩定的網卡端口來發送，這里相當于heartbeat的心跳端口，如果沒有設置那么就用默認的綁定的網卡的IP，也就是interface指定的IP地址
virtual_router_id?：?這里設置VRID，這里非常重要，相同的VRID為一個組，他將決定多播的MAC地址
priority?：?設置本節點的優先級，優先級高的為master
advert_int?：?檢查間隔，默認為1秒。這就是VRRP的定時器，MASTER每隔這樣一個時間間隔，就會發送一個advertisement報文以通知組內其他路由器自己工作正常
authentication?：?定義認證方式和密碼，主從必須一樣
virtual_ipaddress?：?這里設置的就是VIP，也就是虛擬IP地址，他隨著state的變化而增加刪除，當state為master的時候就添加，當state為backup的時候刪除，這里主要是有優先級來決定的，和state設置的值沒有多大關系，這里可以設置多個IP地址
track_script?：?引用VRRP腳本，即在?vrrp_script?部分指定的名字。定期運行它們來改變優先級，并最終引發主備切換。
vrrp_script
告訴?keepalived?在什么情況下切換，所以尤為重要。可以有多個?vrrp_script
script?：?自己寫的檢測腳本。也可以是一行命令如killall?-0?nginx
interval?2?：?每2s檢測一次
weight?-5?：?檢測失敗（腳本返回非0）則優先級?-5
fall?2?：?檢測連續?2?次失敗才算確定是真失敗。會用weight減少優先級（1-255之間）
rise?1?：?檢測?1?次成功就算成功。但不修改優先級

Image(48)

保存退出后，我們啟動keepalived

systemctl?start?keepalived
systemctl?enable?keepalived

Image(49)

然后我們測試ping一下vistual ip

注：如果ping不通vistual ip的話，有可能是keepalived.conf文件的中的網卡配置，centos6的網卡是eth0，而centos7的是 ens160

Image(50)

ping 192.168.5.88

Image(51)

接下來我們配置第二臺備服務器，然后首先安裝及配置nginx

yum?install?-y?nginx

Image(52)

然后備份備服務器的nginx.conf文件

cp?/etc/nginx/nginx.conf?/etc/nginx/nginx.conf.bak

Image(53)

然后我們從主服務器下的nginx.conf配置拷貝到備服務器nginx目錄下，然后替換默認的nginx.conf文件

scp?/etc/nginx/nginx.conf?root@192.168.5.54:/etc/nginx/

Image(54)

然后我們修改備服務器的監聽地址，需要指向備服務器的本地地址--->?????? listen?????? 192.168.5.54:80;???????????? # 監聽ip改為本地ip

#?For?more?information?on?configuration,?see:
#???*?Official?English?Documentation:?http://nginx.org/en/docs/
#?For?more?information?on?configuration,?see:
#???*?Official?English?Documentation:?http://nginx.org/en/docs/
#???*?Official?Russian?Documentation:?http://nginx.org/ru/docs/
user?nginx;
worker_processes?auto;
error_log?/var/log/nginx/error.log;
pid?/run/nginx.pid;
#?Load?dynamic?modules.?See?/usr/share/nginx/README.dynamic.
include?/usr/share/nginx/modules/*.conf;
events?{worker_connections?1024;
}
http?{log_format??main??'$remote_addr?-?$remote_user?[$time_local]?"$request"?''$status?$body_bytes_sent?"$http_referer"?''"$http_user_agent"?"$http_x_forwarded_for"';access_log??/var/log/nginx/access.log??main;sendfile????????????on;tcp_nopush??????????on;tcp_nodelay?????????on;keepalive_timeout???65;types_hash_max_size?2048;include?????????????/etc/nginx/mime.types;default_type????????application/octet-stream;upstream?real_server_pool?{#ip_hash;server??192.168.5.21:80?weight=1?max_fails=2?fail_timeout=30s;server??192.168.5.22:80?weight=1?max_fails=2?fail_timeout=30s;}#?添加一組真實的服務器地址池#?供proxy_pass和fastcgi_pass指令中使用的代理服務器#?后臺如果有動態應用的時候,ip_hash指令可以通過hash算法#?將客戶端請求定位到同一臺后端服務器上,解決session共享,#?但建議用動態應用做session共享#?server用于指定一個后端服務器的名稱和參數#?weight代表權,重默認為1,權重越高被分配的客戶端越多#?max_fails?指定時間內對后端請求失敗的次數#?fail_timeout?達到max_fails指定的失敗次數后暫停的時間#?down參數用來標記為離線,不參與負載均衡.在ip_hash下使用#?backup僅僅在非backup服務器宕機或繁忙的時候使用#?Load?modular?configuration?files?from?the?/etc/nginx/conf.d?directory.#?See?http://nginx.org/en/docs/ngx_core_module.html#include#?for?more?information.include?/etc/nginx/conf.d/*.conf;server?{#??listen???????80?default_server;#??listen???????[::]:80?default_server;listen???????192.168.5.54:80;?????????????#?監聽ip改為本地ipserver_name??localhost;#?server_name??_;root?????????/usr/share/nginx/html;#?Load?configuration?files?for?the?default?server?block.include?/etc/nginx/default.d/*.conf;location?/?{#root???html;#index??index.html?index.htm;proxy_next_upstream?http_502?http_504?error?timeout?invalid_header;proxy_pass?http://real_server_pool;proxy_set_header?Host?$host;proxy_set_header?X-Forwarded-For?$remote_addr;}error_page?404?/404.html;location?=?/40x.html?{}error_page?500?502?503?504?/50x.html;location?=?/50x.html?{}}
#?Settings?for?a?TLS?enabled?server.
#
#????server?{
#????????listen???????443?ssl?http2?default_server;
#????????listen???????[::]:443?ssl?http2?default_server;
#????????server_name??_;
#????????root?????????/usr/share/nginx/html;
#
#????????ssl_certificate?"/etc/pki/nginx/server.crt";
#????????ssl_certificate_key?"/etc/pki/nginx/private/server.key";
#????????ssl_session_cache?shared:SSL:1m;
#????????ssl_session_timeout??10m;
#????????ssl_ciphers?HIGH:!aNULL:!MD5;
#????????ssl_prefer_server_ciphers?on;
#
#????????#?Load?configuration?files?for?the?default?server?block.
#????????include?/etc/nginx/default.d/*.conf;
#
#????????location?/?{
#????????}
#
#????????error_page?404?/404.html;
#????????????location?=?/40x.html?{
#????????}
#
#????????error_page?500?502?503?504?/50x.html;
#????????????location?=?/50x.html?{
#????????}
#????}
}

其實我們也可以只修改添加的內容

??

??upstream?real_server_pool?{#ip_hash;server??192.168.5.21:80?weight=1?max_fails=2?fail_timeout=30s;server??192.168.5.22:80?weight=1?max_fails=2?fail_timeout=30s;}#?添加一組真實的服務器地址池#?供proxy_pass和fastcgi_pass指令中使用的代理服務器#?后臺如果有動態應用的時候,ip_hash指令可以通過hash算法#?將客戶端請求定位到同一臺后端服務器上,解決session共享,#?但建議用動態應用做session共享#?server用于指定一個后端服務器的名稱和參數#?weight代表權,重默認為1,權重越高被分配的客戶端越多#?max_fails?指定時間內對后端請求失敗的次數#?fail_timeout?達到max_fails指定的失敗次數后暫停的時間#?down參數用來標記為離線,不參與負載均衡.在ip_hash下使用#?backup僅僅在非backup服務器宕機或繁忙的時候使用server?{listen???????192.168.5.54:80;?????????????#?監聽ip改為本地ipserver_name??localhost;#charset?koi8-r;#access_log??logs/host.access.log??main;location?/?{#root???html;#index??index.html?index.htm;proxy_next_upstream?http_502?http_504?error?timeout?invalid_header;proxy_pass?http://real_server_pool;proxy_set_header?Host?$host;proxy_set_header?X-Forwarded-For?$remote_addr;}

Image(55)

然后我們啟動備服務器的nginx服務

systemctl?start?nginx
systemctl?enable?nginx

Image(56)

我們在安裝keepalived前，先配置路由轉發

vim?/etc/sysctl.conf
net.ipv4.ip_forward?=?1?#?此參數改為1

Image(26)[1]

sysctl?-p?#?使修改生效

Image(57)

然后安裝keepalived

Image(58)

然后我們備份默認keepalived配置

cp?/etc/keepalived/keepalived.conf?/etc/keepalived/keepalived.conf.bak

Image(59)

然后修改拷貝的數據文件，

scp?/etc/keepalived/keepalived.conf?root@192.168.5.54:/etc/keepalived/

Image(60)

配置備用調度器的keepalived,只需要將state MASTER 改為state BACKUP,降低priority 100 的值:

vim?/etc/keepalived/keepalived.conf
global_defs?{notification_email?{gavin@ixmsoft.com?????????????#?定義通知郵箱,有多個可以換行添加
}notification_email_from?admin@ixmsoft.com#?定義發送郵件的郵箱smtp_server?mail.ixmsoft.com?????????????#?定義發件服務器smtp_connect_timeout?30??????????????????#?定義連接smtp服務器超時時間router_id?LVS_DEVEL
}
vrrp_instance?VI_1?{state?BACKUP???????????????????#?標示主備,備機上改為BACKUPinterface????ens160??????????#?HA監測的端口virtual_router_id?51???????????#?主備的virtual_router_id的值必須相同priority?50???????????????????#?優先級,通常主要比備稍大advert_int?1???????????????????#?VRRP?Multicast?廣播周期秒數authentication?{???????????????#?定義認證auth_type?PASS?????????????#?認證方式auth_pass?1111?????????????#?認證口令字}virtual_ipaddress?{????????????#?定義vip192.168.5.88??????????????#?多個可換行添加,一行一個}
}
virtual_server?192.168.5.88?80?{delay_loop?6?????????????#?每隔?6?秒查詢?realserver?狀態lb_algo?rrlb_kind?NATnat_mask?255.255.255.0persistence_timeout?50???#?同一IP?的連接50秒內被分配到同一臺realserverprotocol?TCP?????????????#?用TCP監測realserver的狀態real_server?192.168.5.53?80?{???????#需要指向nginx負載的地址weight?3????????????????#?權重TCP_CHECK?{connect_timeout?10??#?10秒無響應超時nb_get_retry?3delay_before_retry?3connect_port?80}}real_server?192.168.5.54?80?{???#需要指向nginx負載的地址weight?3TCP_CHECK?{connect_timeout?3delay_before_retry?3connect_port?80}}
}

我們介紹一下主要參數

global_defs
notification_email?：?keepalived在發生諸如切換操作時需要發送email通知地址，后面的?smtp_server?相比也都知道是郵件服務器地址。也可以通過其它方式報警，畢竟郵件不是實時通知的。
router_id?：?機器標識，通常可設為hostname。故障發生時，郵件通知會用到
vrrp_instance
state?：?指定instance(Initial)的初始狀態，就是說在配置好后，這臺服務器的初始狀態就是這里指定的，但這里指定的不算，還是得要通過競選通過優先級來確定。如果這里設置為MASTER，但如若他的優先級不及另外一臺，那么這臺在發送通告時，會發送自己的優先級，另外一臺發現優先級不如自己的高，那么他會就回搶占為MASTER
interface?：?實例綁定的網卡，因為在配置虛擬IP的時候必須是在已有的網卡上添加的
mcast_src_ip?：?發送多播數據包時的源IP地址，這里注意了，這里實際上就是在那個地址上發送VRRP通告，這個非常重要，一定要選擇穩定的網卡端口來發送，這里相當于heartbeat的心跳端口，如果沒有設置那么就用默認的綁定的網卡的IP，也就是interface指定的IP地址
virtual_router_id?：?這里設置VRID，這里非常重要，相同的VRID為一個組，他將決定多播的MAC地址
priority?：?設置本節點的優先級，優先級高的為master
advert_int?：?檢查間隔，默認為1秒。這就是VRRP的定時器，MASTER每隔這樣一個時間間隔，就會發送一個advertisement報文以通知組內其他路由器自己工作正常
authentication?：?定義認證方式和密碼，主從必須一樣
virtual_ipaddress?：?這里設置的就是VIP，也就是虛擬IP地址，他隨著state的變化而增加刪除，當state為master的時候就添加，當state為backup的時候刪除，這里主要是有優先級來決定的，和state設置的值沒有多大關系，這里可以設置多個IP地址
track_script?：?引用VRRP腳本，即在?vrrp_script?部分指定的名字。定期運行它們來改變優先級，并最終引發主備切換。
vrrp_script
告訴?keepalived?在什么情況下切換，所以尤為重要。可以有多個?vrrp_script
script?：?自己寫的檢測腳本。也可以是一行命令如killall?-0?nginx
interval?2?：?每2s檢測一次
weight?-5?：?檢測失敗（腳本返回非0）則優先級?-5
fall?2?：?檢測連續?2?次失敗才算確定是真失敗。會用weight減少優先級（1-255之間）
rise?1?：?檢測?1?次成功就算成功。但不修改優先級

Image(61)

記得修改防火墻配置哦

firewall-cmd?--add-port='tcp/80'?--permanent

然后我們啟動 keepalived服務

systemctl?start?keepalived
systemcltl?enable?keepalived

Image(62)

我們也測試ping一下vistual ip：

ping 192.168.5.88

Image(63)

接下來我們要實現realserver服務器配置

我們需要在兩臺web(http)192.168.5.21,192.168.5.22，需要在這兩臺服務器上配置虛擬VIP，所以在服務器上執行以下腳本

首先在192.168.5.21服務器上新建腳本文件

vim?real_server
然后添加一下腳本內容：
#!/bin/bash
#?chkconfig:?2345?85?35
#?Description:?Start?real?server?with?host?boot
VIP=192.168.5.88
function?start()?{
ifconfig?lo:0?$VIP?netmask?255.255.255.255?broadcast?$VIP
echo?1?>/proc/sys/net/ipv4/conf/lo/arp_ignore
echo?2?>/proc/sys/net/ipv4/conf/lo/arp_announce
echo?1?>/proc/sys/net/ipv4/conf/all/arp_ignore
echo?2?>/proc/sys/net/ipv4/conf/all/arp_announce
echo?“Real?Server?$(uname?-n)?started”
}
function?stop()?{
ifconfig?lo:0?down
ifconfig?lo:0?$VIP?netmask?255.255.255.255?broadcast?$VIP
echo?0?>/proc/sys/net/ipv4/conf/lo/arp_ignore
echo?0?>/proc/sys/net/ipv4/conf/lo/arp_announce
echo?0?>/proc/sys/net/ipv4/conf/all/arp_ignore
echo?0?>/proc/sys/net/ipv4/conf/all/arp_announce
echo?“Real?Server?$(uname?-n)?stopped”
}
case?$1?in
start)
start
;;
stop)
stop
;;
*)
echo?“Usage:?$0?{start|stop}”
exit?1
esac

Image(64)

保存退出后

然后賦予執行權限

chmoe?a+x?realserver

Image(65)

然后將腳本拷貝到第二臺web服務器上：192.168.5.22

scp?/DATA/real_server?root@192.168.5.22/DATA/

Image(66)

我們在第二臺服務器上查看

Image(67)

最后我們說一下如何將realserver的腳本添加到隨系統啟動

我們將使用chkconfig --add 來管理服務器的添加、順序

cp?realserver?/etc/init.d/?將腳本拷貝到指定目錄
chkconfig?--add?realserver?添加realserver腳本到自動啟動
chkconfig?--list?查看自動啟動服務

Image(68)

chkconfig?--list?查看自動啟動服務

Image(69)

通過以上配置后，我們就可以通過服務進行操作了

/etc/init.d/realserver?stop
/etc/init.d/realserver?start

Image(24)

chkconfig?realserver?on?設置為自動啟動

Image(25)

然后我們在第二臺web服務器上做同樣的操作

cp?realserver?/etc/init.d/?將腳本拷貝到指定目錄
chkconfig?--add?realserver?添加realserver腳本到自動啟動
chkconfig?--list?查看自動啟動服務

Image(70)

chkconfig?--list?查看自動啟動服務

Image(71)

通過以上配置后，我們就可以通過服務進行操作了

/etc/init.d/realserver?stop
/etc/init.d/realserver?start

Image(72)

chkconfig?realserver?on?設置為自動啟動

Image(73)

接下來我們就是測試了，我們在兩臺realserver上執行查看ip狀態

a-s
ip?a?sh

Image(74)

b-s
ip?a?sh

Image(75)

最后我們在keelalived服務器上查看ip狀態,從狀態上看，我們的keepalived是監聽在備服務器上的，如果將備服務器keepalibed停止后，會切換到主服務器上的

aa-s??主服務器
ip?a?sh

Image(76)

備服務器：

bb-s?備服務器
ip?a?sh

Image(77)

最后我們就是測試了

我們訪問vistual ip進行測試

Image(78)

Image(79)

我們查看log

tail?-f?/var/log/message

Image(80)

我們停止主服務器的keepalived服務

Image(81)

查看備服務器的log

Image(82)

Image(83)

Image(79)[1]

最后我們說一下keepalived的log;Keepalived默認所有的日志都是寫入到/var/log/message下的，由于message的日志太多了，而Keepalived的日志又很難分離出來，所以本文提供了一個調整Keepalived日志輸出路徑的方法。 ?
具體操作步驟如下： ? ?
一、修改 /etc/sysconfig/keepalived

默認配置

vim?/etc/sysconfig/keepalived

Image(84)

vim?/etc/sysconfig/keepalived
把KEEPALIVED_OPTIONS="-D"?修改為KEEPALIVED_OPTIONS="-D?-d?-S?0"
#其中-S指定syslog的facility

Image(85)

接下來設置syslog
vim??/etc/rsyslog.conf
注：Cnetos6的路勁是/etc/syslog.conf
Centos7的路勁是/etc/rsyslog.conf
添加以下配置
#?keepalived?-S?0
local0.*????/var/log/keepalived.log

Image(86)

保存退出后，重啟服務

systemctl?restart?rsyslog
systemctl?restart?keepalived

Image(87)

接下來我們查看/var/log路勁是否生成了keepalived.log文件

Image(88)

tail?-f?/var/log/keepalived.conf

Image(89)