说明:本实验为双节点nginx为两台apache服务器提供负载均衡,本文不是做lvs,所以realserver不是配置在keepalived.conf而是在nginx的配置文件中upstream。
此架构需考虑的问题:
1)Master没挂,则Master占有vip且nginx运行在Master上
2)Master挂了,则backup抢占vip且在backup上运行nginx服务
3)如果master服务器上的nginx服务挂了,则vip资源转移到backup服务器上
4)检测后端服务器的健康状态
Master和Backup两边都开启nginx服务,无论Master还是Backup,当其中的一个keepalived服务停止后,vip都会漂移到keepalived服务还在的节点上,如果要想使nginx服务挂了,vip也漂移到另一个节点,则必须用脚本或者在配置文件里面用shell命令来控制。
配置步骤如下
1.初始化4台测试server,该关的关了
[[email protected] ~]# vim /etc/hosts 192.168.1.200 ng-vip 192.168.1.101 ng-master 192.168.1.102 ng-slave 192.168.1.161 web1 192.168.1.162 web2 [[email protected] ~]# yum clean all [[email protected] ~]# systemctl stop firewalld.service [[email protected] ~]# systemctl disable firewalld.service [[email protected] ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
2.配置web1,web2的apache服务,两台一样的方法
[[email protected] ~]# yum -y install httpd [[email protected] ~]# systemctl start httpd [[email protected] ~]# systemctl enable httpd ln -s ‘/usr/lib/systemd/system/httpd.service‘ ‘/etc/systemd/system/multiuser.target.wants/httpd.service‘ [[email protected] ~]# cat /var/www/html/index.html hello this lvs-web1 [[email protected] ~]# yum -y install httpd [[email protected] ~]# systemctl start httpd [[email protected] ~]# systemctl enable httpd ln -s ‘/usr/lib/systemd/system/httpd.service‘ ‘/etc/systemd/system/multiuser.target.wants/httpd.service‘ [[email protected] ~]# cat /var/www/html/index.html hello this lvs-web2
3.通过yum安装配置nginx节点,两台一样的方法
[[email protected] ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/x86_64/ gpgcheck=0 enabled=1 [[email protected] ~]# yum clean all [[email protected] ~]# yum -y install nginx [[email protected] ~]# vim /usr/share/nginx/html/index.html <h1>Welcome to ng-master!</h1> [[email protected] ~]# cd /etc/nginx/conf.d/ [[email protected] conf.d]# mv default.conf default.conf.1 [[email protected] ~]# vim /etc/nginx/conf.d/web.conf upstream myapp1 { server web1; server web2; } server { listen 80; location / { proxy_pass http://myapp1; } } [[email protected] ~]# systemctl restart nginx.service [[email protected] ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/x86_64/ gpgcheck=0 enabled=1 [[email protected] ~]# yum clean all [[email protected] ~]# yum -y install nginx [[email protected] ~]# vim /usr/share/nginx/html/index.html <h1>Welcome to ng-master!</h1> [[email protected] ~]# cd /etc/nginx/conf.d/ [[email protected] conf.d]# mv default.conf default.conf.1 [[email protected] ~]# vim /etc/nginx/conf.d/web.conf upstream myapp1 { server web1; server web2; } server { listen 80; location / { proxy_pass http://myapp1; } } [[email protected] ~]# systemctl restart nginx.service
4.在主nginx服务器上安装keepalived,并配置nginx服务健康检测脚本
[[email protected] conf.d]# yum -y install keepalived [[email protected] conf.d]# cd /etc/keepalived/ [[email protected] keepalived]# cp keepalived.conf keepalived.conf.1 [[email protected] keepalived]# vim keepalived.conf global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server smtp.mail.com smtp_connect_timeout 30 router_id HA_MASTER1 #表示运行keepalived服务器的一个标识,发邮件时显示在邮件主题中的信息 } vrrp_script chk_http_port { script "/usr/local/keepalived/nginx.sh" ####检测nginx状态的脚本链接 interval 2 weight 2 } vrrp_instance VI_2 { #vrrp实例 state MASTER #MASTER/BACKUP interface eno16777736 ####HA 监测网络接口 virtual_router_id 51 #虚拟路由标识,是一个数字,同一个VRRP实例使用唯一的标识,master和backup要一样 priority 100 #用于主从模式,优先级主高于100,从低于100 advert_int 1 #主备之间的通告间隔秒数 authentication { #认证用于主从模式,mater和backup配置一样 auth_type PASS ###主备切换时的验证 auth_pass 1111 #密码 } track_script { chk_http_port ### 执行监控的服务 } virtual_ipaddress { 192.168.1.200/24 dev eno16777736 label eno16777736:1 ###########虚拟ip } } [[email protected] keepalived]# mkdir -p /usr/local/keepalived [[email protected] keepalived]# vim /usr/local/keepalived/nginx.sh #!/bin/bash if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then killall keepalived fi [[email protected] keepalived]# chmod 755 /usr/local/keepalived/nginx.sh [[email protected] keepalived]# systemctl start keepalived [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.101 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fefe:6f3 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.200 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback)
5.在备nginx服务器上安装keepalived,并配置nginx服务健康检测脚本,与主略有不同
[[email protected] conf.d]# yum -y install keepalived [[email protected] conf.d]# cd /etc/keepalived/ [[email protected] keepalived]# cp keepalived.conf keepalived.conf.1 [[email protected] keepalived]# vim keepalived.conf global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server smtp.mail.com smtp_connect_timeout 30 router_id HA_MASTER1 #表示运行keepalived服务器的一个标识,发邮件时显示在邮件主题中的信息 } vrrp_script chk_http_port { script "/usr/local/keepalived/nginx.sh" ####检测nginx状态的脚本链接 interval 2 weight 2 } vrrp_instance VI_2 { #vrrp实例 state BACKUP #MASTER/BACKUP interface eno16777736 ####HA 监测网络接口 virtual_router_id 51 #虚拟路由标识,是一个数字,同一个VRRP实例使用唯一的标识,master和backup要一样 priority 80 #用于主从模式,优先级主高于100,从低于100 advert_int 1 #主备之间的通告间隔秒数 authentication { #认证用于主从模式,mater和backup配置一样 auth_type PASS ###主备切换时的验证 auth_pass 1111 #密码 } track_script { chk_http_port ### 执行监控的服务 } virtual_ipaddress { 192.168.1.200/24 dev eno16777736 label eno16777736:1 ###########虚拟ip } } [[email protected] keepalived]# mkdir -p /usr/local/keepalived [[email protected] keepalived]# vim /usr/local/keepalived/nginx.sh #!/bin/bash if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then killall keepalived fi [[email protected] keepalived]# chmod 755 /usr/local/keepalived/nginx.sh [[email protected] keepalived]# systemctl start keepalived [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.102 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fe87:fd0e prefixlen 64 scopeid 0x20<link> ether 00:0c:29:87:fd:0e txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback)
6.测试:通过浏览器访问测试http://192.168.1.200/,可发现流量在web1和web2之间跳转.
6.1测试关闭主nginx节点上的keepalived服务器,发绑定的vip在主节点消失
[[email protected] keepalived]# systemctl stop keepalived.service [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.101 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fefe:6f3 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) vip在却在备节点上出现 [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.102 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fe87:fd0e prefixlen 64 scopeid 0x20<link> ether 00:0c:29:87:fd:0e txqueuelen 1000 (Ethernet) eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.200 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:0c:29:87:fd:0e txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) 通过浏览器访问测试http://192.168.1.200/,可发现流量依然在web1和web2之间跳转。
6.2再次启动主节点的keepalived服务,发现vip又重新漂移会主节点
[[email protected] keepalived]# systemctl start keepalived.service [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.101 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fefe:6f3 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) eno16777736:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.200 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) 通过浏览器访问测试http://192.168.1.200/,可发现流量依然在web1和web2之间跳转。
6.3关闭nginx主节点上的nginx服务,发现vip从主节点消失,keepalived服务关闭,vip在备节点上出现。
[[email protected] keepalived]# systemctl stop nginx.service [[email protected] keepalived]# ifconfig -a eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.101 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fefe:6f3 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:fe:06:f3 txqueuelen 1000 (Ethernet) lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 0 (Local Loopback) [[email protected] keepalived]# systemctl status keepalived keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled) Active: inactive (dead) 通过浏览器访问测试http://192.168.1.200/,可发现流量依然在web1和web2之间跳转。
6.4再次启动主节点的nginx和keepalived服务后,VIP又漂回主节点。
[[email protected] keepalived]# systemctl start nginx.service [[email protected] keepalived]# systemctl start keepalived 通过浏览器访问测试http://192.168.1.200/,可发现流量依然在web1和web2之间跳转。
参考:
http://www.linuxdiyf.com/linux/12955.html
http://nginx.org/en/linux_packages.html
http://blog.csdn.net/e421083458/article/details/30086413
http://my.oschina.net/u/1458120/blog/208740