- 18.1 集群介绍 - 18.2 keepalived介绍 - 18.3/18.4/18.5 用keepalived配置高可用集群 - 扩展 - heartbeat和keepalived比较 http://blog.csdn.net/yunhua_lee/article/details/9788433 - DRBD工作原理和配置 http://502245466.blog.51cto.com/7559397/1298945 - mysql+keepalived http://lizhenliang.blog.51cto.com/7876557/1362313 # 18.1 集群介绍 - Linux集群概述 - 根据功能划分为两大类:高可用和负载均衡 - 高可用集群通常为两台服务器,一台工作,另外一台作为冗余,当提供服务的机器宕机,冗余将接替继续提供服务 //通常 - 对于大企业来说。可用程度达到99.99%或者 是5个9 - 实现高可用的开源软件有:heartbeat、keepalived //centos6 bug 多,而且很久没有更新了,不建议继续使用 ;keepalived不仅有高可用还有负载均衡 - 负载均衡集群,需要有一台服务器作为分发器,它负责把用户的请求分发给后端的服务器处理,在这个集群里,除了分发器外,就是给用户提供服务的服务器了,这些服务器数量至少为2 - 实现负载均衡的开源软件有LVS、keepalived、haproxy、nginx,商业的有F5、Netscaler # 18.2 keepalived介绍 - 在这里我们使用keepalived来实现高可用集群,因为heartbeat在centos6上有一些问题,影响实验效果 - keepalived通过VRRP(Virtual Router Redundancy Protocl 中文为:虚拟路由器冗余协议)来实现高可用。来实现高可用。 - 在这个协议里会将多台功能相同的路由器组成一个小组,这个小组里会有1个master角色和N(N>=1)个backup角色。 - master会通过组播的形式向各个backup发送VRRP协议的数据包,当backup收不到master发来的VRRP数据包时,就会认为master宕机了。此时就需要根据各个backup的优先级来决定谁成为新的mater。 - Keepalived要有三个模块,分别是core、check和vrrp。其中core模块为keepalived的核心,负责主进程的启动、维护以及全局配置文件的加载和解析,check模块负责健康检查,vrrp模块是来实现VRRP协议的。 # 18.3 用keepalived配置高可用集群(上) - 搭建高可用的前提,是先要有一个工具,然后需要有一个服务去让工具实现高可用,这个实验,就是让nginx作为一个服务,让它成为一个高可用的对象;因为nginx在企业里使用量比较大,所以就使用他来做服务对象 - 环境准备 - master :192.168.202.131 (已经做过lnmp) - backup:192.168.202.132(lamp,没有nginx服务) - 两台机器都安装keepalived - 执行yum install -y keepalived - 为了方便做实验 - 检查两台机器的selinux,iptables两个防火墙情况,selinux需要关闭,iptables需要关闭firewalld - 对backup机器安装nginx服务 - yum install -y nginx - 准备两台机器aming-01 是131和aming-02 是 132,131作为master,132作为backup - 两台机器都执行yum install -y keepalived - 先是master 安装ke ``` [[email protected] ~]# yum install -y keepalived 已加载插件:fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.sohu.com * epel: mirrors.ustc.edu.cn 已安装: keepalived.x86_64 0:1.3.5-1.el7 作为依赖被安装: lm_sensors-libs.x86_64 0:3.4.0-4.20160601gitf9185e5.el7 net-snmp-agent-libs.x86_64 1:5.7.2-28.el7 net-snmp-libs.x86_64 1:5.7.2-28.el7 完毕! [[email protected] ~]# | 3.6 kB 00:00:00 ``` - backup 机器安装keppalived ``` [[email protected] ~]# yum install -y keepalived 已加载插件:fastestmirror Loading mirror speeds from cached hostfile 已安装: keepalived.x86_64 0:1.3.5-1.el7 作为依赖被安装: lm_sensors-libs.x86_64 0:3.4.0-4.20160601gitf9185e5.el7 net-snmp-agent-libs.x86_64 1:5.7.2-28.el7 net-snmp-libs.x86_64 1:5.7.2-28.el7 完毕! [[email protected] ~]# ``` - 因为之前aming-01 上面做过nginx,所以上面有nginx服务 ``` 完毕! [[email protected] ~]# ps aux |grep nginx root 868 0.0 0.1 45992 1280 ? Ss 21:55 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 873 0.0 0.4 48480 4180 ? S 21:55 0:00 nginx: worker process nobody 874 0.0 0.3 48480 3924 ? S 21:55 0:00 nginx: worker process root 2435 0.0 0.0 112684 976 pts/0 S+ 22:03 0:00 grep --color=auto nginx [[email protected] ~]# ``` - 而第二台aming-02 机器上看下有没有nginx,没有就用yum install -y nginx 安装一个 ``` [[email protected] ~]# rpm -qa |grep nginx [[email protected] ~]# [[email protected] ~]# yum install -y nginx 已安装: nginx.x86_64 1:1.10.2-2.el7 作为依赖被安装: gd.x86_64 0:2.0.35-26.el7 gperftools-libs.x86_64 0:2.4-8.el7 libXpm.x86_64 0:3.5.12-1.el7 libunwind.x86_64 2:1.2-2.el7 libxslt.x86_64 0:1.1.28-5.el7 nginx-all-modules.noarch 1:1.10.2-2.el7 nginx-filesystem.noarch 1:1.10.2-2.el7 nginx-mod-http-geoip.x86_64 1:1.10.2-2.el7 nginx-mod-http-image-filter.x86_64 1:1.10.2-2.el7 nginx-mod-http-perl.x86_64 1:1.10.2-2.el7 nginx-mod-http-xslt-filter.x86_64 1:1.10.2-2.el7 nginx-mod-mail.x86_64 1:1.10.2-2.el7 nginx-mod-stream.x86_64 1:1.10.2-2.el7 完毕! [[email protected] ~]# ``` - 在这有俩个nginx 一个是源码包安装的nginx ,一个是yum 安装的nginx , - 下面更改配置文件 - master机器配置,服务工具准备好以后,就配置keepalived ,默认的配置文件路径在 - 先把里面内容清空,有个快捷键 >!$ 添加如下配置, ``` [[email protected] ~]# ls /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf [[email protected] ~]# vim /etc/keepalived/keepalived.conf [[email protected] ~]# > !$ > /etc/keepalived/keepalived.conf [[email protected] ~]# ``` - 添加如下配置 ``` [[email protected] ~]# !vim vim /etc/keepalived/keepalived.conf global_defs { notification_email { //邮件 [email protected] } notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/usr/local/sbin/check_ng.sh" //检查服务是否正常,通过脚本实现,检查服务健康状态 interval 3 //检查时间 } vrrp_instance VI_1 { state MASTER //定义master相关 interface ens33 //通过那个网站使用vrrp协议,配置时,需注意你的网卡配置文件是否是哪个。因为系统ens并不是固定的。 virtual_router_id 51 //定义路由器ID ,配置的时候和从机器一致 priority 100 //权重, advert_int 1 authentication { //认证相关信息 auth_type PASS auth_pass aminglinux>com } virtual_ipaddress { //定义一个公有IP(VIP) 192.168.188.100 //更改为192.168.202.100 } track_script { chk_nginx } } -- 插入 -- 30,2 底端 [[email protected] ~]# vim /etc/keepalived/keepalived.conf ``` - 配置文件编辑完了之后还需要定义check脚本 - 定义一个check的脚本 ``` [[email protected] ~]# vim /usr/local/sbin/check_ng.sh #!/bin/bash #时间变量,用于记录日志 d=`date --date today +%Y%m%d_%H:%M:%S` #计算nginx进程数量 n=`ps -C nginx --no-heading|wc -l` #如果进程为0,则启动nginx,并且再次检测nginx进程数量, #如果还为0,说明nginx无法启动,此时需要关闭keepalived if [ $n -eq "0" ]; then /etc/init.d/nginx start n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi ~ ~ ~ ~ -- 插入 -- 15,3 全部 [[email protected] ~]# vim /usr/local/sbin/check_ng.sh [[email protected] ~]# ``` # 18.4 用keepalived配置高可用集群(中) - 写完脚本之后,还需要对它做一个权限更改,如果没有权限加载这个脚本的话,它就没有办法启动keepalived服务。 ``` [[email protected] ~]# chmod 755 /usr/local/sbin/check_ng.sh [[email protected] ~]# ``` - 启动keepalived 服务 ``` [[email protected] ~]# systemctl start keepalived [[email protected] ~]# [[email protected] ~]# ps aux |grep keep root 2498 12.9 0.1 120720 1468 ? Ss 22:35 0:07 /usr/sbin/keepalived -D root 2499 0.0 0.2 120720 2752 ? S 22:35 0:00 /usr/sbin/keepalived -D root 19605 0.0 0.0 112680 980 pts/0 R+ 22:36 0:00 grep --color=auto keep [[email protected] ~]# ps aux |grep nginx root 868 0.0 0.1 45992 1280 ? Ss 21:55 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 873 0.0 0.4 48480 4180 ? S 21:55 0:00 nginx: worker process nobody 874 0.0 0.3 48480 3924 ? S 21:55 0:00 nginx: worker process root 30104 0.0 0.0 112680 980 pts/0 R+ 22:37 0:00 grep --color=auto nginx [[email protected] ~]# ``` - 先停止nginx 看看是否会自动启动,结果是会的, ``` [[email protected] ~]# /etc/init.d/nginx stop Stopping nginx (via systemctl): [ 确定 ] [[email protected] ~]# [[email protected] ~]# /etc/init.d/nginx stop Stopping nginx (via systemctl): [ 确定 ] [[email protected] ~]# ps aux |grep nginx root 115314 0.0 0.1 45992 1296 ? Ss 22:50 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 115318 0.0 0.3 48480 3940 ? S 22:50 0:00 nginx: worker process nobody 115319 0.0 0.3 48480 3940 ? S 22:50 0:00 nginx: worker process root 115345 0.0 0.0 112680 980 pts/0 S+ 22:51 0:00 grep --color=auto nginx [[email protected] ~]# [[email protected] ~]# date 2017年 11月 08日 星期三 22:51:34 CST [[email protected] ~]# ``` - keepalived,它的日志在哪里 ``` [[email protected] ~]# less /var/log/messages Nov 8 22:48:27 aming-01 Keepalived[2498]: VRRP child process(102776) died: Respawning Nov 8 22:48:27 aming-01 Keepalived[2498]: VRRP child process(102777) died: Respawning Nov 8 22:48:27 aming-01 Keepalived[2498]: VRRP child process(102778) died: Respawning Nov 8 22:48:27 aming-01 Keepalived[2498]: VRRP child process(102779) died: Respawning Nov 8 22:48:27 aming-01 rsyslogd-2177: imjournal: begin to drop messages due to rate-limiting (END) ``` - 来看下它的IP地址,要用ip add ,用ifconfig 看不到的 ``` [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:f2 brd ff:ff:ff:ff:ff:ff inet 192.168.202.131/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.100/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.151/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:fc brd ff:ff:ff:ff:ff:ff inet 192.168.202.134/24 brd 192.168.202.255 scope global dynamic ens37 valid_lft 1259sec preferred_lft 1259sec inet6 fe80::707c:946e:3252:cf7f/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ``` - 使用ifconfig ``` [[email protected] ~]# ifconfig ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.131 netmask 255.255.255.0 broadcast 192.168.202.255 inet6 fe80::ecdd:28b7:612b:cb7 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:2e:28:f2 txqueuelen 1000 (Ethernet) RX packets 7277 bytes 6239399 (5.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 4120 bytes 413974 (404.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.151 netmask 255.255.255.0 broadcast 192.168.202.255 ether 00:0c:29:2e:28:f2 txqueuelen 1000 (Ethernet) ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.134 netmask 255.255.255.0 broadcast 192.168.202.255 inet6 fe80::707c:946e:3252:cf7f prefixlen 64 scopeid 0x20<link> ether 00:0c:29:2e:28:fc txqueuelen 1000 (Ethernet) RX packets 86 bytes 9064 (8.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 20 bytes 3096 (3.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 10 bytes 876 (876.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 10 bytes 876 (876.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [[email protected] ~]# ``` - 配置从之前,查看下主服务器 有没有防火墙, 以及selinux,从 也是一样 ``` [[email protected] ~]# iptables -nvL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [[email protected] ~]# [[email protected] ~]# getenforce Permissive [[email protected] ~]# ``` - 检查下从服务器,如果有停止掉 ``` [[email protected] ~]# systemctl stop firewalld [[email protected] ~]# iptables -nvL Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination [[email protected] ~]# [[email protected] ~]# getenforce Enforcing [[email protected] ~]# setenforce 0 [[email protected] ~]# getenforce Permissive [[email protected] ~]# ``` - 也需要配置下keepalived,先需要情况里面的原始的配置文件内容,再添加如下内容 ``` [[email protected] ~]# > /etc/keepalived/keepalived.conf [[email protected] ~]# vi !$ vi /etc/keepalived/keepalived.conf [[email protected] ~]# vi /etc/keepalived/keepalived.conf [email protected] } notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/usr/local/sbin/check_ng.sh" interval 3 } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass aminglinux>com } virtual_ipaddress { 192.168.202.100 这个网段需要和主一样 } track_script { chk_nginx } } :wq [[email protected] ~]# vi /etc/keepalived/keepalived.conf [[email protected] ~]# ``` - 脚本也需要去写下 ``` [[email protected] ~]# vim /usr/local/sbin/check_ng.sh #时间变量,用于记录日志 d=`date --date today +%Y%m%d_%H:%M:%S` #计算nginx进程数量 n=`ps -C nginx --no-heading|wc -l` #如果进程为0,则启动nginx,并且再次检测nginx进程数量, #如果还为0,说明nginx无法启动,此时需要关闭keepalived if [ $n -eq "0" ]; then systemctl start nginx n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi ~ ~ ~ ~ :wq [[email protected] ~]# vim /usr/local/sbin/check_ng.sh [[email protected] ~]# ``` - 也需要改变权限,不然就启动不了 keepalived ``` [[email protected] ~]# chmod 755 /usr/local/sbin/check_ng.sh [[email protected] ~]# ``` - 启动keepalived ``` [[email protected] ~]# ps aux |grep keep root 2535 0.0 0.1 120720 1404 ? Ss 23:16 0:00 /usr/sbin/keepalived -D root 2536 0.0 0.3 127460 3344 ? S 23:16 0:00 /usr/sbin/keepalived -D root 2537 0.0 0.3 131588 3024 ? S 23:16 0:00 /usr/sbin/keepalived -D root 2601 0.0 0.0 112680 980 pts/0 S+ 23:16 0:00 grep --color=auto keep [[email protected] ~]# ``` - 主从都启动了 keepalived,现在俩边 主从 都有 nginx,那你怎么去区分这个nginx? - 先来看下主的ip ``` [[email protected] ~]# ifconfig ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.131 netmask 255.255.255.0 broadcast 192.168.202.255 inet6 fe80::ecdd:28b7:612b:cb7 prefixlen 64 scopeid 0x20<link> ether 00:0c:29:2e:28:f2 txqueuelen 1000 (Ethernet) RX packets 7410 bytes 6251067 (5.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 5672 bytes 512782 (500.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.151 netmask 255.255.255.0 broadcast 192.168.202.255 ether 00:0c:29:2e:28:f2 txqueuelen 1000 (Ethernet) ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.202.134 netmask 255.255.255.0 broadcast 192.168.202.255 inet6 fe80::707c:946e:3252:cf7f prefixlen 64 scopeid 0x20<link> ether 00:0c:29:2e:28:fc txqueuelen 1000 (Ethernet) RX packets 108 bytes 11608 (11.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24 bytes 3900 (3.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 10 bytes 876 (876.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 10 bytes 876 (876.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [[email protected] ~]# ``` - 先去windows 上用浏览器访问下 ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171108/232640129.png?imageslim) - 这台机器的默认配置文件 ``` [[email protected] ~]# cat /usr/local/nginx/conf/vhost/ aaa.com.conf ld.conf proxy.conf ssl.conf test.com.conf [[email protected] ~]# cat /usr/local/nginx/conf/vhost/aaa.com.conf server { listen 80 default_server; server_name aaa.com; index index.html index.htm index.php; root /data/wwwroot/default; location ~ \.php$ { include fastcgi_params; fastcgi_pass unix:/tmp/aming.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME /data/wwwroot/default$fastcgi_script_name; } } [[email protected] ~]# [[email protected] ~]# vim !$ vim /data/wwwroot/default/index.html master master. This is the default site. ~ ~ :wq ``` - ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171108/233113698.png?imageslim) - 关于backup ,从服务器 上面,它的nginx 是yum安装的 ``` [[email protected] ~]# vim /usr/share/nginx/html/index.html [[email protected] ~]# cat !$ cat /usr/share/nginx/html/index.html backup backup. [[email protected] ~]# ``` ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171108/233606639.png?imageslim) ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171109/000815047.png?imageslim) - 查看下nginx服务,这个是我们手动起来的,可以停掉试下 ``` [[email protected] ~]# ps aux |grep nginx root 2561 0.0 0.2 122792 2084 ? Ss 23:16 0:00 nginx: master process /usr/sbin/nginx nginx 2562 0.0 0.3 123224 3576 ? S 23:16 0:00 nginx: worker process root 6985 0.0 0.0 112680 976 pts/0 S+ 23:43 0:00 grep --color=auto nginx [[email protected] ~]# [[email protected] ~]# systemctl stop nginx [[email protected] ~]# !ps ps aux |grep nginx root 7155 0.0 0.2 122792 2084 ? Ss 23:44 0:00 nginx: master process /usr/sbin/nginx nginx 7156 0.0 0.3 123224 3128 ? S 23:44 0:00 nginx: worker process root 7161 0.0 0.0 112680 980 pts/0 S+ 23:44 0:00 grep --color=auto nginx [[email protected] ~]# ``` - 因为开启了keepalived 服务,所以即使停掉nginx服务,它也会自动加载起来的 # 18.5 用keepalived配置高可用集群(下) - 先确定好两台机器上nginx差异,比如可以通过curl -I 来查看nginx版本 - 测试1:关闭master上的nginx服务 我们之前都已经测试过了,关闭还是会继续运行,因为加了那个脚本 开启了keepalived服务 - 测试2:在master上增加iptabls规则 - iptables -I OUTPUT -p vrrp -j DROP - 测试3:关闭master上的keepalived服务 - 测试4:启动master上的keepalived服务 - 这个vip 是在主上 master 上 ``` [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:f2 brd ff:ff:ff:ff:ff:ff inet 192.168.202.131/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.100/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.151/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:fc brd ff:ff:ff:ff:ff:ff inet 192.168.202.134/24 brd 192.168.202.255 scope global dynamic ens37 valid_lft 1369sec preferred_lft 1369sec inet6 fe80::707c:946e:3252:cf7f/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ``` - backup上是没有的 ``` [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:58:33:e6 brd ff:ff:ff:ff:ff:ff inet 192.168.202.132/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.152/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::4500:6d42:8612:4e53/64 scope link valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link tentative dadfailed valid_lft forever preferred_lft forever [[email protected] ~]# ``` - 咱们在主上加一个防火墙规则 - 把主上的vrrp进去的包给封掉 iptables -I OUTPUT -p vrrp -j DROP ``` [[email protected] ~]# iptables -I OUTPUT -p vrrp -j DROP [[email protected] ~]# [[email protected] ~]# iptables -nvL Chain INPUT (policy ACCEPT 109 packets, 5868 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 32 packets, 2968 bytes) pkts bytes target prot opt in out source destination 74 2960 DROP 112 -- * * 0.0.0.0/0 0.0.0.0/0 [[email protected] ~]# ``` - 看下日志 ``` [[email protected] ~]# tail /var/log/messages Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: VRRP_Instance(VI_1) Received advert with lower priority 90, ours 100, forcing new election Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:21:58 aming-01 Keepalived_vrrp[124429]: Sending gratuitous ARP on ens33 for 192.168.202.100 [[email protected] ~]# ``` - 再看下从 ``` [[email protected] ~]# less /var/log/messages Nov 7 22:01:01 aming-02 systemd: Started Session 4 of user root. Nov 7 22:01:01 aming-02 systemd: Starting Session 4 of user root. Nov 7 23:01:01 aming-02 systemd: Started Session 5 of user root. Nov 7 23:01:01 aming-02 systemd: Starting Session 5 of user root. Nov 7 23:49:27 aming-02 kernel: hrtimer: interrupt took 6003226 ns Nov 8 00:01:01 aming-02 systemd: Started Session 6 of user root. Nov 8 00:01:01 aming-02 systemd: Starting Session 6 of user root. Nov 8 00:33:47 aming-02 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="834" x-info="http://www.rsyslog.com"] exiting on signal 15. Nov 8 21:55:29 aming-02 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="827" x-info="http://www.rsyslog.com"] start Nov 8 21:55:09 aming-02 kernel: Initializing cgroup subsys cpuset Nov 8 21:55:09 aming-02 kernel: Initializing cgroup subsys cpu Nov 8 21:55:09 aming-02 kernel: Initializing cgroup subsys cpuacct Nov 8 21:55:09 aming-02 kernel: Linux version 3.10.0-514.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Tue Nov 22 16:42:41 UTC 2016 Nov 8 21:55:09 aming-02 kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-514.el7.x86_64 root=UUID=9a2a0d05-a7ab-4948-bbd2-77b2b7d5f565 ro crashkernel=auto rhgb quiet LANG=zh_CN.UTF-8 Nov 8 21:55:09 aming-02 kernel: Disabled fast string operations Nov 8 21:55:09 aming-02 kernel: e820: BIOS-provided physical RAM map: Nov 8 21:55:09 aming-02 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ebff] usable Nov 8 21:55:09 aming-02 kernel: BIOS-e820: [mem 0x000000000009ec00-0x000000000009ffff] reserved Nov 8 21:55:09 aming-02 kernel: BIOS-e820: [mem 0x00000000000dc000-0x00000000000fffff] reserved /var/log/messages ``` - ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171109/002641992.png?imageslim) - 这个也照常可以访问,说明直接封这里的ip是不能直接达到目的的 - 先把这个iptables 规则恢复 ``` [[email protected] ~]# iptables -F [[email protected] ~]# ``` - 那怎么样模仿现实中宕机的环境, - 只能暂停keepalived服务试下 ``` [[email protected] ~]# systemctl stop keepalived [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:f2 brd ff:ff:ff:ff:ff:ff inet 192.168.202.131/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.151/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:fc brd ff:ff:ff:ff:ff:ff inet 192.168.202.134/24 brd 192.168.202.255 scope global dynamic ens37 valid_lft 1452sec preferred_lft 1452sec inet6 fe80::707c:946e:3252:cf7f/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ``` - 再去从服务器看看 ``` [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:58:33:e6 brd ff:ff:ff:ff:ff:ff inet 192.168.202.132/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.100/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.152/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::4500:6d42:8612:4e53/64 scope link valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link tentative dadfailed valid_lft forever preferred_lft forever [[email protected] ~]# [[email protected] ~]# tail /var/log/messages Nov 9 00:29:54 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:54 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:54 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:54 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 [[email protected] ~]# ``` - 访问日志可以看到,ip被增加了,现在来访问下web - ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171109/003125547.png?imageslim) - 可以看到发生了改变,说明vip 已经到了从机器上 - 那再把这个keepalived服务 开启 ``` [[email protected] ~]# systemctl start keepalived [[email protected] ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:f2 brd ff:ff:ff:ff:ff:ff inet 192.168.202.131/24 brd 192.168.202.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.100/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.202.151/24 brd 192.168.202.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::ecdd:28b7:612b:cb7/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2e:28:fc brd ff:ff:ff:ff:ff:ff inet 192.168.202.134/24 brd 192.168.202.255 scope global dynamic ens37 valid_lft 1276sec preferred_lft 1276sec inet6 fe80::707c:946e:3252:cf7f/64 scope link valid_lft forever preferred_lft forever [[email protected] ~]# ``` - 可以看到马上就能变回来,再看下web访问 - ![mark](http://oqxf7c508.bkt.clouddn.com/blog/20171109/003318786.png?imageslim) - 看下从上面的日志, 显示 removing protocol VIPs ``` [[email protected] ~]# tail /var/log/messages Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:29:59 aming-02 Keepalived_vrrp[10474]: Sending gratuitous ARP on ens33 for 192.168.202.100 Nov 9 00:32:17 aming-02 chronyd[482]: Selected source 120.25.108.11 Nov 9 00:32:17 aming-02 chronyd[482]: System clock wrong by -1.225785 seconds, adjustment started Nov 9 00:32:51 aming-02 Keepalived_vrrp[10474]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 90 Nov 9 00:32:51 aming-02 Keepalived_vrrp[10474]: VRRP_Instance(VI_1) Entering BACKUP STATE Nov 9 00:32:51 aming-02 Keepalived_vrrp[10474]: VRRP_Instance(VI_1) removing protocol VIPs. [[email protected] ~]# ``` - 因为主机器宕机,从机器很快的就加入了vip地址,这个时候访问vip地址的时候,看到的页面是 - 从机器上的默认索引页,证明整个实验成功 - 这就是高可用的一个集群,通常会用俩台机器,有的企业为了保险,可能会用2-3台backup
时间: 2024-08-09 21:58:29