第1章 Keepalived高可用服务
1.1 Keepalived介绍
Keepalived软件起初是专为LVS负载均衡软件设计的, 用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能 Keepalived软件主要是通过VRRP协议实现高可用功能的。VRRP是Virtual Router Redundancy Protocol(虚拟路由器冗余协议)的缩写, VRRP出现的目的就是为了解决静态路由单点故障问题的,它能够保证当个别节点宕机时,整个网络可以不间断地运行 |
1.2 keepalived软件主要功能
①. 管理LVS负载均衡软件 ②. 实现对LVS集群节点健康检查功能 ① . 作为系统网络服务的高可用功能 |
1.3 VRRP协议原理
1)VRRP协议,全称Virtual Router Redundancy Protocol,中文名为虚拟路由冗余协议,VRRP的出现是为了解决静态路由的单点故障。 2)VRRP是用过IP多播的方式(默认多播地址(224.0.0.18))实现高可用对之间通信的。 3)工作时主节点发包,备节点接包,当备节点接收不到主节点发的数据包的时候,就启动接管程序接管主节点的资源。备节点可以有多个,通过优先级竞选,但一般Keepalived系统运维工作中都是一对。 |
1.4 环境准备说明:
lb01 10.0.0.5 lb02 10.0.0.6 web01 10.0.0.8 web02 10.0.0.7 web03 10.0.0.9 |
web集群服务器配置文件环境统一(web01 web02 web03 配置均一致) server { listen 80; server_name bbs.etiantian.org; root html/bbs; index index.html index.htm; } server { listen 80; server_name www.etiantian.org; root html/www; index index.html index.htm; } } |
同步三台web服务器配置: scp -rp {www.conf,bbs.conf} 172.16.1.7:/application/nginx/conf/extra/ scp -rp {www.conf,bbs.conf} 172.16.1.9:/application/nginx/conf/extra/ |
在lb01 lb02上测试web集群服务器是否正常响应访问请求: curl -H host:www.etiantian.org 10.0.0.7/wuxing.html curl -H host:bbs.etiantian.org 10.0.0.7/wuxing.html curl -H host:www.etiantian.org 10.0.0.8/wuxing.html curl -H host:bbs.etiantian.org 10.0.0.8/wuxing.html curl -H host:www.etiantian.org 10.0.0.9/wuxing.html curl -H host:bbs.etiantian.org 10.0.0.9/wuxing.html |
nginx反向代理负载均衡集群服务器配置文件环境统一 [[email protected] conf]# cat nginx.conf ####lb01和lb02 nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; upstream server_pools { server 10.0.0.7:80; server 10.0.0.8:80; server 10.0.0.9:80; } server { listen 80; server_name www.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } } server { listen 80; server_name bbs.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } } } |
#测试访问lb01 和 lb02 是否实现了负载均衡 curl -H host:www.etiantian.org 10.0.0.5/wuxing.html curl -H host:bbs.etiantian.org 10.0.0.5/wuxing.html curl -H host:www.etiantian.org 10.0.0.6/wuxing.html curl -H host:bbs.etiantian.org 10.0.0.6/wuxing.html |
1.5 keepalived服务部署
## 第一个里程碑:keepalived软件安装部署 ### lb01 lb02负载服务器上均安装 yum install -y keepalived rpm -qa keepalived rpm -ql keepalived [[email protected] conf]# rpm -ql keepalived /etc/keepalived /etc/keepalived/keepalived.conf --- keepalived服务主配置文件 /etc/rc.d/init.d/keepalived --- keepalived服务启动脚本文件 |
## 第二个里程碑:进行默认配置测试 ### 启动lb01 lb02的keepalived服务 /etc/init.d/keepalived start ip addr 说明:存在默认配置虚IP地址信息 通过抓包可以看到vrrp数据包信息 |
## 第三个里程碑:进行服务配置文件编写 ### 前提需要了解配置文件内容信息(man keepalived.conf) ### 配置文件的组成部分 · GLOBAL CONFIGURATION ###全局定义(默认配置文件的01-13行) · VRRPD CONFIGURATION ###虚拟ip的配置(默认配置文件15-30行) · LVS CONFIGURATION ###配置与管理lvs ! Configuration File for keepalived global_defs { --- 全局配置标题 notification_email { --- 定义管理员邮箱信息, [email protected] [email protected] } notification_email_from [email protected] --- 定义利用什么邮箱发送邮件 smtp_server smtp.163.com --- 定义邮件服务器信息 smtp_connect_timeout 30 --- 定义邮件发送超时时间 router_id oldboy01 --- (重点参数)局域网keepalived主机身份标识信息 每一个keepalived主机身份标识信息唯一 } vrrp_instance VI_1 { --- vrrp协议相关配置(vip地址设置) state MASTER --- keepalived角色描述(状态)信息,可以配置参数(MASTER BACKUP) interface eth0 --- 表示将生成虚IP地址,设置在指定的网卡上 virtual_router_id 51 --- 表示keepalived家族标识信息 priority 100 --- keepalived服务竞选主备服务器优先级设置(越大越优先) advert_int 1 --- 主服务组播包发送间隔时间 authentication { --- 主备主机之间通讯认证机制, auth_type PASS --- 采用明文认证机制 auth_pass 1111 --- 编写明文密码 } virtual_ipaddress { --- 设置虚拟IP地址信息 10.0.0.3 } } |
### 搭建基础的keepalived配置文件 #lb01 global_defs { router_id LVS_01 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3 } } ################################################ ################################################ #lb02 global_defs { router_id LVS_02 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3 } } |
说明:主备服务器配置文件区别 01. router_id 不同 02. state BACKUP 不同 03. priority 不同 说明:进行抓包观察配置效果;并且对比两个负载均衡服务器的配置文件 |
1.6 高可用集群排错思路
1) 确认lb01 lb02 访问后端web服务是否正常 2)确认用户分别访问lb01 lb02 是否有问题 3)利用vip地址进行访问相应网站 4)做好windows host文件解析 说明:解析时,不能一个域名解析为多个IP地址 |
1.7 keepaliver软件脑裂概念说明
## 开启防火墙即可模拟出脑裂的情况 /etc/init.d/iptables start ### 脑裂情况出现的原因 ### 脑裂情况解决的方法 #### 制作监控脚本---lb02 报警的条件:只要lb02 上面有vip 1.lb01 挂了 2.心碎 #!/bin/bash #desc: jiankong lb02 vip if [ `ip a s eth0 |grep -c "10.0.0.3"` == 1 ];then echo "baojing" fi |
1.8 Nginx负载均衡实现高可用
1)统一lb01 lb02 反向代理 配置文件 ####lb01 nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; upstream server_pools { server 10.0.0.7; server 10.0.0.8; server 10.0.0.9; } server { listen 80; server_name bbs.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_www.log main; } server { listen 80; server_name www.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_blog.log main; } } |
2)进行测试 ## 01 测试10.0.0.5 lb01服务器 curl -H Host:www.etiantian.org 10.0.0.5/nana.html curl -H Host:bbs.etiantian.org 10.0.0.5/nana.html ## 01 测试10.0.0.6 lb01服务器 curl -H Host:www.etiantian.org 10.0.0.6/nana.html curl -H Host:bbs.etiantian.org 10.0.0.6/nana.html 说明:通过以上测试,确认两台lb服务器,均可实现负载调度功能 3)把域名解析到 vip上面 10.0.0.3 www.etiantian.org blog.etiantian.org bbs.etiantian.org |
## 问题小结: 1.是否解析 ping 2.浏览器缓存 3.服务没重启(平滑重启) |
排错过程:(最小化排错) 01:利用负载服务器,在服务器上curl所有节点信息(web服务器配置有问题) --- web服务器有问题 02;curl 负载均衡服务器地址,可以实现负载均衡 --- nginx反向代理有问题 03:windows绑定虚拟IP,浏览器上进行测试 --- keepalived配置或运行有问题 DNS解析 keepalived日志文件(运维能力:看日志) tail -f /var/log/messages |
1.9 企业keepalived服务应用:
1.9.1 实践案例一:更改nginx反向代理只监听vip地址
10.0.0.3/nana.html 可以使用 10.0.0.5/nana.html 不可以使用 10.0.0.6/nana.html 不可以使用 |
第一个里程碑:修改反向代理服务配置文件,只监听vip地址 ####lb01 lb02 nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; upstream server_pools { server 10.0.0.7; server 10.0.0.8; server 10.0.0.9; } server { listen 10.0.0.3:80; server_name www.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_www.log main; } server { listen 10.0.0.3:80; server_name blog.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_blog.log main; } } 说明:在修改反向代理服务器配置文件监听地址时,多个server都需要配置监听地址,否则仍旧使用默认监听所有 |
第二个里程碑:lb02上不存在vip地址,无法监听,需要修改内核文件 [[email protected] conf]# /application/nginx/sbin/nginx -t nginx: the configuration file /application/nginx-1.10.2/conf/nginx.conf syntax is ok nginx: [emerg] bind() to 10.0.0.3:80 failed (99: ) nginx: configuration file /application/nginx-1.10.2/conf/nginx.conf test failed [[email protected] conf]# ip a s eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:27:4e:e9 brd ff:ff:ff:ff:ff:ff inet 10.0.0.5/24 brd 10.0.0.255 scope global eth0 inet6 fe80::20c:29ff:fe27:4ee9/64 scope link valid_lft forever preferred_lft forever [[email protected] conf]# ###nginx 没有办法 监听 本地不存在的ip地址 解决方法: echo 'net.ipv4.ip_nonlocal_bind = 1' >>/etc/sysctl.conf ---实现监听本地不存在的ip地址 ##/etc/sysctl.conf 加上 sysctl -p echo "1" >/proc/sys/net/ipv4/ip_nonlocal_bind |
第三个里程碑:进行测试 |
1.9.2 企业实践案例二:让keepalived监控nginx反向代理服务
####第一个里程碑-keepalived监控nginx条件 1.如何nginx挂了---我如何知道nginx挂了? 1)端口 2)进程 ps -ef |grep nginx |grep -v grep |wc -l 2.keepalived挂了 /etc/init.d/keepalived stop ##> -gt greater than ##>= -ge greater equal ##< -lt less than ##<= -le less equal ##== -eq equal ##!= -ne no equal |
####第二个里程碑-根据条件-书写脚本 #!/bin/bash #name: check_web.sh #desc: check nginx and kill keepalived if [ `ps -ef |grep nginx |grep -v grep |wc -l` -lt 2 ];then /etc/init.d/keepalived stop Fi |
####第三个里程碑-添加权限 [[email protected] conf]# chmod +x /server/scripts/check_web.sh [[email protected] conf]# ll /server/scripts/check_web.sh -rwxr-xr-x 1 root root 174 Mar 30 17:47 /server/scripts/check_web.sh |
####第四个里程碑-测试 |
####第五个里程碑-放入到keepalived.conf ####下面是lb02的配置文件 lb01上面自己修改下。 global_defs { router_id LVS_02 } vrrp_script check_web { script "/server/scripts/web_jiankong.sh" --- 表示将一个脚本信息赋值给变量check_web interval 2 --- 执行监控脚本的间隔时间 weight 2 --- 利用权重值和优先级进行运算,从而降低主服务优先级 使之变为备服务器(建议先忽略) } /server/scripts/check_web.sh vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3/24 dev eth0 label eth0:1 } track_script { check_web } $check_web } |
####第六个里程碑-测试 |
1.9.3 企业实践案例三:keepalived多实例配置
####第一个里程碑-配置keepalived-配置双主 ####lb01 #lb01 global_defs { router_id LVS_01 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3/24 dev eth0 label eth0:1 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.4/24 dev eth0 label eth0:2 } } #lb02 global_defs { router_id LVS_02 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.3/24 dev eth0 label eth0:1 } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 52 priority 150 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 10.0.0.4/24 dev eth0 label eth0:2 } } |
#########第二个里程碑-配置nginx 负载均衡 ####lb01 lb02 nginx.conf worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; upstream server_pools { server 10.0.0.7; server 10.0.0.8; server 10.0.0.9; } server { listen 10.0.0.3:80; server_name www.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_www.log main; } server { listen 10.0.0.4:80; server_name blog.etiantian.org; location / { proxy_pass http://server_pools; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $remote_addr; } access_log logs/access_blog.log main; } } |
#########第三个里程碑-windows hosts解析 10.0.0.3 www.etiantian.org 10.0.0.4 bbs.etiantian.org |
#########第四个里程碑-浏览器进行测试 |
原文地址:http://blog.51cto.com/zhuzhiwei/2136365