Nginx+Keepalived搭建高可用负载均衡集群
一。 环境说明
前端双Nginx+keepalived,nginx反向代理到后端的tomcat集群实现负载均衡，Keepalived实现集群高可用.
- 操作系统： Centos 6.6_X64
- Nginx版本： nginx-1.9.5
- Keepalived版本：keepalived-1.2.13
结构:
Keepalived+nginx-MASTER：10.6.1.210
Keepalived+nginx-BACKUP: 10.6.1.211
VIP: 10.6.1.1.214
Realsever:10.6.1.212,10.6.1.213
二、安装
前端两台主机分别安装nginx和keepalived。
1. 安装keepalived
Yum install keepalived –y
默认安装目录: /etc/keepalived/
配置文件:/etc/keepalived/keepalived.conf
2. 安装Nginx
1) 安装nginx所依赖的包.
yum install gcc gcc-c++ autoconf automake zlib zlib-developenssl openssl-devel pcre pcre-deve –y
2）Nginx官网下载安装包 (http://nginx.org/en/download.html)
此处版本选择nginx-1.9.5.tar.gz
3）编译安装
tar –zxvf nginx-1.9.5.tar.gz
cd nginx-1.9.5
./configure
Make && make install
完成
三、配置
1）前端两台主机nginx的配置完全一样
Vim /usr/local/nginx/conf/nginx.conf
upstream tomcat_8080 {
server10.6.1.212:8080;
server10.6.1.213:8080;
ip_hash;
}
server {
listen 80;
server_name test;
#charsetkoi8-r;
#access_log logs/host.access.log main;
location /{
proxy_passhttp://tomcat_8080;
}
2）nginx_master的keepalived配置
Vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_scriptcheck_nginx {
#检查nginx状态的脚本,后面会介绍.同理mysql也可以这样做。
script"/etc/keepalived/check_nginx.sh"
# 执行间隔2秒
interval 2
}
#VIP1
vrrp_instance VI_1{
state BACKUP #两台主机都是BACKUP
interface eth0
#同一keepalived集群的virtual_router_id必须相同，默认51
virtual_router_id 51
priority 100 #主的优先级高
advert_int 1
#不抢占：如果集群里已存在MASTER状态的主机，即使优先级高于MASTER也不抢占为MASTER。只在优先级高的主机上设置即可。
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
#虚拟IP
10.6.1.214
}
track_script {
check_nginx
}
}
3）nginx_backup的keepalived配置
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_scriptcheck_nginx {
script"/etc/keepalived/check_nginx.sh"
# 执行间隔2秒
interval 2
}
#VIP1
vrrp_instance VI_1{
state BACKUP
interface eth0
virtual_router_id 51
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.6.1.214
}
track_script {
check_nginx
}
}
4）防火墙设置
Iptables -I INPUT -d 224.0.0.18 -j ACCEPT
或者直接关闭防火墙service iptables stop
VRRP报文是通过IP多播形式发送的，组播地址224.0.0.18是VRRP报文的目的地址。
两个主机都是BACKUP，如果同时启动keepalived，VRRP协议通过竞选使优先级高的主机做为MASTER。如果防火墙没有允许VRRP报文通过的话，两个BACKUP都会成为MASTER，你会发现两个主机都启动了虚拟IP。
5）nginx状态检查脚本check_nginx.sh
Vim /etc/keepalived/check_nginx.sh
#!/bin/bash
#This script is used to check_nginx status for keepalived
KK=`ps -ef |grep "nginx: master process" | grep -v grep | wc -l`
if [ $KK -eq 0 ]
then
echo "`date`nginx is dead,prepare to startnginx">>/usr/local/nginx/logs/nginx_status.log 2>&1
/usr/local/nginx/sbin/nginx_start.sh
sleep 5
else
echo "`date` nginx isOK">>/usr/local/nginx/logs/nginx_status.log 2>&1
fi
6）开启keepalived的日志
编辑/etc/sysconfig/keepalived：
|
1 |
KEEPALIVED_OPTIONS="-D -d -S 0" |
编辑/etc/rsyslog.conf：
# 配置文件最后面加上下面一行
local0.* /var/log/keepalived.log
重启rsyslog：
service rsyslog restart
按上面配置后，keepalived会把日志记录到/var/log/keepalived.log。
7）启动服务
设置nginx环境变量。
+++++++++++++++++++++++++++++++++++++++++++++
NGINX=/usr/local/nginx
PATH=$PATH:$HOME/bin:$NGINX/sbin
export NGINX PATH
++++++++++++++++++++++++++++++++++++++++++++++
启动nginx, 用脚本启动, 脚本如下：
+++++++++++++++++++++++++++++++++++++++++++
#!/bin/bash
. $HOME/.bash_profile
nohup /usr/local/nginx/sbin/nginx -c/usr/local/nginx/conf/nginx.conf >/dev/null 2>&1 &
+++++++++++++++++++++++++++++++++++++++++++++++++
启动keepalived
service keepalived start
四、验证
1. nginx_master和nginx_slave同时启动keepalived，观察日志/var/log/keepalived.log，你会发现nginx_master抢占为MASTER，绑定了虚拟IP。
nginx_master：
nginx_backup：
2. 把nginx_master的keepalived服务停掉或者重启系统，同时不断的ping虚拟IP。会出现短暂的中断或者时延较大，虚拟IP会漂移到nginx_backup上面：
nginx_backup：
注意观察在主备切换时nginx_backup日志keepalived.log的变化，如下：
VRRP_Instance(VI_1) Transition to MASTER STATE
VRRP_Instance(VI_1) Entering MASTER STATE
VRRP_Instance(VI_1) setting protocol VIPs.
VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.6.1.214
Netlink reflector reports IP 10.6.1.214 added
VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.6.1.214
3. 最后把nginx_master的keepalived服务开启，虚拟IP并没有漂移回到nginx_master，这是因为nginx_master开启了不抢占模式，即使优先级高，也不会抢占MASTER。