Nginx+Keepalived实现Nginx负载均衡及高可用WEB服务器集群

环境：
操作系统Centos 6.5 X86_64(final)
Nginx-Master:192.168.2.32
Nginx-Backup:192.168.3.31
VIP:192.168.2.33
Web3:192.168.2.29
Web4:192.168.2.30
1、分别在Nginx-Master和Nginx-Backup上安装nginx
[[email protected] ~]# rpm -i http://nginx.org/packages/centos/6/noarch/RPMS/nginx-release-centos-6-0.el6.ngx.noarch.rpm
[[email protected] ~]# rpm -i http://nginx.org/packages/centos/6/noarch/RPMS/nginx-release-centos-6-0.el6.ngx.noarch.rpm

[[email protected] ~]# yum install nginx

[[email protected] ~]# yum install nginx

2、先配置Nginx-Master上的nginx，建立配置一个proxy.conf文件。并复制一份到Nginx-Backup上，删除两台机器上默认的default.conf,启动nginx服务；并将服务添加到开机启动。
[[email protected] ~]# vi /etc/nginx/conf.d/proxy.conf
    upstream web {
    ＃ip_hash;
        server 192.168.2.29:80;   #默认为rr轮询，如需解决session的问题采有哈希（ip_hash)模块。
        server 192.168.2.30:80;
}
        server {
        listen 80;
        index index.php index.html index.htm;
        location / {
        proxy_pass http://web;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}

[[email protected] ~]# scp /etc/nginx/conf.d/proxy.conf [email protected]:/etc/nginx/conf.d/

[[email protected] ~]# service nginx start
Starting nginx:                                    [  OK  ]

[[email protected] ~]# service nginx start
Starting nginx:                                    [  OK  ]

[[email protected] ~]# chkconfig nginx on

[[email protected] ~]# chkconfig nginx on

3、分别在Nginx-Master和Nginx-Backup上安装keepalived并配置。

[[email protected] ~]# yum install keepalived
[[email protected] ~]# yum install keepalived

[[email protected] ~]# vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_nginx {                         #监控nginx服务进程脚本
    script "/root/nginx.sh"          
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    mcast_src_ip 192.168.2.32
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {                                    #调用监控脚本
        chk_nginx
}
    virtual_ipaddress {
        192.168.2.33
    }
}

[[email protected] ~]# vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_nginx {                        #监控nginx服务进程脚本
    script "/root/nginx.sh"       
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    mcast_src_ip 192.168.2.31
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {                                  #调用监控脚本
        chk_nginx

virtual_ipaddress {
        192.168.2.33
    }
}

[[email protected] ~]# service keepalived start
Starting keepalived:                                   [  OK  ]

[[email protected] ~]# service keepalived start
Starting keepalived:                                   [  OK  ]

3.1.对keepalived的不足写的一个脚本，用来检测本机的nginx是否正常的运行，如果nginx挂掉试着重新启动，如果启动后又挂掉，那么就直接停止keepalived进程，keepalived将转移到另一台备用上，实现故障转移
[[email protected] ~]# vi /root/nginx.sh
#!/bin/bash
A=`ps -C nginx --no-header |wc -l`                      # 查看是否有 nginx进程 把值赋给变量A
if [ $A -eq 0 ];then                                                #如果没有进程值为零
                /usr/sbin/nginx
                sleep 1
                if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then
                       /etc/init.d/keepalived stop             # 则停止keepalived 进程
                fi
fi

[[email protected] ~]# chmod 755 /root/nginx.sh

[[email protected] ~]# scp /root/nginx.sh [email protected]:/root/nginx.sh

3.2.重启keepalived服务，查看日志脚本是否正常
Nginx-Master日志
[[email protected] ~]# tail -f /var/log/messages
Sep  5 15:20:48 Nginx-Master Keepalived_vrrp[5136]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Sep  5 15:20:48 Nginx-Master Keepalived_vrrp[5136]: VRRP_Script(chk_nginx) succeeded                                                                     ＃检查脚本成功，表示正常
Sep  5 15:20:48 Nginx-Master Keepalived_vrrp[5136]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  5 15:20:49 Nginx-Master Keepalived_vrrp[5136]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep  5 15:20:49 Nginx-Master Keepalived_vrrp[5136]: VRRP_Instance(VI_1) setting protocol VIPs.

Nginx-Backup日志
[[email protected] ~]# tail -f /var/log/messages
Sep  5 15:20:41 Nginx-Backup Keepalived_vrrp[7670]: Using LinkWatch kernel netlink reflector...
Sep  5 15:20:41 Nginx-Backup Keepalived_vrrp[7670]: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep  5 15:20:41 Nginx-Backup Keepalived_vrrp[7670]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Sep  5 15:20:41 Nginx-Backup Keepalived_vrrp[7670]: VRRP_Script(chk_nginx) succeeded                                                                     #检查脚本成功，表示正常

[[email protected] ~]# tail -f /var/log/messages
Sep  5 15:10:39 Nginx-Backup Keepalived_vrrp[5265]: Process [5913] didn‘t respond to SIGTERM
Sep  5 15:10:41 Nginx-Backup Keepalived_vrrp[5265]: Process [5923] didn‘t respond to SIGTERM                                                                        #如出现这种，看下脚本是否正确。
Sep  5 15:10:43 Nginx-Backup Keepalived_vrrp[5265]: Process [5933] didn‘t respond to SIGTERM

3.3.另外一种查看脚本是否生效方法,停止nginx服务，马上会自动启动

[[email protected] ~]# service nginx status
nginx (pid  1114) is running...
[[email protected] ~]# service nginx stop
Stopping nginx:                                            [  OK  ]
[[email protected] ~]# service nginx status
nginx (pid  28751) is running...

4、测试VIP，检查是否能主备切换

4.1.当两台主机同时启动时，只有Nginx-Master服务器拥有VIP地址，备服务器没有。

[[email protected] ~]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:a6:00:13 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.32/24 brd 192.168.2.255 scope global eth0
    inet 192.168.2.33/32 scope global eth0                        #可以看到Nginx-Master服务器上拥有192.168.2.33这个VIP地址
    inet6 fe80::250:56ff:fea6:13/64 scope link
       valid_lft forever preferred_lft forever

[[email protected] ~]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 08:00:27:77:d3:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.31/24 brd 192.168.2.255 scope global eth0
    inet6 fe80::a00:27ff:fe77:d382/64 scope link                #Nginx-Backup服务器上没有  
       valid_lft forever preferred_lft forever

4.2.当停止Nginx-Master服务器的keepalived服务，再查看下两台主机的VIP地址，发现VIP地址已从主服务器转移到了备服务器

[[email protected] ~]# service keepalived stop
Stopping keepalived:                  [  OK  ]

[[email protected] ~]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:a6:00:13 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.32/24 brd 192.168.2.255 scope global eth0
    inet6 fe80::250:56ff:fea6:13/64 scope link                #Nginx-Master服务器VIP地址已移除      
       valid_lft forever preferred_lft forever

[[email protected] ~]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 08:00:27:77:d3:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.31/24 brd 192.168.2.255 scope global eth0
    inet 192.168.2.33/32 scope global eth0         #VIP地地已自动转移到Nginx-Backup服务器上            
    inet6 fe80::a00:27ff:fe77:d382/64 scope link
       valid_lft forever preferred_lft forever

5、日志查看主备切换过程

5.1.当停止Nginx-Master上的keepalived服务时
[email protected] ~]# service keepalived stop
Nginx-Master日志
[[email protected] ~]# tail -f /var/log/messages
Sep  4 18:04:06 Nginx-Master Keepalived[3278]: Stopping Keepalived v1.2.7 (02/21,2013)                                                                                  ＃主服务器已停掉
Sep  4 18:04:06 Nginx-Master Keepalived_vrrp[3281]: VRRP_Instance(VI_1) sending 0 priority    
Sep  4 18:04:06 Nginx-Master Keepalived_vrrp[3281]: VRRP_Instance(VI_1) removing protocol VIPs.

Nginx-Backup日志
[[email protected] ~]# tail -f /var/log/messages
Sep  4 18:04:07 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  4 18:04:08 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Entering MASTER STATE                                                                  #Nginx-Backup转为MASTER STATE
Sep  4 18:04:08 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep  4 18:04:08 Nginx-Backup Keepalived_healthcheckers[1427]: Netlink reflector reports IP 192.168.2.33 added
Sep  4 18:04:08 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
Sep  4 18:04:13 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33

5.2.当Nginx-Master的keepalived服务再次启动时
[[email protected] ~]# service keepalived restart
Stopping keepalived:                     [  OK  ]
Starting keepalived:                     [  OK  ]
Nginx-Master日志
[[email protected] ~]# tail -f /var/log/messages
Sep  4 18:06:47 Nginx-Master Keepalived_vrrp[3316]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  4 18:06:48 Nginx-Master Keepalived_vrrp[3316]: VRRP_Instance(VI_1) Entering MASTER STATE                                                   #Nginx-Master转回MASTER STATE
Sep  4 18:06:48 Nginx-Master Keepalived_vrrp[3316]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep  4 18:06:48 Nginx-Master Keepalived_healthcheckers[3315]: Netlink reflector reports IP 192.168.2.33 added
Sep  4 18:06:48 Nginx-Master Keepalived_vrrp[3316]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
Sep  4 18:06:53 Nginx-Master Keepalived_vrrp[3316]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33

Nginx-Backup日志
[[email protected] ~]# tail -f /var/log/messages
Sep  4 18:06:47 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Received higher prio advert
Sep  4 18:06:47 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) Entering BACKUP STATE                                                   #Nginx-Backup转回BACKUP STATE
Sep  4 18:06:47 Nginx-Backup Keepalived_vrrp[1428]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep  4 18:06:47 Nginx-Backup Keepalived_healthcheckers[1427]: Netlink reflector reports IP 192.168.2.33 removed

6、安装后端两台httpd服务器，并添加内容测试
[[email protected] ~]# yum install httpd

[[email protected] ~]# vi /var/www/html/index.html
<h1>Welcome to web3(192.168.2.29)</h1>

[[email protected] ~]# service httpd start

[[email protected] ~]# yum install httpd
[[email protected] ~]# vi /var/www/html/index.html
<h1>Welcome to web4(192.168.2.30)</h1>

[[email protected] ~]# service httpd start

7.查看后端web服务器日志

7.1.后端web服务器日志，访问IP还是代理服务器的，不是真实客户端IP
[[email protected] ~]# tail -f /var/log/httpd/access_log
192.168.2.31 - - [05/Sep/2014:15:47:46 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"
192.168.2.32 - - [05/Sep/2014:15:58:02 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"

[[email protected] ~]# tail -f /var/log/httpd/access_log
192.168.2.31 - - [05/Sep/2014:15:48:47 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"
192.168.2.32 - - [05/Sep/2014:15:59:55 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"

7.2.虽然在配置proxy.conf时已配置了proxy_set_header X-Real-IP $remote_addr；但还需要在web端修改配围起文件显示
[[email protected] ~]# vi /etc/httpd/conf/httpd.conf
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
改为:
LogFormat "%{X-Real-IP}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
web4也同样修改以上数据后重启

[[email protected] ~]# service httpd restart
[[email protected] ~]# service httpd restart

7.3.最后再查看后端web服务器日志为真实客户端IP

[[email protected] ~]# tail -f /var/log/httpd/access_log
192.168.2.200 - - [05/Sep/2014:16:58:36 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"

[[email protected] ~]# tail -f /var/log/httpd/access_log
192.168.2.200 - - [05/Sep/2014:16:58:42 +0800] "GET / HTTP/1.0" 200 39 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Maxthon/4.3.1.2000 Chrome/30.0.1599.101 Safari/537.36"

最后测试主keepalived挂掉，会转移到备用keepalived上，nginx负载均衡服务，keepalived会一直调用脚本检查机制，如果nginx服务挂掉或不能从新启动，都会停止掉keepalived并立即转移到备用上继续工作，后端web服务器出现问题，nginx负载均衡能自动切换后端有故障的web服务器。

参考：http://network.51cto.com/art/201007/209823.htm

http://friendlinux.blog.51cto.com/6249249/1433295

时间： 2024-10-19 14:01:44

Nginx+Keepalived实现Nginx负载均衡及高可用WEB服务器集群的相关文章

Keepalived + LVS-DR搭建高可用Web服务器集群

导航: Keepalived概述 LVS概述 编译安装keepalived 配置主调度器 双机热备 一.Keepalived概述 keepalived是一个类似于layer3, 4 & 5交换机制的软件,也就是我们平时说的第3层.第4层和第5层交换.Keepalived是自动完成,不需人工干涉,需要人工做的只是修复故障的web服务器.Layer3,4&5工作在IP/TCP协议栈的IP层,TCP层,及应用层,原理分别如下:Layer3:Keepalived使用Layer3的方式工作式时,Ke

Redis Cluster搭建高可用Redis服务器集群

原文:Redis Cluster搭建高可用Redis服务器集群 一.Redis Cluster集群简介 Redis Cluster是Redis官方提供的分布式解决方案,在3.0版本后推出的,有效地解决了Redis分布式的需求,当一个节点挂了可以快速的切换到另一个节点,当遇到单机内存.并发等瓶颈时,可以采用分布式方案要解决问题. 二.集群原理 Redis Cluster架构图 Redis Cluster集群采用了P2P的模式,完全去中心化,Redis把所有的Key分成了16384个slot,每个R

Nginx之二：负载均衡及高可用

一.Nginx负载均衡及高可用简介 负载均衡是为了解决单个节点压力过大,造成Web服务响应过慢,严重的情况下导致服务瘫痪,无法正常提供服务.负载均衡,将用户的所有HTTP请求均衡的分配到每一台机器上,充分发挥所有机器的性能,提高服务的质量和用户体验.负载均衡常用的Web服务器软件有Nginx.HAProxy.LVS.Apache. Nginx负载均衡是通过upstream模块来实现的,常见四种负载策略: 轮循(默认:将每个请求均匀分配到每台服务器 最少连接:将请求分配给连接数最少的服务器 IP

实战 LVS+Keepalived 实现负载均衡和高可用

1.软件介绍 LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统,可以实现LINUX平台下的简单负载均衡. LVS负载均衡有4种负载均衡方式 a.DR 模式 b.NAT 模式 c.Tunnel 模式 d.Full NAT 模式 LVS负载均衡8种调度算法 (rr,wrr,lc,wlc,lblc,lblcr,dh,sh) 各个模式的原理在此不再赘述,本文使用DR模式,rr调度算法. Keepalived 是运行在lvs 之上,它的主要功能是

keepalived结合nginx状态检测脚本实现对web服务器集群的高可用

实验环境 两台CentOS-7.5虚拟机web1:10.0.11.203web2:10.0.11.204VIP :10.0.11.210web类型:nginx客户端:自用笔记本(win10)nginx状态检测脚本:ck_nginx.sh 实验一.使用keepalived简单实现web集群的高可用功能 1.准备两台web服务器 1)web1网卡情况[[email protected] ~]# [[email protected] ~]# ip a 2)web2网卡情况[[email protect

Keepalived+LVS+Nginx负载均衡之高可用

Keepalived+LVS+Nginx负载均衡之高可用 上一篇写了nginx负载均衡,此篇实现高可用(HA).系统整体设计是采用Nginx做负载均衡,若出现Nginx单机故障,则导致整个系统无法正常运行.针对系统架构设计的高可用要求,我们需要解决Nginx负载均衡出现单机故障时,系统正常运行的需求.所以系统架构引入Keepalived组件,实现系统高可用. 一.Keepalived介绍 Keepalived是分布式部署系统解决系统高可用的软件,结合LVS(Linux Virtual Serve

nginx反向代理tomacat+keepalived实现动静分离、负载均衡、高可用

本文的动静分离主要是通过nginx+tomcat来实现,其中nginx处理图片.html.JS.CSS等静态文件,tomcat处理jsp.servlet等动态请求 服务器名称                                 系统版本                           预装软件                     IP地址 Nginx服务器                             CentOS 7 最小安装              Nginx

Nginx代理MogileFS并实现负载均衡和高可用

Nginx代理MogileFS并实现负载均衡和高可用 MogileFS nginx 负载均衡 大纲 实验环境 实验步骤 配置MogileFS 配置Nginx 总结 前言 上篇文章我们了解分布式系统和MogileFS的基本使用, 但是那样的架构是有问题的, 本篇文章我们来了解一下如何使用nginx-mogilefs-module-master模块来构建一个不一样的 MogileFS Cluster 实验拓扑 实验环境 主机 IP 功用 node6 172.16.1.7 Nginx,Tracker,

用haproxy结合keepalived实现基于LNMP的负载均衡和高可用

今天我们讲haproxy结合keepalived实现LNMP的负载均衡和高可用,现在的公司大部分都基于haproxy实现负载均衡.下面以一个事例去给大家详细讲解如何去实现: 一.用haproxy结合keepalived实现基于lnmp的负载均衡和高可用服务,要求: (1)实现动静分离,图片和css,js都分离到专门的静态服务器上 (2)只允许172.17网段用户访问 (3)对于动态请求,要求实现基于cookie的会话保持 整体架构如下: 1.实验前需要关闭防火墙和selinux,以及同步时间.