常用开源集群软件有：lvs，keepalived，haproxy，nginx，apache，heartbeat
常用商业集群硬件有：F5,Netscaler，Radware，A10等
一、LVS介绍
LVS是linux virtual server的简写linux虚拟服务器，是一个虚拟的服务器集群系统，可以再unix/linux平台下实现负载均衡集群功能。该项目在1998年5月由章文嵩博士组织成立。
LVS的三种工作模式：1.VS/NAT模式（Network address translation）2.VS/TUN模式（tunneling）
3.DR模式（Direct routing）
二、搭建LVS环境
1.LVS/NAT 配置
实验环境准备：
需要准备三台centos6.6系统机器，其中Director机器需要安装两块网卡；
Director的hostname命名为movies，两台real server的hostname各命名longls、bols；
每台机器上要提前安装nginx服务，为避免不必要的错误也把扩展源epel-release给安装了；
以下为各个服务器的IP：
[[email protected] ~]# ifconfig //此为Director机器，eth0为看成外网IP、eth1看成内网IP
eth0 Link encap:Ethernet HWaddr 00:0C:29:E6:9E:DF
inet addr:192.168.1.111 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee6:9edf/64 Scope:Link
eth1 Link encap:Ethernet HWaddr 00:0C:29:E6:9E:E9
inet addr:192.168.217.111 Bcast:192.168.217.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee6:9ee9/64 Scope:Link
[[email protected] ~]# ifconfig //第一台real server机器，eth1看成内网IP
eth1 Link encap:Ethernet HWaddr 00:0C:29:1B:40:8E
inet addr:192.168.217.120 Bcast:192.168.217.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe1b:408e/64 Scope:Link
[[email protected] ~]# ifconfig //第二台real server机器，eth1看成内网IP
eth1 Link encap:Ethernet HWaddr 00:0C:29:77:83:34
inet addr:192.168.217.119 Bcast:192.168.217.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe77:8334/64 Scope:Link
同时也要将Director的eth1网卡的IP设置为两台real server的网关；
[[email protected] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.217.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
0.0.0.0 192.168.217.111 0.0.0.0 UG 0 0 0 eth1
[[email protected] ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.217.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
0.0.0.0 192.168.217.111 0.0.0.0 UG 0 0 0 eth1
开始搭建环境：
[[email protected] ~]# yum install -y ipvsadm //Director安装此命令，两台real server不用安装
[[email protected] ~]# vim /usr/local/sbin/lvs_nat.sh //Direcotr 上添加以下内容
#! /bin/bash
# director 服务器上开启路由转发功能:
echo 1 > /proc/sys/net/ipv4/ip_forward
# 关闭icmp的重定向
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth1/send_redirects
# director 设置nat防火墙
iptables -t nat -F
iptables -t nat -X
iptables -t nat -A POSTROUTING -s 192.168.217.0/24 -j MASQUERADE
# director设置ipvsadm
IPVSADM=‘/sbin/ipvsadm‘
$IPVSADM -C
$IPVSADM -A -t 192.168.1.111:80 -s lc -p 300
$IPVSADM -a -t 192.168.1.111:80 -r 192.168.217.119:80 -m -w 1
$IPVSADM -a -t 192.168.1.111:80 -r 192.168.217.120:80 -m -w 1
[[email protected] ~]# /bin/bash /usr/local/sbin/lvs_nat.sh //执行脚本
[[email protected] ~]# iptables -nvL -t nat //查看规则
Chain PREROUTING (policy ACCEPT 3 packets, 534 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 1 packets, 124 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * * 192.168.217.0/24 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 1 packets, 124 bytes)
pkts bytes target prot opt in out source destinatio
[[email protected] ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.111:80 lc persistent 300
-> 192.168.217.119:80 Masq 1 0 0
-> 192.168.217.120:80 Masq 1 0 0
[[email protected] ~]# /etc/init.d/nginx start //两台real server上启动nginx服务
[[email protected] ~]# /etc/init.d/nginx start
通过浏览器可以看出nginx以正常启动，由于两个的内容一样为避免出现错误故更该了访问文件内容；
[[email protected] ~]# cat /usr/share/nginx/html/index.html
longls.avi
[[email protected] ~]# cat /usr/share/nginx/html/index.html
bols.avi
更改html文件内容后在用浏览器刷新访问会变为更改后的html文件内容；
测试nat规则：
更改轮询规则为lc，权重为2，进行测试：
[[email protected] ~]# /bin/bash /usr/local/sbin/lvs_nat.sh //从新执行脚本
用另一台linux机器curl测试，出现1次longls，1次bols，来回切换说明OK
[[email protected] ~]# curl 192.168.1.111
bols.avi
[[email protected] ~]# curl 192.168.1.111
longls.avi
[[email protected] ~]# curl 192.168.1.111
bols.avi
在Director机器上ipvsadm -ln可以查看，权重比，保持的链接比大概一样；
[[email protected] ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.111:80 lc
-> 192.168.217.119:80 Masq 1 0 7
-> 192.168.217.120:80 Masq 1 0 7
2. LVS/DR 配置
实验环境准备：
需要准备三台机器，每台机器只需要配置1个ip，vip是用脚本执行后会出现的，不用手动设置；
Director的hostname命名为movies，两台real server的hostname各命名longls、bols；
每台机器上要提前安装nginx服务，为避免不必要的错误也把扩展源epel-release给安装了；
[[email protected] ~]# vim /usr/local/sbin/lvs_dr.sh // Director添加以下内容
#! /bin/bash
echo 1 > /proc/sys/net/ipv4/ip_forward
ipv=/sbin/ipvsadm
vip=192.168.1.110
rs1=192.168.1.119
rs2=192.168.1.120
ifconfig eth0:0 down
ifconfig eth0:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip dev eth0:0
$ipv -C
$ipv -A -t $vip:80 -s wrr
$ipv -a -t $vip:80 -r $rs1:80 -g -w 3
$ipv -a -t $vip:80 -r $rs2:80 -g -w 1
[[email protected] ~]# vim /usr/local/sbin/lvs_dr_rs.sh //两台real server都要增加以下脚本
#! /bin/bash
vip=192.168.1.110
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
[[email protected] ~]# /bin/sh /usr/local/sbin/lvs_dr.sh //执行shell脚本
[[email protected] ~]# sh /usr/local/sbin/lvs_dr_rs.sh
[[email protected] ~]# sh /usr/local/sbin/lvs_dr_rs.sh
[[email protected] ~]# ifconfig //通过查看IP发现虚拟IP，dr显示eth0:0，rs1、rs2显示lo:0
eth0:0 Link encap:Ethernet HWaddr 00:0C:29:E6:9E:DF
inet addr:192.168.1.110 Bcast:192.168.1.110 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Base address:0x2000
[[email protected] ~]# ifconfig
lo:0 Link encap:Local Loopback
inet addr:192.168.1.110 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:65536 Metric:1
[[email protected] ~]# ifconfig
lo:0 Link encap:Local Loopback
inet addr:192.168.1.110 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:65536 Metric:1
[[email protected] ~]# ipvsadm -ln //查看规则
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.110:80 wrr
-> 192.168.1.119:80 Route 3 0 0
-> 192.168.1.120:80 Route 1 0 0
在另开一台linux机器进行测，发现出现一次longls后出现三次bols，从而说明wrr轮询规则OK
[[email protected] ~]# curl 192.168.1.110
longls.avi
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
longls.avi
[[email protected] ~]# ipvsadm -ln //查看规则发现比例大概为3：1
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.110:80 wrr
-> 192.168.1.119:80 Route 3 0 8
-> 192.168.1.120:80 Route 1 0 2
DR配置测试：
若一台rs如果挂了之后，还是会轮询访问，若访问到宕掉的机器则会出现错误；
[[email protected] ~]# /etc/init.d/nginx stop //模拟该机器宕机
停止 nginx： [确定]
根据轮询访问算法3：1比例会发现访问未宕掉机时正常访问，而访问宕掉的机器会发现报错；
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
bols.avi
[[email protected] ~]# curl 192.168.1.110
curl: (7) couldn‘t connect to host
通过上面的例子我们会发现在机器宕掉后我们仍能访问，若想将其剔除还要结合keeplived；