实验准备：
1）平台使用RHEL6.5的64位系统平台
2）所需软件为ipvsadm、keepalived、httpd
3）节点如下：
direct1 192.168.3.166
direct2 192.168.3.168
rip1 192.168.3.33
rip2 192.168.3.34
VIP 192.168.3.35
1、安装lvs-1.26
1) yum -y install gcc gcc-c++
2) yum -y install libnl* libpopt*
3) yum -y install popt-devel
4) rpm -ivh popt-static-1.13-7.el6.x86_64.rpm
5) tar -xf ipvsadm-1.26.tar.gz
6) cd ipvsadm-1.26
7) make && make install
2、安装keepalived
1）yum -y install openssl openssl-devel
2）yum install -y libnfnetlink-devel
3) tar -xf keepalived-1.3.5.tar.gz
3）cd keepalived-1.3.5
4) ./configure --prefix=/usr/local/keepalived
5) make && make install
6)配置keepalived文件的路径
cp /opt/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
3、在其中一个direct上配置/etc/keepalived/keepalived.conf，当前是在LVS的MASTER端
global_defs {
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS1
}
vrrp_sync_group test {
group {
test_1
}
}
vrrp_instance test_1 {
state MASTER
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.3.35
}
}
virtual_server 192.168.3.35 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
real_server 192.168.3.33 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.3.34 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
4、编辑好配置文件后，可以把这个文件复制到LVS的BACKUP端，即另一个direct节点，然后再BACKUP端修改state 为BACKUP,priority为99即可，如下：
global_defs {
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS2
}
vrrp_sync_group test {
group {
test_1
}
}
vrrp_instance test_1 {
state BACKUP
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.3.35
}
}
virtual_server 192.168.3.35 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
real_server 192.168.3.33 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.3.34 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
5、在两个direct上启动keepalived服务
/etc/init.d/keepalived start
6、当keepalived启动完成之后，可以检查虚拟IP地址是否已经生产，且虚拟IP地址位于MASTER上。
由于在配置文件中将direct1设置为MASTER，因此，只需要在direct上执行如下命令，即可检查配置是否成功，如下：
# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 7a:d2:18:41:d3:c8 brd ff:ff:ff:ff:ff:ff
inet 192.168.3.166/24 brd 192.168.3.255 scope global eth0
inet 192.168.3.35/32 scope global eth0
inet6 fe80::78d2:18ff:fe41:d3c8/64 scope link
valid_lft forever preferred_lft forever
7、测试VIP的漂移
1)当MASTER发生故障时，测试VIP是否会漂移到BACKUP上去。
可以在MASTER上执行如下命令来模拟BACKUP发生故障：
/etc/init.d/keepalived stop
2)之后去BACKUP端执行以下命令：
# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 46:6c:ca:46:84:18 brd ff:ff:ff:ff:ff:ff
inet 192.168.3.168/24 brd 192.168.3.255 scope global eth0
inet 192.168.3.35/32 scope global eth0
inet6 fe80::446c:caff:fe46:8418/64 scope link
valid_lft forever preferred_lft forever
3）以上即说明，当MASTER发生故障时，VIP就会漂移到BACKUP上去
8、keepalived的高可用已经实现，之后就是要配置与LVS有关的内容，需在两台真实服务器上操作
在两台真实服务器上安装apache，并且启动httpd服务
在这最好是编辑一下两台真实服务器的httpd的默认根路径下的文件，使两个文件的内容不同，以便于测试。
1）以上工作做好，开始在两台真实服务器上配置VIP，如下
vim /etc/init.d/realserver.sh
#!/bin/bash
# description: Config realserver lo and apply noarp
. /etc/rc.d/init.d/functions
VIP=192.168.3.35
host=`/bin/hostname`
case "$1" in
start)
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
# sysctl -p >/dddev/null 2>&1
# echo "RealServer Start OK"
#/sbin/ifconfig lo:0$VIP broadcast $VIP netmask 255.255.255.255up
#/sbin/route add -host $VIP dev lo:0
/sbin/ifconfig lo:0 192.168.3.35 broadcast 192.168.3.35 netmask 255.255.255.255 up
/sbin/route add -host 192.168.3.35 dev lo:0
;;
stop)
/sbin/ifconfig lo:0 down
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
status)
# Status of LVS-DR real server.
islothere=`/sbin/ifconfig lo:0 | grep 192.168.3.35`
isrothere=`netstat -rn | grep "lo:0" | grep 192.168.3.35`
if [ ! "$islothere" -o ! "isrothere" ];then
# Either the route or the lo:0 device
# not found.
echo "LVS-DR real server Stopped."
else
echo "LVS-DR real server Running."
fi
;;
*)
# Invalid entry.
echo "$0: Usage: $0 {start|status|stop}"
exit 1
;;
esac
2）配置文件写好之后，加权限再运行
chmod +x /etc/init.d/realserver.sh
/etc/init.d/realserver.sh start
3）然后再运行ifconfig，可查看到VIP已配置好
#ifconfig
eth0 Link encap:Ethernet HWaddr 42:58:57:BE:25:BE
inet addr:192.168.3.33 Bcast:192.168.3.255 Mask:255.255.255.0
inet6 addr: fe80::4058:57ff:febe:25be/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2515650 errors:0 dropped:0 overruns:0 frame:0
TX packets:16369 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:164807767 (157.1 MiB) TX bytes:1947231 (1.8 MiB)
Interrupt:165
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4920 errors:0 dropped:0 overruns:0 frame:0
TX packets:4920 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:339452 (331.4 KiB) TX bytes:339452 (331.4 KiB)
lo:0 Link encap:Local Loopback
inet addr:192.168.3.35 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
4）两台真实服务器上的VIP的配置都是一致的
5）在MASTER上查看IPVS规则是否成功生成
#ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.3.35:80 rr
-> 192.168.3.33:80 Route 1 0 0
-> 192.168.3.34:80 Route 1 0 0
9、测试
1）在浏览器上输入http://192.168.3.35，然后观察页面内容，多刷新几次，就会发现页面内容会发生变化，即成功调用两台真实服务器的htppd默认网页的内容。
2）当后端的真实服务器发生故障或运行在真实服务器上的服务发生故障时，则与这个真实服务器相关的IPVS规则会被删除。
如在rip1上执行如下命令，来模拟httpd服务发生故障
/usr/local/apache/bin/apachectl stop
然后再在作为MASTER上的服务器上查看IPVS规则，使用如下命令即可查看IPVS规则，这时我的direct1是作为MASTER。
#ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.3.35:80 rr
-> 192.168.3.34:80 Route 1 0 0
这时会发现与rip1相关的IPVS规则被删除，测试成功。
补充：
本人在做好LVS+Keepalived之后，真实服务器也做好的相关的配置，但是在MASTER运行 ipvsadm -L -n 时，就只有一个真实服务器被添加到列表中，另一台没有添加成功！
经过网上搜索，排查错误，最后找到原因，即在两个节点上配置keepalived.conf文件时，有一个地方不规范，即添加realserver时，语法出现错误
real_server 192.168.3.33 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.3.34 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
这里的TCP_CHECK后面一定要有空格，是TCP_CHECK { ，TCP_CHECK与{ 之间存在空格，而不是TCP_CHECK{ ，中间没有空格，就会出现以上的错误。