一、环境介绍
二、安装后端服务器
三、安装前端Apache服务
四、配置Apache使用mod_jk模块实现代理及负载均衡
五、配置Apache基于mod_proxy模块实现代理及负载均衡
六、论坛安装
七、安装并配置Keepalived实现前端高可用
一、环境介绍
系统版本：CentOS 6.4_x86_64
Mysql版本：mysql-5.1.66-2.el6_3.x86_64
Apache版本：httpd-2.2.25 点此下载
Keepalived版本：keepalived-1.2.7 点此下载
Tomcat版本：apache-tomcat-7.0.33 点此下载
Jdk版本： jdk-7u40-linux-x64 点此下载
Tomcat-commectors版本：tomcat-connectors-1.2.37 点此下载
论坛版本：JspRun!_6.0.0_GBK 点此下载
环境介绍：
当用户通过 "www.allen.com" 域名访问时，首先是由前端两台Apache代理服务器响应并转发到后端Tomcat服务器上，而在Apache上做的是反向代理负载均衡到后端Tomcat服务器上，前端服务器使用Keepalived做的高可用集群，【Apache1默认为Master，Apache2为Backup】虚拟IP地址为"172.16.14.10"模拟为公网IP；当然这里数据库只有一台主机会成为点故障，这里不要介绍Mysql的高可用，如果有兴趣可以看前面写的博客
提示：这里使用Keepalived做高可用时，前面所有的服务都已完成后，最后实现的对前端服务器做高可用
二、安装后端Tomcat与Mysql服务器
1、在Tomcat1与Tomcat2服务器上安装Tomcat，安装方法相同，这里只介绍一次
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
安装JDK
# rpm -ivh jdk-7u40-linux-x64.rpm
# vim /etc/profile.d/java.sh
export JAVA_HOME=/usr/java/latest
export PATH=$JAVA_HOME/bin:$PATH
# . /etc/profile.d/java.sh
# java -version
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
------------------------------------------------------------------------
安装Tomcat
# tar xf apache-tomcat-7.0.33.tar.gz -C /usr/local/
# cd /usr/local/
# ln -s apache-tomcat-7.0.33 tomcat
# vim /etc/profile.d/tomcat.sh
export CATALINA_HOME=/usr/local/tomcat
export PATH=$CATALINA_HOME/bin:$PATH
# . /etc/profile.d/tomcat.sh
# catalina.sh version
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/java/latest
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Server version: Apache Tomcat/7.0.33
Server built: Nov 18 2012 04:15:21
Server number: 7.0.33.0
OS Name: Linux
OS Version: 2.6.32-358.el6.x86_64
Architecture: amd64
JVM Version: 1.7.0_40-b43
JVM Vendor: Oracle Corporation
|
2、启动Tomcat服务并访问测试，默认访问端口为"8080";如下
1
2
3
4
5
6
7
8
9
10
11
|
# catalina.sh start
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/java/latest
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
# jps
2283 Jps
2272 Bootstrap
# ss -tanlp | grep 8080
LISTEN 0 100 :::8080 :::* users:(("java",2272,40))
|
3、开启Tomcat管理及状态页面
1
2
3
4
5
6
|
# vim /usr/local/tomcat/conf/tomcat-users.xml
<role rolename="manager-gui"/>
<role rolename="admin-gui"/>
<user username="allen" password="admin" roles="manager-gui,admin-gui"/>
# catalina.sh stop
# catalina.sh start
|
4、访问Tomcat后台管理
5、安装Mysql服务器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
######安装Mysql并创建数据库，授权用户访问新创建的数据库有所有权，方便后面使用
[[email protected] ~]# yum -y install mysql-server
[[email protected] ~]# service mysqld start
[[email protected] ~]# mysql
mysql> create database bbs;
mysql> grant all on bbs.* to ‘bbsuser‘@‘172.16.%.%‘ identified by ‘bbspass‘;
mysql> flush privileges;
----------------------------------------------------------------------
远程连接数据库测试授权用户是否能连接成功
# mysql -ubbsuser -pbbspass -h 172.16.14.5 -e ‘show databases‘;
+--------------------+
| Database |
+--------------------+
| information_schema |
| bbs |
| test |
+--------------------+
|
三、安装前端Apache服务
1、在Apache1与Apache2服务器上安装Httpd软件，安装方法相同，这里只介绍一次
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
# tar xf httpd-2.2.25.tar.bz2
# cd httpd-2.2.25
# yum -y install gcc gcc-c++ openssl-devel pcre-devel
# ./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd --enable-modules=most --enable-so --enable-ssl --enable-cgi --enable-rewrite --with-zlib --with-pcre --enable-mpms-shared=all --with-mpm=prefork --enable-proxy --enable-proxy-http --enable-proxy-ajp --enable-proxy-balancer --enable-cache --enable-cache-disk --enable-mem-cache --enable-file-cache --enable-proxy-connect
# make && make install
# cp build/rpm/httpd.init /etc/init.d/httpd
------修改如下内容
# vim /etc/init.d/httpd
httpd=${HTTPD-/usr/local/apache/bin/httpd}
pidfile=${PIDFILE-/usr/local/apache/logs/${prog}.pid}
lockfile=${LOCKFILE-/var/lock/subsys/${prog}}
RETVAL=0
# check for 1.3 configuration
check13 () {
CONFFILE=/etc/httpd/httpd.conf
----------------------------------------------------------------------
修改环境变量并把Httpd加入到系统服务
# echo "PATH=/usr/local/apache/bin:$PATH" >> /etc/profile.d/http.sh
# . /etc/profile.d/http.sh
# ln -s /usr/local/apache/include/ /usr/include/httpd
# chkconfig --add httpd
----------------------------------------------------------------------
创建测试页
# echo "<h1>APACHE</h1>" > /usr/local/apache/htdocs/index.html
|
2、在Apache1与Apache2服务器上安装"tomcat-connectors"
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# tar xf tomcat-connectors-1.2.37-src.tar.gz
# cd tomcat-connectors-1.2.37-src/native/
# ./configure --with-apxs=/usr/local/apache/bin/apxs
# make && make install
------------------------------------------------------------------------
安装完成后会生成如下文件
# ls /usr/local/apache/modules/
mod_jk.so
------------------------------------------------------------------------
启动Httpd服务
#service httpd start
# ss -tanl |grep 80
LISTEN 0 128 :::80 :::*
|
3、访问测试Apache是否正常工作
四、配置Apache使用mod_jk模块实现代理及负载均衡
1、修改Apache主配置文件，包含一个文件并创建该文件；实现代理功能
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
将下面指令添加到文件末尾即可
# vim /etc/httpd/httpd.conf
Include /etc/httpd/extra/mod_jk.conf
------------------------------------------------------------------------
创建文件添加如下内容
# vim /etc/httpd/extra/mod_jk.conf
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/extra/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* TomcatA
JkMount /status/ stat1
------------------------------------------------------------------------
# vim /etc/httpd/extra/workers.properties
worker.list=TomcatA,stat1
worker.TomcatA.port=8009
worker.TomcatA.host=172.16.14.3
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.stat1.type = status
----------------------------------------------------------------------
# service httpd restart
|
2、修改后端Tomcat1服务器，添加一个虚拟主机并提供测试页面
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[[email protected] ~]# cd /usr/local/tomcat/conf/
[[email protected] conf]# cp server.xml server.xml.bak
[[email protected] conf]# vim server.xml
######修改如下内容
<Engine name="Catalina" defaultHost="www.allen.com" jvmRoute="TomcatA">
######在"Engine"中添加如下内容
<Host name="www.allen.com" appBase="/web/allen"
unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="allen_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Context path="" docBase="/web/allen" />
</Host>
----------------------------------------------------------------------
######创建网站存放目录并创建测试页
[[email protected] ~]# mkdir -p /web/allen
[[email protected] ~]# vim /web/allen/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatA</title></head>
<body>
<h1><font color="red">TomcatA </font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("abc","abc"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
----------------------------------------------------------------------
######重启Tomcat服务并查看是否启动成功
[[email protected] ~]# catalina.sh stop
[[email protected] ~]# catalina.sh configtest
[[email protected] ~]# catalina.sh start
[[email protected] ~]# ss -tanlp | grep java
LISTEN 0 100 :::8080 :::* users:(("java",6323,40))
LISTEN 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",6323,52))
LISTEN 0 100 :::8009 :::* users:(("java",6323,41))
|
3、访问Apache1主机，验证是否代理成功
4、修改Apache配置文件(mod_jk.conf、workers.properties)；实现负载均衡
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
# vim /etc/httpd/extra/mod_jk.conf
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/extra/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* lbcluster
JkMount /status/ stat1
------------------------------------------------------------------------
# vim /etc/httpd/extra/workers.properties
worker.list=lbcluster,stat1
worker.TomcatA.port=8009
worker.TomcatA.host=172.16.14.3
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.port = 8009
worker.TomcatB.host=172.16.14.4
worker.TomcatB.type = ajp13
worker.TomcatB.lbfactor = 1
worker.lbcluster.type = lb
worker.lbcluster.sticky_session = 0 #取值为{1|0}1将用户session与后端服务器绑定，0为不绑定，如果支持session复制或session共享可以设置为0
worker.lbcluster.balance_workers = TomcatA, TomcatB
worker.stat1.type = status
----------------------------------------------------------------------
# service httpd reload
|
5、修改后端Tomcat2服务器，添加一个虚拟主机并提供测试页面
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[[email protected] ~]# cd /usr/local/tomcat/conf/
[[email protected] conf]# cp server.xml server.xml.bak
[[email protected] conf]# vim server.xml
######修改如下内容
<Engine name="Catalina" defaultHost="www.allen.com" jvmRoute="TomcatB">
######在"Engine"中添加如下内容
<Host name="www.allen.com" appBase="/web/allen"
unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="allen_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Context path="" docBase="/web/allen" />
</Host>
----------------------------------------------------------------------
######创建网站存放目录并创建测试页
[[email protected] ~]# mkdir -p /web/allen
[[email protected] ~]# vim /web/allen/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatB</title></head>
<body>
<h1><font color="blue">TomcatB </font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("abc","abc"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
----------------------------------------------------------------------
######重启Tomcat服务并查看是否启动成功
[[email protected] ~]# catalina.sh stop
[[email protected] ~]# catalina.sh configtest
[[email protected] ~]# catalina.sh start
[[email protected] ~]# ss -tanlp | grep java
LISTEN 0 100 :::8080 :::* users:(("java",6323,40))
LISTEN 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",6323,52))
LISTEN 0 100 :::8009 :::* users:(("java",6323,41))
|
6、访问Apache服务器，测试是否实现负载均衡
由上可见，基于Apache的mod_jk模块已成功实现负载均衡功能
五、配置Apache基于mod_proxy模块实现代理及负载均衡
1、修改Apache的主配置文件，包含一个文件并创建该文件；实现代理功能
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
# vim /etc/httpd/httpd.conf
#Include /etc/httpd/extra/mod_jk.conf #注释此行
Include /etc/httpd/extra/mod_proxy.conf
------------------------------------------------------------------------
# vim /etc/httpd/extra/mod_proxy.conf
ProxyVia On
ProxyRequests Off
ProxyPreserveHost Off
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
ProxyPass / ajp://172.16.14.3:8009/
ProxyPassReverse / ajp://172.16.14.3:8009/
<Location / >
Order allow,deny
Allow from all
</Location>
----------------------------------------------------------------------
# service httpd reload
======================================================================
注释：
ProxyPass / ajp://172.16.14.3:8009/ #使用的是ajp协议
ProxyPassReverse / ajp://172.16.14.3:8009/
可以更改为使用http协议；如下
ProxyPass / http://172.16.14.3:8080/
ProxyPassReverse / http://172.16.14.3:8080/
######这里只演示使用ajp协议,如果是Apache与Tomcat结合建议使用ajp协议
|
2、访问Apache1服务器，测试是否代理成功
3、修改Apache配置文件(mod_proxy.conf)；实现负载均衡
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
# vim /etc/httpd/extra/mod_proxy.conf
ProxyVia Off
ProxyRequests Off
ProxyPreserveHost Off
<Proxy balancer://allen>
BalancerMember ajp://172.16.14.3:8009 loadfactor=1
BalancerMember ajp://172.16.14.4:8009 loadfactor=1
ProxySet lbmethod=bytraffic
</Proxy>
<Location /allen> #设置状态及管理页面
SetHandler balancer-manager
Proxypass !
Order allow,deny
Allow from all
</Location>
<Proxy *>
Order allow,deny
Allow from all
</Proxy>
ProxyPass / balancer://allen/ stickysession=JSESSIONID
ProxyPassReverse / balancer://allen/
<Location / >
Order allow,deny
Allow from all
</Location>
----------------------------------------------------------------------
注释：
BalancerMember ajp://172.16.14.3:8009 loadfactor=1 #使用ajp协议
BalancerMember ajp://172.16.14.4:8009 loadfactor=1
可更改为使用http协议
BalancerMember http://172.16.14.3:8080 loadfactor=1
BalancerMember http://172.16.14.4:8080 loadfactor=1
######这里只介绍使用ajp协议，如果有兴趣可以更改一下；loadfactor：权重
----------------------------------------------------------------------
ProxySet lbmethod=bytraffic #设置调度算法
byrequests：即基于权重将统计请求个数进行调度(默认)
bytraffic：则执行基于权重的流量计数调度
bybusyness：通过考量每个后端服务器的当前负载进行调度
----------------------------------------------------------------------
# service httpd reload
|
4、访问Apache服务器，测试基于mod_proxy模块是否实现负载均衡
5、访问状态页面
六、论坛安装
1、在Tomcat1服务器上安装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
# unzip JspRun\!_6.0.0_GBK.zip
# mv /web/allen/index.jsp /web/allen/test.jsp
# cp -r upload/* /web/allen/
------------------------------------------------------------------------
修改论坛数据库连接文件
# vim /web/allen/config.properties
dbhost = 172.16.14.5
dbport=3306
dbuser = bbsuser
dbpw = bbspass
dbname = bbs
------------------------------------------------------------------------
# catalina.sh stop
# catalina.sh start
|
2、安装论坛程序
3、点击下一步-->我同意-->下一步，进入数据库配置页面
4、点下一步-->(填写管理员密码)下一步-->进入创建数据表阶段-->然后点
-->进入首页
5、将论坛程序拷贝到Tomcat2服务器上一份并访问测试
1
2
3
4
5
|
# mv /web/allen/index.jsp /web/allen/test.jsp #把原来的测试页重命名
------------------------------------------------------------------------
# scp -r 172.16.14.3:/web/allen/* /web/allen/
# catalina.sh stop
# catalina.sh start
|
6、访问Apache服务器，测试是否能正常访问论坛程序，这里就不在做访问测试
七、安装并配置Keepalived
1、将Apache1服务器的Httpd配置文件复制到Apache2服务器
1
2
3
4
|
[[email protected] ~]# cd /etc/httpd/
[[email protected] httpd]# scp httpd.conf 172.16.14.2:/etc/httpd/
[[email protected] httpd]# cd extra/
[[email protected] extra]# scp mod_jk.conf mod_proxy.conf workers.properties 172.16.14.2:/etc/httpd/extra/
|
2、在Apache1与Apache2服务器上安装Keepalived；这里使用rpm包安装，光盘映像中有
1
2
3
|
[[email protected] ~]# yum -y install keepalived
--------------------------------------------
[[email protected] ~]# yum -y install keepalived
|
3、配置Apache1服务器上的Keepalived
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 172.16.0.0
smtp_connect_timeout 30
router_id LVS_ALLEN
}
vrrp_script chk_httpd {
script "killall -0 httpd"
interval 1
weight -2
}
vrrp_instance httpd_1 {
state MASTER
interface eth0
virtual_router_id 58
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1058
}
virtual_ipaddress {
172.16.14.10
}
track_script {
chk_httpd
}
}
[[email protected] ~]# service keepalived start
[[email protected] ~]# chkconfig keepalived on
[[email protected] ~]# chkconfig --list keepalived
keepalived 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[[email protected] ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2c:1a:24 brd ff:ff:ff:ff:ff:ff
inet 172.16.14.1/16 brd 172.16.255.255 scope global eth0
inet 172.16.14.10/32 scope global eth0
inet6 fe80::20c:29ff:fe2c:1a24/64 scope link
valid_lft forever preferred_lft forever
|
4、配置Apache2服务器上的Keepalived
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 172.16.0.0
smtp_connect_timeout 30
router_id LVS_ALLEN
}
vrrp_script chk_httpd {
script "killall -0 httpd"
interval 1
weight -2
}
vrrp_instance httpd_1 {
state BACKUP
interface eth0
virtual_router_id 58
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1058
}
virtual_ipaddress {
172.16.14.10
}
track_script {
chk_httpd
}
}
[[email protected] ~]# service keepalived start
[[email protected] ~]# chkconfig --list keepalived
keepalived 0:off 1:off 2:on 3:on 4:on 5:on 6:off
|
5、使用"www.allen.com" 域名访问，验证是否能访问到论坛；由于没有DNS服务器解析，修改了"hosts"文件；如：
1
2
3
|
######添加如下内容解析
C:\Windows\System32\drivers\etc\hosts
172.16.14.10 www.allen.com
|
6、模拟前端一台服务器出现故障；查看虚拟IP是否转移
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
######停止Apache1服务器上的Httpd服务来模拟故障；并查看IP
[[email protected] ~]# service httpd stop
[[email protected] ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2c:1a:24 brd ff:ff:ff:ff:ff:ff
inet 172.16.14.1/16 brd 172.16.255.255 scope global eth0
inet6 fe80::20c:29ff:fe2c:1a24/64 scope link
valid_lft forever preferred_lft forever
========================================================================
######查看Apache2服务器上的IP
[[email protected] ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:ec:f6:3f brd ff:ff:ff:ff:ff:ff
inet 172.16.14.2/16 brd 172.16.255.255 scope global eth0
inet 172.16.14.10/32 scope global eth0
inet6 fe80::20c:29ff:feec:f63f/64 scope link
valid_lft forever preferred_lft forever
|
7、从上一步可以看出，虚拟IP地址已成功转移，说明还能正常提供服务，可以再次访问测试；如果服务器修复好重新上线，虚拟IP还会转移回去；这里就不再测试
到此，基于Apache做反向代理实现了Tomcat的负载均衡；而Keepalived实现了前端服务器的高可用；关于session保存的问题，方案有很多，比如可以保存到"memcached"、"redis"等，使用哪种方案具体还要根据自己的需求而定...
时间： 2024-10-12 16:00:27
-->进入首页
