上篇文章，已经明确了tomcat集群的必要性，主要从高可用和高并发量考虑。一般使用tomcat集群搭建，都要使用前置设一个nginx或apache作个反向代理，将请求转发给后端tomcat。使用tomcat集群，必然会引来session数据共享问题。如何解决这个问题？可借助memcached实现。
主要内容
- 测试单台tomcat的吞吐率
- 搭建apache+mod_jk+2个tomcat集群环境
- 搭建apache+mod_jk+2个tomcat+memcached集群环境
- 对比2个环境的吞吐率情况
环境准备
确认apache服务位置及版本号
[[email protected] httpd]# httpd -v Server version: Apache/2.2.15 (Unix) Server built: Feb 22 2013 11:17:44 [[email protected] httpd]# which httpd /usr/sbin/httpd #apache主目录:/etc/httpd
确认JDK 及tomcat服务
[[email protected] tomcat-sets]$ ls /u01/tomcat-sets apache-tomcat-node1 [[email protected] apache-tomcat-node1]$ cat /proc/meminfo MemTotal: 1937148 kB ... [[email protected] /]$ /u01/tomcat-sets/apache-tomcat-node1/bin/version.sh Using CATALINA_BASE: /u01/tomcat-sets/apache-tomcat-node1 Using CATALINA_HOME: /u01/tomcat-sets/apache-tomcat-node1 Using CATALINA_TMPDIR: /u01/tomcat-sets/apache-tomcat-node1/temp Using JRE_HOME: /usr/local/java/jdk1.6.0_45 ... Server version: Apache Tomcat/7.0.33 Server built: Nov 18 2012 04:15:21 Server number: 7.0.33.0 OS Name: Linux OS Version: 2.6.32-358.el6.i686 Architecture: i386 JVM Version: 1.6.0_45-b06 JVM Vendor: Sun Microsystems Inc.
1.测试单台tomcat的吞吐率
1.1准备一个静态测试资源（staticProtocal.html，大小50K）
资源可以通过http://192.168.163.146:8080/examples/staticProtocal.html访问
[[email protected] examples]$ ll /u01/tomcat-sets/apache-tomcat-node1/webapps/examples total 72 -rw-r--r--. 1 hadoop hadoop 1179 Nov 19 2012 index.html drwxr-xr-x. 22 hadoop hadoop 4096 Aug 10 18:48 jsp drwxr-xr-x. 3 hadoop hadoop 4096 Aug 10 18:48 servlets -rw-rw-r--. 1 hadoop hadoop 50663 Aug 12 21:09 staticProtocal.html drwxr-xr-x. 7 hadoop hadoop 4096 Aug 10 18:48 WEB-INF drwxr-xr-x. 2 hadoop hadoop 4096 Aug 10 18:48 websocket
1.2启动apache-tomcat-node1，端口为默认的8080
1.3 在另外一台主机使用apache ab命令测试
C:\Apache24\bin>ab -n 10000 -c 4 http://192.168.163.146:8080/examples/staticProtocal.html This is ApacheBench, Version 2.3 <$Revision: 1604373 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.163.146 (be patient) Completed 1000 requests ... Completed 10000 requests Finished 10000 requests Server Software: Apache-Coyote/1.1 Server Hostname: 192.168.163.146 Server Port: 8080 Document Path: /examples/staticProtocal.html Document Length: 50663 bytes Concurrency Level: 4 Time taken for tests: 12.065 seconds Complete requests: 10000 Failed requests: 0 Total transferred: 509120000 bytes HTML transferred: 506630000 bytes Requests per second: 828.81 [#/sec] (mean) Time per request: 4.826 [ms] (mean) Time per request: 1.207 [ms] (mean, across all concurrent requests) Transfer rate: 41207.63 [Kbytes/sec] received ...
经过测试,单台tomcat的结果是：QPS:828.81
都有这样一个共识，apache处理静态文件的能力，比tomcat强大。是骡子是马，比比看。
1.4 简单编辑apache配置文件httpd.conf。(最好修改之前备份下)
$ cat /etc/httpd/conf/httpd.conf ... ServerName localhost:80 DocumentRoot "/var/www/html" ...
1.5 将刚才的staticProtocal.html文件，复制到DocumentRoot“/var/www/html”目录下。
$ ls /var/www/html/ staticProtocal.html
1.6 启动httpd
$ service httpd start Starting httpd: [ OK ]
这时候可以通过链接访问资源：http://192.168.163.146/staticProtocal.html
1.7 测试apache吞吐量
C:\Apache24\bin>ab -n 10000 -c 4 http://192.168.163.146/staticProtocal.html This is ApacheBench, Version 2.3 <$Revision: 1604373 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.163.146 (be patient) Completed 1000 requests ... Finished 10000 requests Server Software: Apache/2.2.15 Server Hostname: 192.168.163.146 Server Port: 80 Document Path: /staticProtocal.html Document Length: 50663 bytes Concurrency Level: 4 Time taken for tests: 11.902 seconds Complete requests: 10000 Failed requests: 0 Total transferred: 509350000 bytes HTML transferred: 506630000 bytes Requests per second: 840.17 [#/sec] (mean) Time per request: 4.761 [ms] (mean) Time per request: 1.190 [ms] (mean, across all concurrent requests) Transfer rate: 41790.94 [Kbytes/sec] received
经过测试,单台apache的结果是：QPS:840.17
显而易见，apache处理静态文件能力略高于tomcat.
接下来，那就利用apache搭建一个双节点tomcat集群，顺带验证下QPS能达到多少。这是下一小节内容。
2.搭建apache+mod_jk+2个tomcat集群环境
2.1下载mod_jk并安装
要安装httpd-devel,否则会configure: error: Cannot find the WebServer
# yum install httpd-devel ## 安装完httpd-devel之后，新添加目录/usr/sbin/apxs ##下载 # wget ##解压 # tar zxvf tomcat-connectors-1.2.41-src.tar.gz # cd tomcat-connectors-1.2.41-src/native # ./buildconf.sh --with-apxs=/usr/sbin/apxs # make
到这里mod_jk.so已经生成了，最后将mod_jk.so放到apache的modules 目录下
cp /u01/software/tomcat-connectors-1.2.41-src/native/apache-2.0/mod_jk.so /etc/httpd/modules
2.2配置mod_jk
请参考：http://tomcat.apache.org/connectors-doc/webserver_howto/apache.html
mod_jk：支持平台Linux, FreeBSD, AIX, HP-UX, MacOS X, Solaris，Apache 1.3 and/or 2.x
mod_jk: 支持Tomcat 3.2 to Tomcat 8.
redirector 使用协议：ajp12 and ajp13
ajp12 协议针对Tomcat 3.2.x and 3.3.x.
ajp13协议支持tomcat 4+
mod_jk:2个组成部分。
The Apache HTTP Server module。由平台决定，一般是mod_jk.so, mod_jk.nlm or MOD_JK.SRVPGM
workers.properties ：描述Tomcat 相关的host(s) and port(s)文件。
2.3启用jk_module模块
将下面内容片段，加入/etc/httpd/httpd.conf
# Load mod_jk module LoadModule jk_module modules/mod_jk.so JkWorkersFile conf/workers.properties JkLogFile logs/mod_jk.log JkLogLevel info include "conf/mod_jk.conf"
2.4 配置后端tomcat 主机及端口映射,编辑或新建文件，/etc/httpd/workers.properties
配置说明参照：https://tomcat.apache.org/connectors-doc/reference/workers.html
worker.list=tomcat1 worker.tomcat1.port=8009 worker.tomcat1.host=127.0.0.1 worker.tomcat1.type=ajp13 #worker.tomcat1.lbfactor=1
现在达到的效果是apache反代1个tomcat。所以不需要配置额外的负载均衡等信息。
2.5 编辑conf/mod_jk.conf，增加资源映射
#LoadModule jk_module modules/mod_jk.so JkMount /*.jsp tomcat1 JkMount /examples/* tomcat1 #JkWorkersFile conf/workers.properties
注释的两行，由于已经在httpd.conf中已经配置，所以不需要重复配置。
2.6 启动tomcat,及apache
笔者，在这一步经过测试，不能正常达到代理效果。
通过查看日志信息(/var/log/httpd/mod_jk.log)
[Fri Aug 12 22:49:23.051 2016] [9739:3078039376] [error] init_jk::mod_jk.c (3574): Initializing shm:/etc/httpd/logs/jk-runtime-status.9739 errno=13. Load balancing workers will not function properly.
通过参照网上解决方案：临时将selinux关闭
确认selinux状态 /usr/sbin/sestatus -v 临时关闭 setenforce 0 永久关闭 修改/etc/selinux/config 设置为 SELINUX=disabled
访问测试，没有问题
http://192.168.163.146/examples/servlets/
http://192.168.163.146:8080/examples/servlets/
到了这里，完成了apache+单台tomcat的配置。
C:\Apache24\bin>ab -n 10000 -c 4 http://192.168.163.146/examples/staticProtocal.html This is ApacheBench, Version 2.3 <$Revision: 1604373 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.163.146 (be patient) Completed 1000 requests ... Completed 10000 requests Finished 10000 requests Server Software: Apache/2.2.15 Server Hostname: 192.168.163.146 Server Port: 80 Document Path: /examples/staticProtocal.html Document Length: 50663 bytes Concurrency Level: 4 Time taken for tests: 13.555 seconds Complete requests: 10000 Failed requests: 0 Total transferred: 509320000 bytes HTML transferred: 506630000 bytes Requests per second: 737.73 [#/sec] (mean) Time per request: 5.422 [ms] (mean) Time per request: 1.356 [ms] (mean, across all concurrent requests) Transfer rate: 36693.36 [Kbytes/sec] received
很遗憾,apache+tomcat组合的QPS:737.73
-------------
未完待续