文章 原始出处 :http://sofar.blog.51cto.com/353572/1601552
==========================================================================================
一、基础介绍
==========================================================================================
1、背景描述
目前我们的高可用DB的代理层采用的是360开源的Atlas,从上线以来,已稳定运行2个多月。无论是从性能上,还是稳定性上,相比其他开源组件(amoeba、cobar、MaxScale、MySQL-Proxy等),还是很出色的。
当初我们之所以选择Atlas,主要看中它有以下优点:
(1)、基于mysql-proxy-0.8.2进行修改,代码完全开源;
(2)、比较轻量级,部署配置也比较简单;
(3)、支持DB读写分离;
(4)、支持从DB读负载均衡,并自动剔除故障从DB;
(5)、支持平滑上下线DB;
(6)、具备较好的安全机制(IP过滤、账号认证);
(7)、版本更新、问题跟进、交流圈子都比较活跃。
在测试期间以及线上问题排查过程中,得到了360 Atlas作者朱超的热心解答,在此表示感谢。有关更多Atlas的介绍,我就不一一例举,可以参考以下链接:
https://github.com/Qihoo360/Atlas/blob/master/README_ZH.md
2、总体架构图
3、系统环境
CentOS 6.3 x86_64
==========================================================================================
二、安装部署
==========================================================================================
1、需注意的地方
(1)、本次安装不使用系统默认的glib库,之前的yum安装只是为了先解决依赖库的问题;
(2)、LUA库的版本不能太高,为5.1.x即可;
(3)、glib库的版本也不能太高,为glib-2.32.x即可;
(4)、对于编译不成功的情况,注意查看下面的说明。
2、GLIB依赖的基础库安装
# yum -y install *glib*
3、LUA库安装
http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz
# tar xvzf ncurses-5.9.tar.gz
# cd ncurses-5.9
# ./configure --prefix=/usr/local
# make && make install
ftp://ftp.gnu.org/gnu/readline/readline-6.2.tar.gz
# tar xvzf readline-6.2.tar.gz
# cd readline-6.2
# ./configure --prefix=/usr/local
# make && make install
http://www.lua.org/ftp/lua-5.1.5.tar.gz
# tar xvzf lua-5.1.5.tar.gz
# cd lua-5.1.5
# make linux install
注意:
修改当前目录下的“Makefile”中的 INSTALL_TOP= /usr/local为 INSTALL_TOP= /usr/local/lua
主要是为了避免与系统自带的lua库发生冲突的可能
在“src/Makefile”文件中加入“-lncurses”,完整内容如下:
linux:
$(MAKE) $(ALL) SYSCFLAGS="-DLUA_USE_LINUX" SYSLIBS="-Wl,-E -ldl -lncurses -lreadline"
4、GLIB库安装
ftp://sourceware.org/pub/libffi/libffi-3.0.13.tar.gz
# tar xvzf libffi-3.0.13.tar.gz
# cd libffi-3.0.13
# ./configure --prefix=/usr/local
# make && make install
http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.14.tar.gz
# tar xvzf libiconv-1.14.tar.gz
# cd libiconv-1.14
# ./configure --prefix=/usr/local
# make && make install
http://tukaani.org/xz/xz-5.0.5.tar.gz
# tar xvzf xz-5.0.5.tar.gz
# cd xz-5.0.5
# ./configure --prefix=/usr/local
# make && make install
# /sbin/ldconfig
http://ftp.gnome.org/pub/gnome/sources/glib/2.32/glib-2.32.4.tar.xz
# xz -d glib-2.32.4.tar.xz
# tar -xvf glib-2.32.4.tar
# cd glib-2.32.4
# ./configure --prefix=/usr/local/glib-2.32.4 \
--with-libiconv=/usr/local \
LIBFFI_CFLAGS="-I/usr/local/include" \
LIBFFI_LIBS="-L/usr/local/lib -lffi"
# make && make install
注意:编译报错处理
(1)、configure阶段
# vim ./glib/gconvert.c
注释掉第26、28行的内容
注释掉从61行到67行的内容
# vim ./configure
在7880行之上添加如下内容:
found_iconv=yes
(2)、make阶段
# ln -s /usr/local/lib/libffi-3.0.13/include/ffi.h /usr/local/include
# ln -s /usr/local/lib/libffi-3.0.13/include/ffitarget.h /usr/local/include
glib库需要安装在单独的目录“/usr/local/glib-2.32.4”,也是为了避免与系统自带的glib库发生冲突的可能
5、Atlas安装
(1)、其他基础组件安装
https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
# tar xvzf libevent-2.0.21-stable.tar.gz
# cd libevent-2.0.21-stable
# ./configure --prefix=/usr/local
# make && make install
http://www.openssl.org/source/openssl-1.0.1h.tar.gz
# tar xvzf openssl-1.0.1h.tar.gz
# cd openssl-1.0.1h
# ./config shared --prefix=/usr/local
# make && make install
(2)、MySQL安装(无需启动)
http://wwwNaNake.org/files/v2.8/cmake-2.8.10.2.tar.gz
# tar -xvzf cmake-2.8.10.2.tar.gz
# cd cmake-2.8.10.2
# ./bootstrap --prefix=/usr/local
# gmake --jobs=`grep processor /proc/cpuinfo | wc -l`
# gmake install
http://downloads.mysql.com/archives/get/file/mysql-5.5.24.tar.gz
# tar -xvzf mysql-5.5.24.tar.gz
# cd mysql-5.5.24
# rm-f CMakeCache.txt
# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DMYSQL_UNIX_ADDR=/var/run/mysql/mysql.sock \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci \
-DEXTRA_CHARSETS=all \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DENABLED_LOCAL_INFILE=1 \
-DWITH_EMBEDDED_SERVER=1 \
-DMYSQL_DATADIR=/data/dbdata/data \
-DMYSQL_TCP_PORT=3306
# make --jobs=`grep processor /proc/cpuinfo | wc -l`
# make install
(3)、DB中间件安装
https://github.com/Qihoo360/Atlas/archive/2.2.1.tar.gz
# tar xvzf Atlas-2.2.1.tar.gz
# cd Atlas-2.2.1
# ./configure --prefix=/usr/local/mysql-proxy \
--with-lua=/usr/local/lua \
--with-mysql=/usr/local/mysql \
GLIB_CFLAGS="-I/usr/local/glib-2.32.4/include/glib-2.0" \
GLIB_LIBS="-L/usr/local/glib-2.32.4/lib/glib-2.0 -lglib-2.0" \
GMODULE_CFLAGS="-I/usr/local/glib-2.32.4/include" \
GMODULE_LIBS="-L/usr/local/glib-2.32.4/lib -lgmodule-2.0" \
GTHREAD_CFLAGS="-I/usr/local/glib-2.32.4/include" \
GTHREAD_LIBS="-L/usr/local/glib-2.32.4/lib -lgthread-2.0" \
LUA_CFLAGS="-I/usr/local/lua/include" \
LUA_LIBS="-L/usr/local/lua/lib -llua-5.1" \
CFLAGS="-DHAVE_LUA_H -O2" \
LDFLAGS="-L/usr/local/lib -L/usr/local/lib64 -lm -ldl -lcrypto"
# make && make install
注意:
编译报错处理
# ln -s /usr/local/glib-2.32.4/lib/glib-2.0/include/glibconfig.h /usr/local/glib-2.32.4/include/glib-2.0
# cd /usr/local
# mv mysql-proxy atlas-2.2.1 && ln -s atlas-2.2.1 mysql-proxy
6、DB中间层配置
(1)、主配置
# vim /usr/local/mysql-proxy/conf/mysql-proxy.cnf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
(2)、启动脚本
# vim /etc/init.d/mysql-proxy
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
|
# chmod +x /etc/init.d/mysql-proxy
# chmod 0660 /usr/local/mysql-proxy/conf/mysql-proxy.cnf
# service mysql-proxy start
# ps aux | grep mysql-prox[y]
7、Atlas高可用【Keepalived】环境安装
http://rpm5.org/files/popt/popt-1.14.tar.gz
# tar xvzf popt-1.14.tar.gz
# cd popt-1.14
# ./configure --prefix=/usr/local
# make && make install
http://www.carisma.slowglass.com/~tgr/libnl/files/libnl-3.2.24.tar.gz
# tar xvzf libnl-3.2.24.tar.gz
# cd libnl-3.2.24
# ./configure --prefix=/usr/local
# make && make install
# ln -s /usr/local/include/libnl3/netlink /usr/local/include
# /sbin/ldconfig
http://www.keepalived.org/software/keepalived-1.2.10.tar.gz
# tar xvzf keepalived-1.2.10.tar.gz
# cd keepalived-1.2.10
# ./configure --prefix=/usr/local/keepalived
# make && make install
# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin
# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig
# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d
# chkconfig --add keepalived
8、Atlas高可用【Keepalived】配置
# mkdir –p /etc/keepalived /data/scripts
(1)、主节点配置
# vim /etc/keepalived/keepalived.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
|
(2)、备用节点配置
# vim /etc/keepalived/keepalived.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
|
(3)、VIP切换通知脚本
# vim /data/scripts/notify.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
|
(4)、DB中间层进程检查脚本
# vim /data/scripts/keepalived_check_mysql_proxy.sh
1 2 3 4 5 6 7 |
|
# chmod +x /data/scripts/*.sh
# service keepalived start
# ip addr show eth1
# ps aux | grep keepalive[d]
==========================================================================================
三、其他设置
==========================================================================================
1、Atlas服务监控
# vim /usr/local/mysql-proxy/bin/check_service.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
|
2、Atlas访问日志切割
# vim /data/scripts/cut_and_clear_access_log.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
3、crontab内容添加
# touch /var/lock/check_service.lock
# echo ‘touch /var/lock/check_service.lock‘ >> /etc/rc.d/rc.local
# crontab -uroot -e
1 2 |
|
4、平滑设置功能
# mysql -h10.209.6.101 -P3307 -usysadmin -p‘[email protected]()‘