一、Mysql Cluster拓扑环境
Management:ClusterManager 222.9.9.161
ndb1:Clusterndb1 222.9.9.162
ndb2:Clusterndb2 222.9.9.163
sql1:Clustersql1 222.9.9.164
sql1:Clustersql2 222.9.9.165
拓扑中的服务器均为HyperV虚拟机镜像拷贝,仅有Centos 6.4操作系统,需要做一些基本配置
1、配置网卡IP地址(可直接在网卡管理器上进行,删除原来的system eth0,新建一个有线网卡)
2、备份my.cnf
[[email protected] ~]# mv /etc/my.cnf /etc/my.cnf.bak
3、配置/etc/hosts
[[email protected] ~]# gedit /etc/hosts
222.9.9.161 ClusterManager
222.9.9.162 Clusterndb1
222.9.9.163 Clusterndb2
222.9.9.164 Clustersql1
222.9.9.165 Clustersql2
保存退出(所有服务器均须配置,步骤相同)
重启服务器
二、配置SSH免密码登录及应用
1、生成公钥和私钥(此一步在每台服务器上均需要执行)
[[email protected] ~]# ssh-keygen -t rsa
一路回车
[[email protected] ~]# cd /root/.ssh/
[[email protected] ~]# ll
目录下生成2个文件:id_rsa和id_rsa.pub,分别为私钥和公钥。
2、导入公钥到认证文件,更改权限
[[email protected] ~]# mv ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
[[email protected] ~]# chmod 700 ~/.ssh
[[email protected] ~]# chmod 600 ~/.ssh/authorized_keys
3、同步认证文件至其他4台服务器
[[email protected] ~]# scp ~/.ssh/authorized_keys [email protected]:/root/.ssh/
[[email protected] ~]# scp ~/.ssh/authorized_keys [email protected]:/root/.ssh/
[[email protected] ~]# scp ~/.ssh/authorized_keys [email protected]:/root/.ssh/
[[email protected] ~]# scp ~/.ssh/authorized_keys [email protected]:/root/.ssh/
过程中需要输入4台服务器的root密码,输入后回车即可
4、测试
[[email protected] ~]# ssh Clusterndb1
Last login: Tue Sep 22 10:57:45 2015 from clustermanager
[[email protected] ~]#
表示配置完成,其他三台同理测试。
5、将/data/目录下所有文件拷贝至其他4台服务器上
[[email protected] ~]# scp -r /data/* [email protected]:/data/
[[email protected] ~]# scp -r /data/* [email protected]:/data/
[[email protected] ~]# scp -r /data/* root[email protected]:/data/
[[email protected] ~]# scp -r /data/* [email protected]:/data/
三、部署ClusterManager(ndb1、ndb2、sql1、sql2节点部署时1~3步与此相同)
1、安装依赖库文件
[[email protected] ~]# yum install -y cmake gcc gcc-c++ bison ncurses-devel
2、配置java环境
[[email protected] data]# tar zxf jdk-7u7-linux-i586.tar.gz -C /usr/local/java
另起一个终端
[[email protected] ~]# gedit /etc/profile
export JAVA_HOME=/usr/local/java/jdk1.7.0_07
export CLASSPATH=/usr/local/java/jdk1.7.0_07/lib
export PATH=$JAVA_HOME/bin:$PATH
保存退出
[[email protected] ~]# source /etc/profile
[[email protected] ~]# java -version
java version "1.7.0_09-icedtea"
OpenJDK Runtime Enviroment (rhel-2.3.4.1.el6_3-i386)
OpenJDK Server VM (build 23.2-b09, mixed mode)
表示JDK环境配置正确,已生效。
3、安装mysql-cluster
回到第一个终端
[[email protected] data]# tar zxf mysql-cluster-gpl-7.2.8.tar.gz
[[email protected] data]# cd mysql-cluster-gpl-7.2.8
[[email protected] mysql-cluster-gpl-7.2.8]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DWITH_INNOBASE_STORAGE_ENGINE=ON -DWITH_MYISAM_STORAGE_ENGINE=1 -DEXTRA_CHARSETS=all -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci
[[email protected] mysql-cluster-gpl-7.2.8]# make
[[email protected] mysql-cluster-gpl-7.2.8]# make install
[[email protected] mysql-cluster-gpl-7.2.8]# cd
4、配置管理节点
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd mysql -g mysql
[[email protected] ~]# cd /usr/local/mysql
[[email protected] mysql]# mkdir -p clusterconf
[[email protected] mysql]# cp support-files/ndb-config-2-node.ini /usr/local/mysql/clusterconf/ndb-config.ini
[[email protected] mysql]# chown mysql:mysql -R /usr/local/mysql/
[[email protected] mysql]# cd clusterconf/
[[email protected] clusterconf]# gedit /ndb-config.ini
—————————————————————————————————
# Copyright (c) 2006, 2010, Oracle and/or its affiliates. All rights reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
# Example Ndbcluster storage engine config file.
#
[ndbd default]
NoOfReplicas= 2
MaxNoOfConcurrentOperations= 10000
DataMemory= 80M
IndexMemory= 24M
TimeBetweenWatchDogCheck= 30000
DataDir= /usr/local/mysql/data
MaxNoOfOrderedIndexes= 512
[ndb_mgmd default]
DataDir= /usr/local/mysql/data
[ndb_mgmd]
Id=1
HostName= 222.9.9.161
Datadir=/Data/apps/mysql/data
[ndbd]
Id= 2
HostName= 222.9.9.162
Datadir=/Data/apps/mysql/ndbdata
[ndbd]
Id= 3
HostName= 222.9.9.163
Datadir=/Data/apps/mysql/ndbdata
[mysqld]
Id= 4
HostName= 222.9.9.164
[mysqld]
Id= 5
HostName= 222.9.9.164
[mysqld]
Id= 6
[mysqld]
Id= 7
# choose an unused port number
# in this configuration 63132, 63133, and 63134
# will be used
[tcp default]
PortNumber= 63132
——————————————————————————————————
四、部署Clusterndb1及Clusterndb2
1、同ClusterManager部署第1步
2、同ClusterManager部署第2步
3、同ClusterManager部署第3步
4、配置Clusterndb1
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd mysql -g mysql
[[email protected] ~]# cd /usr/local/mysql
[[email protected] mysql]# mkdir -p ndbdata
[[email protected] mysql]# chown mysql:mysql -R /usr/local/mysql/ndbdata
[[email protected] mysql]# cp support-files/my-medium.cnf /etc/my.cnf
[[email protected] mysql]# gedit /etc/my.cnf
在末尾加上如下内容:
[mysqld]
datadir=/usr/local/mysql/ndbdata
ndbclustter
ndb-connectstring=222.9.9.161
[mysql_cluster]
ndb-connectstring=222.9.9.161
并修改server id 为2
保存退出
5、配置Clusterndb2
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd mysql -g mysql
[[email protected] ~]# cd /usr/local/mysql
[[email protected] mysql]# mkdir -p ndbdata
[[email protected] mysql]# chown mysql:mysql -R /usr/local/mysql/ndbdata
[[email protected] mysql]# cp support-files/my-medium.cnf /etc/my.cnf
[[email protected] mysql]# gedit /etc/my.cnf
在末尾加上如下内容:
[mysqld]
datadir=/usr/local/mysql/ndbdata
ndbclustter
ndb-connectstring=222.9.9.161
[mysql_cluster]
ndb-connectstring=222.9.9.161
并修改server id 为3
保存退出
五、部署Clustersql1及Clustersql2
1、同ClusterManager部署第1步
2、同ClusterManager部署第2步
3、同ClusterManager部署第3步
4、配置Clustersql1
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd mysql -g mysql
[[email protected] ~]# cd /usr/local/mysql
[[email protected] mysql]# cp support-files/my-medium.cnf /etc/my.cnf
在末尾加上如下内容:
[mysqld]
datadir=/usr/local/mysql/ndbdata
ndbclustter
default-storage-engine=ndbcluster
[mysql_cluster]
ndb-connectstring=222.9.9.161
并修改server id 为4
保存退出
初始化数据库:
[[email protected] mysql]# /usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data
5、配置Clustersql2
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd mysql -g mysql
[[email protected] ~]# cd /usr/local/mysql
[[email protected] mysql]# cp support-files/my-medium.cnf /etc/my.cnf
在末尾加上如下内容:
[mysqld]
datadir=/usr/local/mysql/ndbdata
ndbclustter
default-storage-engine=ndbcluster
[mysql_cluster]
ndb-connectstring=222.9.9.161
并修改server id 为5
保存退出
初始化数据库:
[[email protected] mysql]# /usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data
六、启动Mysql集群
正确的启动顺序为:管理节点 → 数据节点 → SQL节点
关闭顺序为:SQL节点 → 数据节点 → 管理节点
1、启动管理节点
[[email protected] ~]# /usr/local/mysql/bin/ndb_mgmd -f /usr/local/mysql/clusterconf/ndb-config.ini
若需要每次开机后自行启动,可将命令加入/etc/rc.d/rc.local配置文件中,如
[[email protected] ~]# gedit /etc/rc.d/rc.local
在touch /var/lock/subsys/local后回车,加入命令行
/usr/local/mysql/bin/ndb_mgmd -f /usr/local/mysql/clusterconf/ndb-config.ini
保存退出即可
2、启动数据节点
首次启动或重新初始化时需添加 --initial选项,之后不用加。
[[email protected] ~]# /usr/local/mysql/bin/ndbd --initial
2015-09-22 15:48:17 [ndbd] INFO --Angel connected to ‘222.9.9.161:1186‘
2015-09-22 15:48:17 [ndbd] INFO --Angel allocated nodeid: 2
[[email protected] ~]# /usr/local/mysql/bin/ndbd --initial
2015-09-22 15:48:18 [ndbd] INFO --Angel connected to ‘222.9.9.161:1186‘
2015-09-22 15:48:18 [ndbd] INFO --Angel allocated nodeid: 3
同样也可以配置开机启动
在最后加上一行命令即可
/usr/local/mysql/bin/ndbd
3、启动SQL节点
由于配置时已经进行了初始化,直接启动即可
[[email protected] ~]# /usr/local/mysql/bin/mysqld_safe &
[[email protected] ~]# /usr/local/mysql/bin/mysqld_safe &
或者将其设置为service启动
[[email protected] ~]# cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld
[[email protected] ~]# chmod +x /etc/init.d/mysqld
[[email protected] ~]# chkconfig mysqld on
Clustersql2配置相同,之后就可以通过以下命令来启动了
[[email protected] ~]# service mysqld start
4、登录管理节点查看
[[email protected] ~]# /usr/local/mysql/bin/ndb_mgm
-- NDB Cluster -- Management CLient --
ndb_mgm>show
Connected to Management Server at: localhost:1186
Cluster Configuration
------------------------
[ndbd(NDB)]2 node(s)
id=2@222.9.9.162(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0, Master)
id=3@222.9.9.163(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0)
[ndbd_mgmd(MGM)]1 node(s)
id=1@222.9.9.161(mysql-5.5.27 ndb-7.2.8)
[mysqld(API)]4 node(s)
id=4@222.9.9.164(mysql-5.5.27 ndb-7.2.8)
id=5@222.9.9.165(mysql-5.5.27 ndb-7.2.8)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)
ndb_mgm>exit
七、简单高可用性测试
1、数据同步测试
在sql1上执行写数据库、表操作:
首先关联系统mysql链接
[[email protected] ~]# ln -s /usr/local/mysql/bin/mysql /usr/bin
进入mysql>模式
[[email protected] ~]# mysql -u root
mysql>
新建test1库并使用
mysql> create database test1;
Query OK , 1 row affected (0.13 sec)
mysql> use test1;
Database changed
新建tab1表
mysql> create table tab1(id int(4));
Query OK , 0 rows affected (1.70 sec)
插入测试记录
mysql> insert into test1.tab1 values(1234).(5678);
Query OK , 2 rows affected (0.01 sec)
Records: 2 Duplicates : 0 Warnings : 0
确认添加的表记录
mysql> select * from tab1;
+---------+
| id |
+---------+
| 1234 |
| 5678 |
+---------+
2 rows in set (0.00 sec)
mysql>
在sql2上确认结果
[[email protected] ~]# ln -s /usr/local/mysql/bin/mysql /usr/bin
[[email protected] ~]# mysq -u root
mysql> show databases;
+------------------------+
| Database |
+------------------------+
| information_schema |
| mysql |
| ndbinfo |
| performance_schema |
| test |
| test1 |
+------------------------+
6 rows in set (0.00 sec)
以上信息确认可以看到test1库
切换到test1库
mysql> use test1;
Database changed
mysql> select * from tab1;
+---------+
| id |
+---------+
| 1234 |
| 5678 |
+---------+
2 rows in set (0.04 sec)
确认查看到的表记录也一致
2、数据节点高可用测试
关闭数据节点Clusterndb1
[[email protected] ~]# netstat -anpt|grep ndbd
[[email protected] ~]# killall -9 ndbd
在管理节点上查看集群状态
[[email protected] ~]# /usr/local/mysql/bin/ndb_mgm
ndb_mgm>show
Connected to Management Server at: localhost:1186
Cluster Configuration
------------------------
[ndbd(NDB)]2 node(s)
id=2 (not connected, accepting connect from 222.9.9.162)
id=3@222.9.9.163(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0, Master)
[ndbd_mgmd(MGM)]1 node(s)
id=1@222.9.9.161(mysql-5.5.27 ndb-7.2.8)
[mysqld(API)]4 node(s)
id=4@222.9.9.164(mysql-5.5.27 ndb-7.2.8)
id=5@222.9.9.165(mysql-5.5.27 ndb-7.2.8)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)
发现Clusterndb1已经断开连接,Clusterndb2接管成Master状态。
从sql1或sql2上进行读写数据库,如在test1.tab1表中再添加1条记录:
mysql> insert into test1.tab1 values(4021);
Query OK , 1 row affected (0.00 sec)
mysql> select * from test1.tab1;
+---------+
| id |
+---------+
| 1234 |
| 4021 |
| 5678 |
+---------+
3 rows in set (0.00 sec)
以上测试说明:只要还有一台数据节点可用,Mysql数据库整体依然可用。
重新启动Clusterndb1节点的ndbd服务。
[[email protected] ~]# /usr/local/mysql/bin/ndbd
2015-09-23 09:29:46 [ndbd] INFO --Angel connected to ‘222.9.9.161:1186‘
2015-09-23 09:29:46 [ndbd] INFO --Angel allocated nodeid: 2
在管理节点上查看集群状态
ndb_mgm>show
Connected to Management Server at: localhost:1186
Cluster Configuration
------------------------
[ndbd(NDB)]2 node(s)
id=2@222.9.9.162(mysql-5.5.27 ndb-7.2.8, starting,Nodegroup: 0)
id=3@222.9.9.163(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0, Master)
[ndbd_mgmd(MGM)]1 node(s)
id=1@222.9.9.161(mysql-5.5.27 ndb-7.2.8)
[mysqld(API)]4 node(s)
id=4@222.9.9.164(mysql-5.5.27 ndb-7.2.8)
id=5@222.9.9.165(mysql-5.5.27 ndb-7.2.8)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)
发现Clusterndb1已经重新接入集群,但是没有抢占Clusterndb2的Master状态
接下来关闭Clusterndb2节点
[[email protected] ~]# netstat -anpt|grep ndbd
[[email protected] ~]# killall -9 ndbd
在管理节点上查看集群状态
ndb_mgm>show
Cluster Configuration
------------------------
[ndbd(NDB)]2 node(s)
id=2@222.9.9.162(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0, Master)
id=3 (not connected, accepting connect from 222.9.9.163)
[ndbd_mgmd(MGM)]1 node(s)
id=1@222.9.9.161(mysql-5.5.27 ndb-7.2.8)
[mysqld(API)]4 node(s)
id=4@222.9.9.164(mysql-5.5.27 ndb-7.2.8)
id=5@222.9.9.165(mysql-5.5.27 ndb-7.2.8)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)
Clusterndb2未连接,Clusterndb1接管了Master
在Clustersql1或Clustersql2上确认结果
mysql> select * from test1.tab1;
+---------+
| id |
+---------+
| 1234 |
| 4021 |
| 5678 |
+---------+
3 rows in set (0.00 sec)
以上测试说明:因故障中断的数据节点(Clusterndb1)恢复后,会立即从正常的数据节点(Clusterndb2)上同步数据。
3、SQL节点高可用测试
关闭Clustersql1节点
[[email protected] ~]# netstat -anptu |grep mysql
[[email protected] ~]# service mysqld stop
在管理节点查看集群状态
Cluster Configuration
------------------------
[ndbd(NDB)]2 node(s)
id=2@222.9.9.162(mysql-5.5.27 ndb-7.2.8, Nodegroup: 0, Master)
id=3 (not connected, accepting connect from 222.9.9.163)
[ndbd_mgmd(MGM)]1 node(s)
id=1@222.9.9.161(mysql-5.5.27 ndb-7.2.8)
[mysqld(API)]4 node(s)
id=4 (not connected, accepting connect from 222.9.9.164)
id=5@222.9.9.165(mysql-5.5.27 ndb-7.2.8)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)
由于之前Clusterndb2已关闭,现在群集只剩下管理节点、Clusterndb1及Clustersql2,确认是否可用。
在Clustersql2上确认结果
mysql> select * from test1.tab1;
+---------+
| id |
+---------+
| 1234 |
| 4021 |
| 5678 |
+---------+
3 rows in set (0.01 sec)
结果正常。
同理,再开启Clustersql1,关闭Clustersql2节点,在Clustersql1上确认结果
mysql> select * from test1.tab1;
+---------+
| id |
+---------+
| 1234 |
| 4021 |
| 5678 |
+---------+
3 rows in set (0.01 sec)
结果亦正常。
测试完成。