一、服务器概况
Galera Cluster需要至少三个节点，在此次实验过程中，三个节点IP地址：
192.168.56.101
192.168.56.102
192.168.56.103
OS为centos 7.2
服务器配置：
4G 内存，2核CPU，20G数据磁盘空间。
关闭防火墙:
service iptables stop systemctl stop firewalld chkconfig iptables off
关闭SELinux:
/etc/selinux/config中的SELINUX=disabled
修改主机名
vi /etc/hostname MariaDB01 MariaDB02 MariaDB03 systemctl restart systemd-hostnamed
三台主机上加入/etc/hosts:
192.168.56.101 MariaDB01 192.168.56.102 MariaDB02 192.168.56.103 MariaDB03
二、MariaDB Galera Cluster软件二进制安装
官网下载：https://downloads.mariadb.org/ 10.0.29
初始化节点1
su - mysql cd /app/mysql/ vi my.cnf
--my.cnf [client] port = 3306 socket = /tmp/mysql.sock [mysqld] port = 3306 socket = /tmp/mysql.sock character-set-server=utf8 skip-name-resolve basedir=/app/mysql datadir=/data/mysqldata back_log = 50 binlog_cache_size = 1M max_heap_table_size = 64M read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M thread_cache_size = 8 thread_concurrency = 8 ft_min_word_len = 4 binlog_format=ROW default-storage-engine=innodb innodb_autoinc_lock_mode=2 thread_stack = 240K transaction_isolation = REPEATABLE-READ tmp_table_size = 64M log-bin=/data/binlog/mysql-bin slow_query_log long_query_time = 2 log_error=/data/logs/error.log slow_query_log_file=/data/logs/slow.log server-id = 2 key_buffer_size = 32M bulk_insert_buffer_size = 64M myisam_sort_buffer_size = 128M myisam_max_sort_file_size = 10G myisam_repair_threads = 1 myisam_recover innodb_additional_mem_pool_size = 16M innodb_buffer_pool_size = 1G innodb_data_file_path = ibdata1:500M;ibdata2:16M:autoextend innodb_file_per_table = 1 innodb_support_xa = on sync_binlog = 0 table_definition_cache=65535 table_open_cache=4000 max_connections=3100 max_user_connections=3000 max_connect_errors=65536 max_allowed_packet=16M innodb_strict_mode = 1 innodb_open_files = 4000 innodb_file_format = Barracuda innodb_file_format_max = Barracuda innodb_flush_method = O_DIRECT expire_logs_days=3 innodb_data_home_dir = /data/mysqldata innodb_write_io_threads = 8 innodb_read_io_threads = 8 innodb_thread_concurrency = 16 innodb_flush_log_at_trx_commit = 1 innodb_log_buffer_size = 128M innodb_log_file_size = 256M innodb_log_files_in_group = 3 innodb_log_group_home_dir = /data/redolog innodb_max_dirty_pages_pct = 90 innodb_flush_method=O_DIRECT innodb_lock_wait_timeout = 120 [galera] bind-address=0.0.0.0 wsrep_provider=none wsrep_node_address=‘192.168.56.101‘ wsrep_provider=/app/mariadb-galera-10.0.29-linux/lib/galera/libgalera_smm.so wsrep_provider_options=‘gcache.size=1999M‘ wsrep_cluster_name=‘galera_cluster‘ wsrep_cluster_address =‘gcomm://192.168.56.101,192.168.56.102,192.168.56.103‘ wsrep_node_name=‘MariaDB01‘ wsrep_slave_threads=4 wsrep_certify_nonPK=1 wsrep_max_ws_rows=131072 wsrep_max_ws_size=1073741824 wsrep_debug=0 wsrep_convert_LOCK_to_trx=0 wsrep_retry_autocommit=1 wsrep_auto_increment_control=1 wsrep_replicate_myisam=1 wsrep_drupal_282555_workaround=0 wsrep_causal_reads=0 wsrep_sst_method=rsync wsrep_sst_auth= sst_user:dbpass [mysqldump] quick max_allowed_packet = 16M [mysql] no-auto-rehash [myisamchk] key_buffer_size = 512M sort_buffer_size = 512M read_buffer = 8M write_buffer = 8M [mysqlhotcopy] interactive-timeout [mysqld_safe] open-files-limit = 8192
./scripts/mysql_install_db --defaults-file=my.cnf ./bin/mysqld_safe --defaults-file=my.cnf --wsrep-new-cluster &
初始化节点2、3
./scripts/mysql_install_db --defaults-file=my.cnf ./bin/mysqld_safe --defaults-file=my.cnf &
为集群创建帐号
mysql> DELETE FROM mysql.user WHERE user=‘‘; mysql> GRANT ALL ON *.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘dbpass‘; mysql> GRANT USAGE ON *.* to [email protected]‘%‘ IDENTIFIED BY ‘dbpass‘; mysql> GRANT ALL PRIVILEGES on *.* to [email protected]‘%‘; mysql> FLUSH PRIVILEGES;
三、日常维护
1. 集群启动
启动第一个节点：service mysql start --wsrep-new-cluster
启动其它节点：server mysql start
关闭节点：server mysql stop
2. 通过查看4567端口确认集群是否启动
netstat -plantu | grep mysqld
3. 查询 galera插件是否已启用
SHOW status LIKE ‘wsrep_ready‘; SHOW VARIABLES LIKE ‘wsrep_cluster_address‘;
4. 检查集群是否正常
查看Galera集群状态: mysql> show status like ‘wsrep%‘;
(1)SHOW GLOBAL STATUS LIKE ‘wsrep_cluster_state_uuid‘; 集群中每个节点的值必须一致。 (2)SHOW GLOBAL STATUS LIKE ‘wsrep_cluster_size‘; 显示集群的规模，此例应该显示”3”。 (3)SHOW GLOBAL STATUS LIKE ‘wsrep_cluster_status‘; 显示”Primary”为正常。 (4)SHOW GLOBAL STATUS LIKE ‘wsrep_ready‘; 显示”ON”为正常。 (5) SHOW GLOBAL STATUS LIKE ‘wsrep_connected‘; 显示”ON”为正常。 (6) SHOW GLOBAL STATUS LIKE ‘wsrep_local_state_comment‘; 显示” Joined”为正常。
cluster完整性检测
status 变量:
wsrep_cluster_state_uuid: cluster中的所有node的该变量的值必须一样, 不一样表示node没有连接到cluster。
wsrep_cluster_conf_id: 次变量用来表示node是否在它相应的cluster中。cluster中所有node的该变量值应该一样， 不一样则表示nodes被分隔开了, node恢复的时候该变量也会恢复。
wsrep_cluster_size: 表示cluster中有多少node节点， 等于预期的数量则表示所有node连接到了cluster。
wsrep_cluster_status: 正常情况下值为Primary, 如果不为Primay,则该node当前不能操作(归咎于多成员关系的改变和quorum的缺失), 同时也可能满足split-brain的条件。
如果cluster中没有node 连接上(connected) PC(就是所有node属于同一部分,但是node都是non-primary状态),
wsrep_sst_method的参数说明：
Method Speed Blocks Donor? Available on Live Node Type Requires root access to database server? mysqldump slow yes yes logical donor and joiner rsync fastest yes no physical none xtrabackup fast briefly no physical donor only
rsync在数据同步（SST和IST)的时候，速度最快，但是会锁住提供数据的节点，xtrabackup只会短暂的锁住节点。
四、高可用测试
1、节点数据同步写入
2、故障节点修复后数据自行修复
附录
MariaDB Galera Cluste安装报错处理
170525 23:19:48 [ERROR] WSREP: wsrep_load(): dlopen(): libssl.so.6: cannot open shared object file: No such file or directory 170525 23:19:48 [ERROR] WSREP: wsrep_load(/app/mariadb-galera-10.0.29-linux/lib/galera/libgalera_smm.so) failed: Invalid argument (22). Reverting to no provider. cd /app/mysql/lib/galera/ ldd ./libgalera_smm.so cd /usr/lib64 ln -s libcrypto.so.1.0.1e libcrypto.so.6 ln -s libssl.so.1.0.1e libssl.so.6
170526 0:04:07 [Warning] WSREP: Gap in state sequence. Need state transfer. 170526 0:04:07 [Note] WSREP: Running: ‘wsrep_sst_rsync --role ‘joiner‘ --address ‘192.168.56.103‘ --datadir ‘/data/mysqldata/‘ --defaults-file ‘/app/mariadb-galera-10.0.29-linux/my.cnf‘ --defaults-group-suffix ‘‘ --parent ‘2832‘ --binlog ‘/data/binlog/mysql-bin‘ ‘ which: no rsync in (/usr/sbin:/sbin:/app/mariadb-galera-10.0.29-linux//bin:/app/mysql/bin:/bin:/usr/bin:/usr/local/bin:/usr/sbin) 170526 0:04:07 [ERROR] WSREP: Failed to read ‘ready <addr>‘ from: wsrep_sst_rsync --role ‘joiner‘ --address ‘192.168.56.103‘ --datadir ‘/data/mysqldata/‘ --defaults-file ‘/app/mariadb-galera-10.0.29-linux/my.cnf‘ --defaults-group-suffix ‘‘ --parent ‘2832‘ --binlog ‘/data/binlog/mysql-bin‘ Read: ‘‘rsync‘ not found in PATH‘ 170526 0:04:07 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role ‘joiner‘ --address ‘192.168.56.103‘ --datadir ‘/data/mysqldata/‘ --defaults-file ‘/app/mariadb-galera-10.0.29-linux/my.cnf‘ --defaults-group-suffix ‘‘ --parent ‘2832‘ --binlog ‘/data/binlog/mysql-bin‘ : 2 (No such file or directory) 170526 0:04:07 [ERROR] WSREP: Failed to prepare for ‘rsync‘ SST. Unrecoverable. 170526 0:04:07 [ERROR] Aborting yum install rsync.x86_64
which: no lsof in (/usr/sbin:/sbin:/app/mariadb-galera-10.0.29-linux//bin:/app/mysql/bin:/bin:/usr/bin:/usr/local/bin:/usr/sbin:/app/xtrabackup/percona-xtrabackup/bin) 170526 10:13:28 [ERROR] WSREP: Failed to read ‘ready <addr>‘ from: wsrep_sst_rsync --role ‘joiner‘ --address ‘192.168.56.103‘ --datadir ‘/data/mysqldata/‘ --defaults-file ‘/app/mariadb-galera-10.0.29-linux/my.cnf‘ --defaults-group-suffix ‘‘ --parent ‘2821‘ --binlog ‘/data/binlog/mysql-bin‘ Read: ‘‘lsof‘ not found in PATH‘ 170526 10:13:28 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role ‘joiner‘ --address ‘192.168.56.103‘ --datadir ‘/data/mysqldata/‘ --defaults-file ‘/app/mariadb-galera-10.0.29-linux/my.cnf‘ --defaults-group-suffix ‘‘ --parent ‘2821‘ --binlog ‘/data/binlog/mysql-bin‘ : 2 (No such file or directory) 170526 10:13:28 [ERROR] WSREP: Failed to prepare for ‘rsync‘ SST. Unrecoverable. 170526 10:13:28 [ERROR] Aborting yum install lsof
时间： 2024-08-07 08:45:37