DRDB实现mariadb高可用

------本文大纲

简介

DRBD资源属性

原理

安装DRDB

DRDB实现高可用

============

一、简介

DRBD是一种块设备，可以被用于高可用(HA)之中。它类似于一个网络RAID-1功能。当你将数据写入本地文件系统时，数据还将会被发送到网络中另一台主机上。以相同的形式记录在一个文件系统中。本地(主节点)与远程主机(备节点)的数据可以保证实时同步。当本地系统出现故障时，远程主机上还会保留有一份相同的数据，可以继续使用。在高可用(HA)中使用DRBD功能，可以代替使用一个共享盘阵。因为数据同时存在于本地主机和远程主机上。切换时，远程主机只要使用它上面的那份备份数据，就可以继续进行服务了。

二、DRBD资源属性

resource name

可以使用除空白字符外的任意ACSII表中的字符；

drbd设备

drbd的设备的访问路径，设备文件/dev/drbd#；

disk

各节点为组成此drbd设备所提供的块设备，通常是一个磁盘分区；

网络属性

节点间为了实现跨主机磁盘镜像而使用的网络配置；

注意：用户空间工具与drdb与内核中使用的模块版本要保持一致，只有在使用drbdadm工具时才会读取配置文件，对多个资源的公共配置，可以提取出来只配置一次，通常保存在Common中，此外还有global配置，这种配置跟资源本身没有关系的；

三、原理

需要知道，DRBD是工作在系统内核空间，而不是用户空间，它直接复制的是二进制数据，这是它速度快的根本原因。

其次，DRBD至少需要两个节点来工作，一主一次。

DRBD的文件同步过程和普通复制过程的不同：

DRBD在数据进入Buffer Cache时，先经过DRBD这一层，复制一份数据经过TCP/IP协议封装，发送到另一个节点上，另一个节点通过TCP/IP协议来接受复制过来的数据，同步到次节点的DRBD设备上。

  • DRBD底层设备支持

DRBD需要构建在底层设备之上，然后构建出一个块设备出来。对于用户来说，一个DRBD设备，就像是一块物理的磁盘，可以在上面内创建文件系统。DRBD所支持的底层设备有以下这些类：

1、一个磁盘，或者是磁盘的某一个分区；

2、一个soft raid 设备；

3、一个LVM的逻辑卷；

  • DRBD在远程传输上支持三种模式：

协议A:异步复制协议

只要主节点完成本地写操作就认为写操作完成，并且需要复制的数据包会被存放本地TCP发送缓存中，当发生failover故障，数据可能会丢失。

协议B：内存同步（半同步，semi-synchronous）复制协议

当本地磁盘的写己经完成，并且复制数据包己经到达对应从节点，此时主节点才认为磁盘写入己经完成，failover不会导致数据丢失，但当所有节点同时出现故障时，主节点上多数最新写入的数据可能丢失。

协议C:同步复制

只有在本地和远程磁盘都确定写入己完成后，主节点才会认为写入完成。这样可确保发生单点故障时不会导致任何数据丢失，如果发生数据丢失，那也只会在所有节点同时存在错误存储时才会发生这种情况

  • 角色、模式

    • 角色
      在drbd构造的集群中，资源具有角色的概念，分别为primary和secondary。
      所有设为primary的资源将不受限制进行读写操作。可以创建文件系统，可以使用裸设备,甚至直接io。所有设为secondary的设备中不能挂载，不能读写
    • 模式

      drbd也有drbd mode：单主模型（主从），双主模型
      在单主模型下drbd可以使用任意的文件系统
      单在双主模型下只能使用集群文件系统，常用的开源的集群文件系统有:ocfs2和gfs2

四、安装DRDB

前提：

设置主机名（要与uname -n一致，主机名可解析）

时间同步

无密钥登录

本机环境

系统 主机名 ip地址
Centos6.5 x86_64 essun.node1.com 192.168.1.109
Centos6.5 x86_64 essun.node2.com 192.168.1.108

注：

内核模块程序包一定要跟你的系统内核保持一致，uname -r查看你的内核版本，内核模块的版本必须要严格对应，而用户空间的模块就不那么严格要求了，两个节点的时间也需要保持一致性；

第一步、下载与内核版本相应的内核模块与用户空间工具（地址：http://packages.atrpms.net/dist/el6/drbd/）

[[email protected] download]# uname -r
2.6.32-431.el6.x86_64
[[email protected] download]# ls
drbd-8.4.3-33.el6.x86_64.rpm  drbd-kmdl-2.6.32-431.el6-8.4.3-33.el6.x86_64.rpm
[[email protected] download]# yum isntall drbd-kmdl-2.6.32-431.el6-8.4.3-33.el6.x86_64.rpm drbd-8.4.3-33.el6.x86_64.rpm -y
  • 修改配置文件（/etc/drbd.d/global_common.conf）
  • [[email protected] drbd.d]# vim global_common.conf
[[email protected] drbd.d]# cat global_common.conf
global {
    usage-count no;
#  是否参加DRBD使用者统计.默认是yes
    # minor-count dialog-refresh disable-ip-verification
}

common {
    handlers {
        # These are EXAMPLE handlers only.
        # They may have severe implications,
        # like hard resetting the node under certain circumstances.
        # Be careful when chosing your poison.

        pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
        pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
        local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
        # fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
        # split-brain "/usr/lib/drbd/notify-split-brain.sh root";
        # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
        # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";
        # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
    }

    startup {
        # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb

        #wfc-timeout 120;
                 #(等待另一个节点上线的超时时长)
       # degr-wfc-timeout 240;
                #(等待超时后做降级处理)
        #outdated-wfc-timeout
                #(过期的等待超时)

                # wait-after-sb
                #(脑裂之后等待多长时长)
    }

    options {
        # cpu-mask on-no-data-accessible
    }

    disk {
        # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes
        on-io-error detach;
                # 同步错误的做法是分离
        # disk-drain md-flushes resync-rate resync-after al-extents
                # c-plan-ahead c-delay-target c-fill-target c-max-rate
                # c-min-rate disk-timeout
    }

    net {
        protocol C;
 # 使用协议C.表示收到远程主机的写入确认后,则认为写入完成.
        cram-hmac-alg "sha1";
        shared-secret "drdb.essun.drdb.org";
 # 设置主备机之间通信使用的信息算法.
        # protocol timeout max-epoch-size max-buffers unplug-watermark
        # connect-int ping-int sndbuf-size rcvbuf-size ko-count
        # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri
        # after-sb-1pri after-sb-2pri always-asbp rr-conflict
        # ping-timeout data-integrity-alg tcp-cork on-congestion
        # congestion-fill congestion-extents csums-alg verify-alg
        # use-rle
    }
    syncer {

        rate 1000M;
}
 # 设置主备节点同步时的网络速率最大值,单位是字节. 

}

    提供物理块设备（两个节点的分区号最好相同），不需要格式化

[[email protected] drbd.d]# fdisk -cu /dev/s
scd0      sda1      sg0       shm/      snd/      stderr    stdout
sda       sda2      sg1       snapshot  sr0       stdin     systty
[[email protected] drbd.d]# fdisk -cu /dev/sda
Command (m for help): p
Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders, total 251658240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006c47d
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      411647      204800   83  Linux
/dev/sda2          411648   126240767    62914560   8e  Linux LVM
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 3
First sector (126240768-251658239, default 126240768):
Using default value 126240768
Last sector, +sectors or +size{K,M,G} (126240768-251658239, default 251658239): +10G
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[[email protected] drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
[[email protected] drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3
[[email protected] drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3
[[email protected] drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3
  • 定义DRBD磁盘资源
[[email protected] drbd.d]# vim dbresouce.res
resource dbresource {
        on essun.node1.com { #on说明在哪个节点上，跟uname -n保持一致
                device /dev/drbd0; #在磁盘上表现的drbd叫什么名；
                disk   /dev/sda3; #所使用的物理磁盘设备是哪个；
                address 192.168.1.109:7789; #在node1这个节点上监听的套接字,默认监听在7789端口上；
                meta-disk internal; #保存drbd源数据信息，表示就放在自己的磁盘区分上，也可以放在外部的磁盘上；
        }
        on essun.node2.com{
                device /dev/drbd0;
                disk /dev/sda3;
                address 192.168.1.108:7789;
                meta-disk internal;
        }
}
  • 将这两个文件拷贝到node1中的相同目录下
[[email protected] drbd.d]# scp dbresouce.res global_common.conf essun.node1.com:/etc
/drbd.d/
dbresouce.res                               100%  259     0.3KB/s   00:00
global_common.conf                          100% 2044     2.0KB/s   00:00
[[email protected] drbd.d]#
  • 在各自的节点上初始化资源
[[email protected] drbd.d]# drbdadm create-md dbresource
#初始化资源列表
Writing meta data...
initializing activity log
NOT initializing bitmap
lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd) failed: No such file or directory
New drbd meta data block successfully created.
lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd) failed: No such file or directory
[[email protected] drbd.d]# service drbd start
#启动服务
Starting DRBD resources: [
     create res: dbresource
   prepare disk: dbresource
    adjust disk: dbresource
     adjust net: dbresource
]
.
[[email protected] drbd.d]# drbd-overview
#查看drbd的状态
  0:dbresource/0  Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
[[email protected] drbd.d]# watch -n 1 ‘cat /proc/drbd‘
#实时查看drbd的状态
Every 1.0s: cat /proc/drbd                                                     Thu May  1 08:48:23 2014
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2013-11-29 12:28:00
 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:10485404

上面表示两个节点的drbd都是从的，这是因为我们并没有对其中任何一个提升主的。

  • 将其中一个节点的DRBD资源提升为主资源
[[email protected] drbd.d]# drbdadm primary --force dbresource
#将当前节点的drbd资源提升为主资源
[[email protected] drbd.d]# drbd
[[email protected] drbd.d]# drbd-overview
  0:dbresource/0  SyncSource Primary/Secondary UpToDate/Inconsistent C r---n-
    [>...................] sync‘ed:  6.1% (9624/10236)M
  • 初始化之后（角色己经发生改变了）
[[email protected] drbd.d]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----
  • 格式化DRBD
[[email protected] drbd.d]# mke2fs -t ext4 /dev/drbd0
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2621351 blocks
131067 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[[email protected] drbd.d]# mount /dev/drbd0 /mnt
[[email protected] drbd.d]# cd mnt
-bash: cd: mnt: No such file or directory
[[email protected] drbd.d]# cd /mnt
[[email protected] mnt]# ls
lost+found
[[email protected] mnt]# cp /etc/inittab .
[[email protected] mnt]# ls
inittab  lost+found
[[email protected] mnt]#
  • 提升另一个节点的DRBD资源
[[email protected] /]# umount /mnt
#卸载drbd
[[email protected] /]# drbdadm secondary dbresource
# 降级DRBD资源
[[email protected] /]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
#以下是在另一个节点操作
[[email protected] ~]# drbdadm primary dbresource
[[email protected] ~]# mount /dev/drbd0 /mnt
[[email protected] ~]# cd /mnt
[[email protected] mnt]# ls
inittab  lost+found

如果想让DRBD能够自动切换的话，就要借助于messages layer程序了。

首先在对DRBD降级，停止服务，禁止开机自动启动

[[email protected] /]# umount /mnt
[[email protected] /]# drbdadm secondary dbresource
[[email protected] /]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[[email protected] /]# service drbd stop
Stopping all DRBD resources: .
[[email protected] /]# chkconfig drbd off
[[email protected] /]# chkconfig --list drbd
drbd            0:off   1:off   2:off   3:off   4:off   5:off   6:off

第二步、安装mariadb

[[email protected] download]# tar xf mariadb-10.0.10-linux-x86_64.tar.gz
[[email protected] download]# mv mariadb-10.0.10-linux-x86_64 /usr/local
[[email protected] download]# cd /usr/local/
[[email protected] local]# ln -s mariadb-10.0.10-linux-x86_64 mysql
[[email protected] local]# groupadd -r mysql
[[email protected] local]# useradd -r -g mysql mysql
[[email protected] local]# chown -R mysql.mysql mari/
  • 启动DRBD服务，提升主资源
[[email protected] ~]# service drbd start
Starting DRBD resources: [
     create res: dbresource
   prepare disk: dbresource
    adjust disk: dbresource
     adjust net: dbresource
]
.
[[email protected] local]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[[email protected] local]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----
  • 挂载数据目录
[[email protected] local]# mkdir /mariadb
[[email protected] local]# mount /dev/drbd0 /mariadb/
[[email protected] local]# cd /mariadb/
[[email protected] mariadb]# mkdir {data,logs}
[[email protected] mariadb]# ls
data   logs  lost+found
[[email protected] /]# chown -R mysql.mysql /mariadb/
[[email protected] /]# ll /mariadb/
total 24
drwxr-xr-x 2 mysql mysql  4096 May  1 11:09 data
drwxr-xr-x 2 mysql mysql  4096 May  1 11:09 logs
drwx------ 2 mysql mysql 16384 May  1 09:42 lost+found

注：

此目录在另一个节点上也要一样。

  • 修改配置文件与添加启动脚本
[[email protected] mari]# cp support-files/my-large.cnf /etc/cnf
[[email protected] mari]# vim /etc/my.cnf
[[email protected] mari]# grep -v "^#" /etc/my.cnf |grep -v "
[client]
port        = 3306
socket      = /tmp/mysql.sock
[mysqld]
port        = 3306
socket      = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 256M
max_allowed_packet = 1M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
datadir=/mariadb/data
log-bin=/mariadb/logs/mysql-bin
binlog_format=mixed
server-id   = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
[[email protected] mari]# cp support-files/mysql.server /etc/rc.d/init.d/mari
[[email protected] mari]# chmod +x /etc/rc.d/init.d/mari
[[email protected] mari]# chkconfig --add mari
[[email protected] mari]# chkconfig mari off
  • 初始化数据库,并登录测试
[[email protected] mari]# scripts/mysql_install_db --user=mysql --datadir=/mariadb/data/
Installing MariaDB/MySQL system tables in ‘/mariadb/data/‘ ...
140501 11:22:15 [Note] InnoDB: The InnoDB memory heap is disabled
140501 11:22:15 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
140501 11:22:15 [Note] InnoDB: Compressed tables use zlib 1.2.3
........省略中........
Please report any problems at http://mariadb.org/jira
The latest information about MariaDB is available at http://mariadb.org/.
You can find additional information about the MySQL part at:
http://dev.mysql.com
Support MariaDB development by buying support/new features from
SkySQL Ab. You can contact us about this at [email protected]com.
Alternatively consider joining our community based development effort:
http://mariadb.com/kb/en/contributing-to-the-mariadb-project/
[[email protected] local]# service mari start
Starting MySQL........                                     [  OK  ]
[[email protected] local]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.00 sec)
MariaDB [(none)]>
  • 到另一个节点测试一下，先停止服务，卸载、降级
[[email protected] local]# service mari stop
Shutting down MySQL..                                      [  OK  ]
[[email protected] local]# umount /mariadb/
[[email protected] local]# drbdadm secondary dbresource
[[email protected] local]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[[email protected] local]# scp /etc/my.cnf essun.node1.com:/etc/
my.cnf                                                               100% 4936     4.8KB/s   00:00
[[email protected] local]# scp -p /etc/rc.d/init.d/mari essun.node1.com:/etc/rc.d/init.d/
mari                                                                 100%   11KB  11.4KB/s   00:00
[[email protected] local]#
==============以下是另一个节点的操作
[[email protected] ~]# mkdir /mariadb
[[email protected] ~]# drbdadm primary dbresource
[[email protected] ~]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----
[[email protected] ~]# chown -R mysql.mysql /mariadb/
[[email protected] ~]# mount /dev/drbd0 /mariadb/
[[email protected] ~]# cd /mariadb/
[[email protected] mariadb]# ll
total 24
drwxr-xr-x 5 mysq mysql  4096 May  1 11:29 data
drwxr-xr-x 2 mysq mysql  4096 May  1 11:29 logs
drwx------ 2 mysq mysql 16384 May  1 09:42 lost+found
[[email protected] /]# service mari start
Starting MySQL.......                                      [  OK  ]
[[email protected] /]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.03 sec)
MariaDB [(none)]>
  • 两个节点都要停止服务
[[email protected] /]# service mari stop
Shutting down MySQL..                                      [  OK  ]
[[email protected] /]# chkconfig mari off
[[email protected] /]# chkconfig --list mari
mari            0:off   1:off   2:off   3:off   4:off   5:off   6:off
[[email protected] /]# service drbd stop
Stopping all DRBD resources:
.
[[email protected] /]# chkconfig drbd off
[[email protected] /]# chkconfig --list drbd
drbd            0:off   1:off   2:off   3:off   4:off   5:off   6:off

第三步、安装corosync、pacemaker

[[email protected] download]# yum install corosync pacemaker -y
[[email protected] download]# yum install crmsh-1.2.6-4.el6.x86_64.rpm pssh-2.3.1-2.el6.x86_64.rpm -y
  • 修改配置文件，生成密钥
[[email protected] /]# cd /etc/corosync/
[[email protected] corosync]# ls
corosync.conf.example       service.d
corosync.conf.example.udpu  uidgid.d
[[email protected] corosync]# cp corosync.conf.example corosync.conf
[[email protected] corosync]# vim corosync.conf
[[email protected] corosync]# cat corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank
totem {
    version: 2
    secauth: off
    threads: 0
    interface {
        ringnumber: 0
        bindnetaddr: 192.168.1.1
        mcastaddr: 226.94.1.1
        mcastport: 5405
        ttl: 1
    }
}
logging {
    fileline: off
    to_stderr: no
    to_logfile: yes
    to_syslog: no
    logfile: /var/log/cluster/corosync.log
    debug: off
    timestamp: on
    logger_subsys {
        subsys: AMF
        debug: off
    }
}
amf {
    mode: disabled
}
service {
        name:   pacemaker
        ver:    0
}
aisexce {
        user:   root
        group:  root
}
[[email protected] corosync]# mv /dev/random /dev/x
[[email protected] corosync]# mv /dev/urandom /dev/random
[[email protected] corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.
[[email protected] corosync]# mv /dev/random /dev/urandom
[[email protected] corosync]# mv /dev/x /dev/random
  • 将这两个文件复制到另一个节点上一份
[[email protected] corosync]# scp authkey corosync.conf essun.node1.com:/etc/corosync/
authkey                100%  128     0.1KB/s   00:00
corosync.conf          100%  554     0.5KB/s   00:00
[[email protected] corosync]#
  • 启动服务（两个节点都要启动服务）
[[email protected] corosync]# service corosync start
Starting Corosync Cluster Engine (corosync):            [  OK  ]
[[email protected] corosync]# crm status
Last updated: Thu May  1 10:05:25 2014
Last change: Thu May  1 10:05:21 2014 via crmd on essun.node2.com
Stack: classic openais (with plugin)
Current DC: essun.node2.com - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
2 Nodes configured, 2 expected votes
0 Resources configured
Online: [ essun.node1.com essun.node2.com ]
[[email protected] corosync]#
  • 禁用stonith裁决，忽略投票规则
crm(live)configure# property stonith-enabled=false
crm(live)configure# property no-quorum-policy=ignore
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node essun.node1.com
node essun.node2.com
property $id="cib-bootstrap-options"     dc-version="1.1.10-14.el6_5.3-368c726"     cluster-infrastructure="classic openais (with plugin)"     expected-quorum-votes="2"     stonith-enabled="false"     no-quorum-policy="ignore"
  • 定义VIP，mariadb服务
crm(live)configure# primitive vip ocf:heartbeat:IPaddr params ip=192.168.1.150 op monitor interval=20s timeout=20s on-fail=restart
crm(live)configure# verify
crm(live)configure# primitive myserver lsb:mari op monitor interval=20s timeout=20s on-fail=restart
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
crm(live)configure# show
node essun.node1.com     attributes standby="off"
node essun.node2.com
primitive myserver lsb:mari     op monitor interval="20s" timeout="20s" on-fail="restart"
primitive vip ocf:heartbeat:IPaddr     params ip="192.168.1.150"     op monitor interval="20s" timeout="20s" on-fail="restart"
property $id="cib-bootstrap-options"     dc-version="1.1.10-14.el6_5.3-368c726"     cluster-infrastructure="classic openais (with plugin)"     expected-quorum-votes="2"     stonith-enabled="false"     no-quorum-policy="ignore"
  • 配置drdb集群资源
crm(live)configure# primitive dbdrbd ocf:linbit:drbd params drbd_resource=dbresource op monitor role=Master interval=50s timeout=30s op monitor role=Slave interval=60s timeout=30s op start timeout=240s interval=0 op stop timeout=100s interval=0
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node essun.node1.com         attributes standby="off"
node essun.node2.com
node essun.node1.com         attributes standby="off"
node essun.node2.com
primitive dbdrbd ocf:linbit:drbd         params drbd_resource="dbresource"         op monitor role="Master" interval="50s" timeout="30s"         op monitor role="Slave" interval="60s" timeout="30s"         op start timeout="240s" interval="0"         op stop timeout="100s" interval="0"
primitive myserver lsb:mari         op monitor interval="20s" timeout="20s" on-fail="restart"
primitive vip ocf:heartbeat:IPaddr         params ip="192.168.1.150"         op monitor interval="20" .168.1.150         op monitor interval="20s" timeout="20s" on-fail="restart"
property $id="cib-bootstrap-options"         dc-version="1.1.10-14.el6_5.3-368c726"         cluster-infrastructure="classic openais (with plugin)"         expected-quorum-votes="2"         stonith-enabled="false"         no-quorum-policy="ignore"

drbd需要同时运行在两个节点上，但只能有一个节点（primary/secondary模型）是Master，而另一个节点为Slave；因此，它是一种比较特殊的集群资源，其资源类型为多态（Multi-state）clone类型，即主机节点有Master和Slave之分，且要求服务刚启动时两个节点都处于slave状态，一般来说，只要是定义克隆资源，首先其本身必须是主资源

crm(live)configure# ms ms_dbdrdb dbdrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show ms_dbdrdb
ms ms_dbdrdb dbdrbd     meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
  • 到node1验证一下，看看是不是变成了主资源
[[email protected] /]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:bf:c2:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.109/24 brd 255.255.255.255 scope global eth0
    inet 192.168.1.150/24 brd 255.255.255.255 scope global secondary eth0
    inet6 fe80::20c:29ff:febf:c282/64 scope link tentative dadfailed
       valid_lft forever preferred_lft forever
  • 定义文件系统资源（以便挂载DRBD的主资源，此处挂载的是mariadb的数据目录)
crm(live)configure# primitive dbstore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/mariadb fstype=ext4 op monitor interval40s timeout=40s op start timeout=60s interval=0 op stop timeout=60 interval=0
crm(live)configure# verify
crm(live)configure# commit
WARNING: CIB changed in the meantime: won‘t touch it!
Do you still want to commit? y
crm(live)configure# show dbstore
primitive dbstore ocf:heartbeat:Filesystem     params device="/dev/drbd0" directory="/mariadb" fstype="ext4"     op monitor timeout="40s" interval="0" interval40s     op start timeout="60s" interval="0"     op stop timeout="60" interval="0
  • 结合drbd的特性，只能在drbd服务将某节点设置为Primary以后方可启动。因此，还需要为这两个资源建立排列约束和顺序约束。
colocation vip_with_myserver inf: vip myserver
crm(live)configure# colocation myserver_with_dbststore inf: myserver
dbstore
crm(live)configure# colocation dbstore_with_ms_dbdrbd inf: dbstore ms_dbdrdb:Master
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# order ms_dbdrbd_before_dbstore inf: ms_dbdrdb:promote dbstore:start
crm(live)configure# verify
crm(live)configure# order vip_before_myserver inf: vip myserver
crm(live)configure# verify
crm(live)configure# order dbstore_before_myserver inf: dbstore:start myserver:start
crm(live)configure# verify
crm(live)configure# commit
crm(live)# status
Last updated: Thu May  1 15:29:04 2014
Last change: Thu May  1 15:28:55 2014 via crmd on essun.node2.com
Stack: classic openais (with plugin)
Current DC: essun.node2.com - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
2 Nodes configured, 2 expected votes
5 Resources configured
Online: [ essun.node1.com essun.node2.com ]
 vip    (ocf::heartbeat:IPaddr):        Started essun.node1.com
 myserver       (lsb:mari):     Started essun.node1.com
 Master/Slave Set: ms_dbdrdb [dbdrbd]
     Masters: [ essun.node1.com ]
     Slaves: [ essun.node2.com ]
 dbstore        (ocf::heartbeat:Filesystem):    Started essun.node1.com
  • 到节点node1上看一下maridb下是否己经挂载DRBD的主资源
[[email protected] mariadb]# hostname
essun.node1.com
[[email protected] mariadb]# ls
data  logs  lost+found

这己表示在DRBD己经挂载到node1的mariadb目下

第四步、测试高可用mariadb

  • 授权测试用户
[[email protected] /]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.
MariaDB [(none)]> grant all on *.* to [email protected]‘192.168.1.%‘ identified by ‘mari‘;
Query OK, 0 rows affected (0.00 sec)
  • 使用vip登录测试
[[email protected] mariadb]# hostname
essun.node2.com
[[email protected] mariadb]# /usr/local/mysql/bin/mysql -uroot -p -h192.168.1.150
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.08 sec)
  • 模拟节点node1故障
  • crm(live)node# standby essun.node1.com
[[email protected] ~]# crm status
Last updated: Thu May  1 16:28:59 2014
Last change: Thu May  1 16:27:34 2014 via crmd on essun.node2.com
Stack: classic openais (with plugin)
Current DC: essun.node1.com - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
2 Nodes configured, 2 expected votes
5 Resources configured
Node essun.node1.com: standby
Online: [ essun.node2.com ]
 vip    (ocf::heartbeat:IPaddr):    Started essun.node2.com
 myserver   (lsb:mari): Started essun.node2.com
 Master/Slave Set: ms_dbdrdb [dbdrbd]
     Masters: [ essun.node2.com ]
     Stopped: [ essun.node1.com ]
 dbstore    (ocf::heartbeat:Filesystem):    Started essun.node2.com

    再次使用vip登录测试

[[email protected] mariadb]# hostname
essun.node1.com
[[email protected] mariadb]# /usr/local/mysql/bin/mysql -uroot -p -h192.168.1.150
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (1.28 sec)

===================================完====================================================

DRDB实现mariadb高可用,布布扣,bubuko.com

时间： 2024-10-10 13:38:32

DRDB实现mariadb高可用的相关文章

corosync+drbd实现mariadb高可用集群

配置前的准备 (1)节点间时间必须同步:使用ntp协议实现: [[email protected] ~]# ntpdate 172.16.0.1       #172.16.0.1是我这里的时间服务器 [[email protected] ~]# ntpdate 172.16.0.1 (2) 节点间需要通过主机名互相通信,必须解析主机至IP地址: (a)建议名称解析功能使用hosts文件来实现: (b)通信中使用的名字与节点名字必须保持一致:"uname -n"命令,或"ho

MariaDB-Galera-cluster实现mariadb高可用的详细步骤

MariaDB的Galera-cluster实现高可用的详细步骤 ? A. 实验环境说明 ? a) 至少3台centos7主机 ? b) 角色: a. master1:192.168.36.121 b. master2:192.168.36.120 c. master3:192.168.36.27 ? B. 配置yum源 vim /etc/yum.repos.d/galera.repo [mysql] baseurl=https://mirrors.tuna.tsinghua.edu.cn/ma

heartbeat+drdb+nfs实现高可用

一.环境 nfsserver01:192.168.127.100 centos7.3 nfsserver02:192.168.127.102 centos7.3 nfsclient01:192.168.127.101 centos7.3 二.原理 1.DRDB,分布式复制块设备(Distributed Replicated Block Device),基于linux的软件组件:primary负责接受写入的数据,并且把写入的数据发送给Secondary端. 支持底层设备:磁盘/分区.LVM逻辑卷.

部署mariadb高可用

上传文件到opt目录下 Galera能够实现MySQL/MariaDB数据库的主主复制和多主复制等模式,这些复制模式都是同步进行的,同步时间非常短. 每一个节点都可以同时写入和读取,当某一节点发生故障时,可自动从集群中自动剔除. HAProxy能提供负载均衡和故障判断等功能解决服务器系统存在的单点故障. Keepalived能提供客户端连接数据库时使用的虚拟IP地址(VIP). (1)主节点 IP地址:node-1 192.168.1.74 (2)备份节点 IP地址:node-2 192.168

MariaDB高可用架构之MHA

MHA(Master High Availability) 该软件由两部分组成:MHA Manager(管理节点)和MHA Node(数据节点).MHA Manager可以单独部署在一台独立的机器上管理多个master-slave集群,也可以部署在一台slave节点上.MHA Node运行在每台MySQL服务器上,MHA Manager会定时探测集群中的master节点,当master出现故障时,它可以自动将最新数据的slave提升为新的master,然后将所有其他的slave重新指向新的mas

corosync+pacemaker实现高可用的MariaDB

一.准备工作 承接上文:corosync+pacemaker使用crmsh构建高可用集群 二.MariaDB server配置(基于nfs) nfs server准备 1.在nfs server准备LVM存储空间 [[email protected] ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS dis

corosync和pacemaker高可用mariadb和haproxy

高可用的解决方案keepalived只是提供了最简单的高可用功能,真正高级的功能keepalived很难完成.openAIS规范提供完善的解决方案,但是很重量级很多功能考虑的很全面.很细致,了解这些我们才可以更加深入的理解高可用的完整的体系,当遇到特殊的高可用场景我们必须使用这些方案才可以解决. OpenAIS规范的解决方案 这个规范一直迭代到今天,形成的完整的体系如图1.1 图1.1 既然多个主机要组成一个集群,那么就要有一个软件帮助多个主机间实现心跳信息通告,这个实现通告的在OpenAIS规

corosync+pacemaker简单实践实现mariadb数据库高可用模型

友情提醒:文中实验环境 vmware10 + Centos 6.6 X86_64,文中命令请谨慎使用 一  实验环境和拓扑 主机 IP地址 角色 Test01.lijun.com eth1:192.168.100.1/24 后台NFS共享服务器 Test04.lijun.com eth1:192.168.100.4/24 eth0:172.16.34.4/16 集群节点 Test03.lijun.com eth1:192.168.100.3/24 eth0:172.16.34.3/16 集群节点

图形界面配置heartbeat高可用集群

修改ha.cf配置文件,启用crm功能,表示资源配置交给crm管理 [[email protected] ~]# vim /etc/ha.d/ha.cf crm on 使用自带的ha_propagate脚本同步配置文件 [[email protected] ~]# /usr/lib/heartbeat/ha_propagate 安装heartbeat-gui包 [[email protected] heartbeat2]# rpm -ivhheartbeat-gui-2.1.4-12.el6.x