DRBD 安装及配置
DRBD主：10.1.1.5
DRBD备：10.1.1.6
1.下载源码包：
地址：www.drbd.org
2.安装：
2.1 安装依赖:
yum install kernel-devel kernel-headers
2.2.编译安装
tar -zxf drbd-8.3.16.tar.gz
./configure --prefix=/usr/local/drbd --with-km
make && make install
2.3 拷贝配置文件模板
mv /usr/local/drbd/etc/drbd.conf /usr/local/drbd/drbd.conf.old
cp /tmp/soft/drbd-8.3.16/scripts/drbd.conf.exampl /usr/local/drbd/etc/drbd.conf
2.4 设置链接
ln -s /usr/local/drbd/etc/drbd.conf /etc/drbd.conf
ln -s /usr/local/drbd/etc/rc.d/init.d/drbd /etc/init.d/drbd
3.DRBD 配置文件（主备配置完全一致）：
3.1 配置文件解析：
# vi /etc/drbd.conf
global {
usage-count no; #是否加入DRBD 使用统计
}
common {
syncer { rate 200M; } #设置传输速率
}
#resource r0 设置资源名称
resource r0 {
protocol C; #设置协议C，写入完成才认为同步完成，最安全。
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
fence-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";
pri-lost "/usr/lib/drbd/notify-pri-lost.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
split-brain "/usr/lib/drbd/notify-split-brain.sh root";
out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
}
startup {
degr-wfc-timeout 120; #超时时间 120秒
outdated-wfc-timeout 2; #超时时间 2分钟
}
disk {
on-io-error detach;
fencing resource-only;
#使用dpod 功能，(drbd outdate-peer daemon)保证数据不同步时不进行切换
}
net {
cram-hmac-alg "sha1"; #使用的加密方式
shared-secret "ckl893";#使用加密密码
}
on ckl-bd1 { #设置主机名称
device /dev/drbd0; #DRBD名称
disk /dev/sda5; #共享的磁盘分区
address 10.1.1.5:7788; #主机地址
flexible-meta-disk internal; #DRBD 元数据存放方式
}
on ckl-bd2 {
device /dev/drbd0;
disk /dev/sda5;
address 10.1.1.6:7788;
meta-disk internal;
}
}
3.2 创建DRBD数据块
在启动DRBD之前,你需要分别在两台主机的分区上,创建供DRBD记录信息的数据块.分别在两台主机上执行:
# drbdadm create-md r0 或者执行drbdadm create-md all
# drbdadm create-md r0
md_offset 40275800064
al_offset 40275767296
bm_offset 40274534400
Found ext3 filesystem
39331840 kB data area apparently used
39330600 kB left usable by current configuration
Device size would be truncated, which
would corrupt data and result in
‘access beyond end of device‘ errors.
You need to either
* use external meta data (recommended)
* shrink that filesystem first
* zero out the device (destroy the filesystem)
Operation refused.
Command ‘drbdmeta 0 v08 /dev/sda5 internal create-md‘ terminated with exit code 40
drbdadm create-md r0: exited with code 40
报错：drbdadm create-md r0: exited with code 40
解决办法：初始化磁盘文件格式, dd if=/dev/zero bs=1M count=1 of=/dev/sdXYZ; sync
在1.5 和 1.6 上分别执行
#dd if=/dev/zero bs=1M count=1 of=/dev/sda5;sync
#drbdadm create-md r0
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
4.启动服务
4.1 启动：
# /etc/init.d/drbd start
Starting DRBD resources: [ d(r0) 0: Failure: (104) Can not open backing device.
[r0] cmd /sbin/drbdsetup 0 disk /dev/sda5 /dev/sda5 internal --set-defaults --create-device --on-io-error=detach --fencing=resource-only failed - continuing!
s(r0) n(r0) ]..........
***************************************************************
DRBD‘s startup script waits for the peer node(s) to appear.
- In case this node was already a degraded cluster before the
reboot the timeout is 120 seconds. [degr-wfc-timeout]
- If the peer was available before the reboot the timeout will
expire after 0 seconds. [wfc-timeout]
(These values are for resource ‘r0‘; 0 sec -> wait forever)
To abort waiting enter ‘yes‘ [ 17]:yes
启动报错：Starting DRBD resources: [ d(r0) 0: Failure: (104) Can not open backing device
解决办法：我们挂载了/data 分区，必须卸载
#umount /data
再次启动：
# /etc/init.d/drdb start
Starting DRBD resources: [ d(r0) s(r0) n(r0) ]..........
***************************************************************
DRBD‘s startup script waits for the peer node(s) to appear.
- In case this node was already a degraded cluster before the
reboot the timeout is 120 seconds. [degr-wfc-timeout]
- If the peer was available before the reboot the timeout will
expire after 0 seconds. [wfc-timeout]
(These values are for resource ‘r0‘; 0 sec -> wait forever)
To abort waiting enter ‘yes‘ [ 15]:yes
5.DRBD 信息说明
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by [email protected], 2014-05-01 18:12:33
0: cs:WFConnection ro:Secondary/Unknown ds:Diskless/DUnknown C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
cs 表示连接状态
ro 表示角色信息，第一次启动drbd时，两个drbd节点默认都处于Secondary状态
ds 磁盘状态信息“Inconsistent/Inconsisten”，即为“不一致/不一致”状态，表示两个节点的磁盘数据处于不一致状态。
ns表示网络发送的数据包信息
nr 网络接收的数据包
dw 磁盘写信息
dr 磁盘读信息
6.设置主节点：
#drbdsetup /dev/drbd1 primary –o
或者执行下面命令：
#drbdadm -- --overwrite-data-of-peer primary all
第一次执行完此命令后，在后面如果需要设置哪个是主节点时，就可以使用另外一个命令：
#/sbin/drbdadm primary r0或者/sbin/drbdadm primary all
# drbdsetup /dev/drbd0 primary -o
再次查看信息（主）：
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by [email protected], 2014-05-01 18:12:33
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:394240 nr:0 dw:0 dr:394904 al:0 bm:22 lo:0 pe:62 ua:0 ap:0 ep:1 wo:f oos:38944296
[>....................] sync‘ed: 1.0% (38028/38408)M
finish: 0:42:03 speed: 15,420 (15,452) K/sec
ro:Primary/Secondary 变为主节点
ds:UpToDate/Inconsistent 不同步状态，现在还在同步中
等待一段时间，同步完成：
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by [email protected], 2014-05-01 18:12:33
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:39330600 nr:0 dw:0 dr:39331264 al:0 bm:2339 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
cs:Connected 状态连接
ds:UpToDate/UpToDate 磁盘状态，一致
7.格式化DRBD
在主格式化DRBD，格式化操作只需要执行一次
# mkfs.ext4 /dev/drbd0
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
2460976 inodes, 9832650 blocks
491632 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
301 block groups
32768 blocks per group, 32768 fragments per group
8176 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 29 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
挂载DRBD：
# mount /dev/drbd0 /mnt
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 20G 1.3G 18G 7% /
tmpfs 499M 0 499M 0% /dev/shm
/dev/sda1 485M 33M 428M 8% /boot
/dev/drbd0 37G 176M 35G 1% /mnt
8.验证数据同步：
8.1 主节点上创建文件：
# ls
lost+found nihao
查看文件内容：
# cat nihao
ni hao drbd
8.2 主节点停止DRBD 服务：
# /etc/init.d/drdb stop
Stopping all DRBD resources:
.
查看分区已经被自动卸载：
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 20G 1.3G 18G 7% /
tmpfs 499M 0 499M 0% /dev/shm
/dev/sda1 485M 33M 428M 8% /boot
8.3. 备用节点执行:
# drbdsetup /dev/drbd0 primary -o
# mount /dev/drbd0 /mnt
# cd /mnt/
# ls
lost+found nihao
查看文件内容：
# cat nihao
ni hao drbd
#查看信息
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by [email protected], 2014-05-01 18:13:20
0: cs:WFConnection ro:Primary/Unknown ds:UpToDate/Outdated C r-----
ns:0 nr:40081660 dw:40081664 dr:1029 al:1 bm:2338 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4
cs:WFConnection 状态不可连接
ro:Primary/Unknown 磁盘信息unknown
在1.5 启动服务：
# /etc/init.d/drdb start
Starting DRBD resources: [ d(r0) s(r0) n(r0) ].
在1.6 查看信息：
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by [email protected], 2014-05-01 18:13:20
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:4 nr:40081660 dw:40081664 dr:1033 al:1 bm:2339 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
9.第二种方式
现在1.6是主,在1.6上执行：
卸载分区：# umount /mnt
执行主节点转换为备节点命令：#drbdadm secondary all
在1.5上执行：
#drbdadm primary all
# mount /dev/drbd0 /mnt