corosync+pacemaker+crm实现drbd高可用

系统：Centos 7.2 内核3.10.0-327.el7.x86_64      # 内核需要高于2.6.32

Drbd : 192.168.8.111：node1/dev/drdb0 /mydeta

192.168.8.112 : node2

Mysql_vip: 192.168.8.200     #下章实现

# 需要的软件包：

mariadb-5.5.53-linux-i686.tar.gz

drbd84-utils-8.9.1-1.el7.elrepo.x86_64.rpm

kmod-drbd84-8.4.6-1.el7.elrepo.x86_64.rpm

yum 需要安装的软件 pcs, ansible, crmsh, libxslt

# drbd是基于主机名通信的在/etc/hosts中增加

# 以本地ip地址如192.168.8.111 node1这样的方式添加一个地址一行、 127.0.0.1 不要填uname -n的名称

# 安装crmsh yum源如下 [[email protected] ~]# vim /etc/yum.repos.d/ha.repo

[network_ha-clustering_Stable]

name=Stable High Availability/Clustering packages (CentOS_CentOS-7)

type=rpm-md

baseurl=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/

gpgcheck=1

gpgkey=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7//repodata/repomd.xml.key

enabled=1

# 安装crmsh  单台主机安装就行

yum -y install  crmsh libxslt ansible pcs

# 主机互信

[[email protected] ~]# ssh-keygen

[[email protected] ~]# cat .ssh/id_rsa.pub > .ssh/authorized_keys

[[email protected] ~]# chmod go= .ssh/authorized_keys

[[email protected] ~]# scp -p .ssh/id_rsa.pub .ssh/authorized_keys 192.168.8.112:/root/.ssh/

# 出现以下错误

scp: /root/.ssh/: No such file or directory

# 解决方案

连接node2创建.ssh   mkdir /root/.ssh

# drbd配置段

# 安装drbd前先准备两块硬盘    node1 跟node2 同样配置

rpm -ivh http://elrepo.reloumirrors.net/elrepo/el7/x86_64/RPMS/drbd84-utils-8.9.1-1.el7.elrepo.x86_64.rpm

yum -y install http://elrepo.reloumirrors.net/elrepo/el7/x86_64/RPMS/kmod-drbd84-8.4.6-1.el7.elrepo.x86_64.rpm

[[email protected] drbd-8.4.8-1]# modprobe drbd

[[email protected] drbd-8.4.8-1]# lsmod | grep drbd

drbd                  405537  0

libcrc32c              12644  2 xfs,drbd

# 将新建的磁盘分区

[[email protected] sbin]# fdisk /dev/sdb# 自己新建就成、两边大小保持一致跟raid1一样

# 配置drbd

[[email protected] sbin]#

# drbd配置: http://502245466.blog.51cto.com/7559397/1298945

# drbd常见错误: http://www.j3j5.com/post-128.html

[[email protected] drbd.d]# vim global_common.conf

global {

usage-count no;

}

common {

protocol C;

handlers {

pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";

pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";

local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";

}

startup {

}

options {

}

disk {

on-io-error detach;

}

net {

cram-hmac-alg "sha1";

shared-secret "xiong123";

}

syncer {

rate 1024M;

}

}

[[email protected] drbd.d]# vim mydrbd.res

resource mydrbd {

device /dev/drbd0;

disk  /dev/sdb1;

meta-disk internal;

on node1{on为开启，node1为节点名称

address 192.168.8.111:7789;

}

on node2{

address 192.168.8.112:7789;

}

}

[[email protected] drbd.d]# scp mydrbd.res global_common.conf node2:/usr/local/drbd/etc/drbd.d/

############测试drbd是否成功######

[[email protected] drbd.d]# drbdadm create-md mydrbd

initializing activity log

NOT initializing bitmap

Writing meta data...

New drbd meta data block successfully created

# 配置启动否则corosync ocf无法直接定义drbd启动

[[email protected] sbin]# systemctl enable drbd.service

# 此处为测试drbd是否完整  两边都得同时启动

[[email protected] sbin]# systemctl start drbd.service

[[email protected] drbd.d]# drbdadm primary --force mydrbd # 将主机强制成为主节点

[[email protected] drbd.d]# drbd-overview

0:mydrbd/0  Connected Primary/Secondary UpToDate/UpToDate# Primary/Secondary 主从，另一端是Secondary/Primary

#格式磁盘

[[email protected] drbd.d]# mkfs.ext4 /dev/drbd0

# 挂载     两个节点都需要创建 /mydata/

[[email protected] drbd.d]# mount /dev/drbd0 /mydata/

[[email protected] drbd.d]# df -Th

/dev/drbd0              ext4      4.8G   20M  4.6G    1% /mydata

复制一个文件至/mydata目录中，并将主drbd设置为从

[[email protected] ~]# umount /mydata/

[[email protected] ~]# drbdadm secondary mydrbd

[[email protected] ~]# drbd-overview

0:mydrbd/0  Connected Secondary/Secondary UpToDate/Diskless

# 节点2操作配置 无需格式化/dev/drbd0

[[email protected] drbd.d]# drbdadm primary mydrbd

[[email protected] drbd.d]# mkdir /mydata

[[email protected] drbd.d]# mount /dev/drbd0 /mydata/

[[email protected] drbd.d]# cd /mydata/

[[email protected] mydata]# ls

issue  lost+found

# 最后将drbd服务都停止

[[email protected] /]# umount /mydata/

[[email protected] /]# systemctl  stop drbd

[[email protected] /]# systemctl  stop drbd

############################测试结束########################################

# pcs配置drbd高可用

# 安装

# 安装pcs软件如果报/sbin/xxx命令的名字将crmsh.repo先改成crmsh.repo.bak 再安装

[[email protected] sbin]# ansible ha -m yum -a "name=pcs state=installed"

# pcs认证需要

[[email protected] sbin]# ansible ha -m shell -a ‘echo "xiong" | passwd --stdin hacluster‘

# 启动pcs服务

[[email protected] sbin]# ansible ha -m service -a "name=pcsd state=started enabled=yes"

# 认证集群服务

[[email protected] sbin]# pcs cluster auth node1 node2

Username: hacluster

Password:

node1: Authorized

node2: Authorized

# 添加一个集群名为myha 并设置组播地址为235.35.32.24

[[email protected] sbin]# pcs cluster setup --name myha node1 node2 --mcast0 235.35.32.24

# 检查集群服务是否正常

[[email protected] sbin]# corosync-cmapctl | grep mem

runtime.totem.pg.mrp.srp.memb_commit_token_rx (u64) = 4

runtime.totem.pg.mrp.srp.memb_commit_token_tx (u64) = 4

runtime.totem.pg.mrp.srp.memb_join_rx (u64) = 5

runtime.totem.pg.mrp.srp.memb_join_tx (u64) = 3

runtime.totem.pg.mrp.srp.memb_merge_detect_rx (u64) = 332

runtime.totem.pg.mrp.srp.memb_merge_detect_tx (u64) = 332

runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.8.111)

runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.1.status (str) = joined

runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0

runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.8.112)

runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1

runtime.totem.pg.mrp.srp.members.2.status (str) = joined

# 检查状态

[[email protected] sbin]# pcs status

Cluster name: myha

WARNING: no stonith devices and stonith-enabled is not false

Stack: corosync

Current DC: node1 (version 1.1.13-10.el7-44eb2dd) - partition with quorum

2 nodes and 0 resources configured

Online: [ node1 node2 ]

# 配置高可用

crm(live)configure# primitive mydrbd ocf:linbit:drbd params drbd_resource="mystore" op monitor role="Master" interval=30s timeout=30s op monitor role="Slave" timeout=30s op start timeout=240s op stop timeout=100s

####################正确模板##########################################

primitive mydatas ocf:linbit:drbd \

params drbd_resource=mystore \

op monitor role=Master interval=10s timeout=20s \

op monitor role=Slave interval=20s timeout=20s \

op start timeout=240s interval=0 \

op stop timeout=100s interval=0

######################################################################

crm(live)configure# ms mydrbd_master mydrbd meta clone-max=2 clone-node-max=1 master-max=1 master-node-max=1

crm(live)configure#  verify     #检查是否成功

crm(live)# status

Online: [ node73 node74 ]

Master/Slave Set: ms_mydatas [mydatas]

Masters: [ node73 ]

Slaves: [ node74 ]

# 配置文件挂载目录

crm(live)configure# primitive mount ocf:heartbeat:Filesystem params device="/dev/drbd0" directory="/mydata" fstype="ext4" op start timeout=60s op stop timeout=60s

# 定义排序约束 让mount一直跟着ms_mydatas的主

crm(live)configure# colocation mount_with_ms_mydatas inf: mount ms_mydatas:Master

# 检查是否成功

[[email protected] ~]# df -Th

/dev/drbd0              ext4       16G   45M   15G    1% /mydata

[[email protected] ~]# drbd-overview

0:mystore/0  Connected Primary/Secondary UpToDate/UpToDate /mydata ext4 16G 45M 15G 1%

####################### drbd主从配置文件###############################

primitive mount Filesystem \

params device="/dev/drbd0" directory="/mydata" fstype=ext4 \

op start timeout=60s interval=0 \

op stop timeout=60s interval=0

primitive mydatas ocf:linbit:drbd \

params drbd_resource=mystore \

op monitor role=Master interval=10s timeout=20s \

op monitor role=Slave interval=20s timeout=20s \

op start timeout=240s interval=0 \

op stop timeout=100s interval=0

ms ms_mydatas mydatas \

meta clone-max=2 clone-node-max=1 master-max=1 master-node-max=1 notify=true

colocation mount_with_ms_mydatas inf: mount ms_mydatas:Master

#########################################################################

# 错误整理

1、ERROR: error: unpack_resources:Resource start-up disabled since no STONITH resources have been defined

error: unpack_resources:Either configure some or disable STONITH with the stonith-enabled option

error: unpack_resources:NOTE: Clusters with shared data need STONITH to ensure data integrity

解决：crm(live)configure# property stonith-enabled=false

# 错误

[[email protected] drbd.d]# drbdadm primary mystore

0: State change failed: (-2) Need access to UpToDate data

Command ‘drbdsetup-84 primary 0‘ terminated with exit code 17

# 解决方案

[[email protected] drbd.d]# drbdadm primary mystore --force

# 错误

[[email protected] drbd.d]# pcs status

Error: cluster is not currently running on this node

# 解决方案

[[email protected] drbd.d]# pcs cluster start --all

# 启动主从失败

Failed Actions:

* mydrbd_stop_0 on node74 ‘not configured‘ (6): call=8, status=complete, exitreason=‘none‘,

last-rc-change=‘Wed May 24 15:27:42 2017‘, queued=0ms, exec=138ms

* mydrbd_stop_0 on node73 ‘not configured‘ (6): call=8, status=complete, exitreason=‘none‘,

last-rc-change=‘Wed May 24 15:27:41 2017‘, queued=0ms, exec=134ms

# 解决方案

设置各种超时时长都加上"s"  primitive mydrbd ocf:linbit:drbd params drbd_resource="mystore" op monitor role="Master" interval=30s timeout=30s op monitor role="Slave" timeout=30s op start timeout=240s op stop timeout=100s

drbd yum包下载地址：http://elrepo.reloumirrors.net/elrepo/el7/x86_64/RPMS/

配置参阅链接： http://blog.csdn.net/qq_21127313/article/details/68210977

时间： 2024-08-26 21:59:44

corosync+pacemaker+crm实现drbd高可用的相关文章

corosync+pacemaker使用pcs构建高可用集群

一.corosync+pacemaker集群前提准备 集群前提准备 --> HA-web 承接上文 --> corosync+pacemaker使用crmsh构建高可用集群 二.准备pcs [[email protected] ~]# yum install pcs 禁用stonith设备 [[email protected] ~]# pcs property set stonith-enable=false [[email protected] ~]# pcs property set no

corosync+pacemaker+nfs提供mysql高可用

corosync/openais+pacemaker+nfs提供mariadb高可用 节点一, one, 172.16.249.122/16, 172.16.0.1, CentOS6.6, mariadb节点二, two, 172.16.249.123/16, 172.16.0.1, CentOS6.6, mariadb节点三, three, 172.16.249.124/16, 172.16.0.1, CentOS6.6, NFSVIP: 172.16.249.121 节点一: one 172

corosync + pacemaker实现httpd服务高可用集群

实验环境: 服务器: 192.168.145.208(集群节点一) 192.168.145.209(集群节点二) 192.168.145.210(NFS server) 操作系统:三台都为centos7 配置步骤: 在两个集群节点中分别进行无密钥通讯配置: ssh-keygen -t rsa -P "" ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected] 关闭各个节点的防火墙: systemctl stop firewalld 编辑co

corosync+pacemaker实现web服务高可用

前提: 本配置共有两个测试节点,分别node1和node2,相的IP地址分别为202.207.178.6和202.207.178.7 (为避免影响,先关闭防火墙和SElinux) 一.安装配置corosync及相关软件包 1.准备工作 1)节点名称必须跟uname -n命令的执行结果一致 node1: # hostname node1 # vim /etc/sysconfig/network HOSTNAME=node1 node2: # hostname node2 # vim /etc/sy

HA集群之DRBD浅析及实现DRBD高可用

一.DRBD概述 DRBD (Distributed Replicated Block Device) 是 Linux 平台上的分散式储存系统. 由内核模组.用户空间工具组成,通常用于高可用性(high availability, HA)集群.其实现方式是通过网络在服务器之间的对块设备(硬盘,分区,逻辑卷等)进行镜像.你可以把它看作是一种网络RAID1. DRBD的工作模式如下图: DRBD工作的位置在文件系统的buffer cache和磁盘调度器之间,数据进入Buffer Cache后,DRB

Corosync部署MySQL+DRBD高可用服务

介绍篇 高可用集群主要是有两个或者多个节点进行工作,ha基本组成部分包括四个部分:1.位于最底层的信息和基础架构层(Messaging Layer),主要用于节点之间传递心跳信息,故也称为心跳层.节点之间传递心跳信息可以通过广播,组播,单播等方式.2.第二层为成员关系(Membership)层,这层最重要的作用是主节点通过cluster consensus menbership service(CCM或者CCS)这种服务由第一层提供的信息,来生产一个完整的成员关系.这层主要实现承上启下的作用,承

corosync 实现 MariaDB 数据库服务的高可用

方案: corosync + pacemaker  提供HA的解决方案. 使用NFS共享服务器导出的共享文件系统做为数据目录: 拓扑图如下: 一.配置HA高可用集群的各节点间能够基于主机名通讯. 1.设置主机名 (1).在 192.168.60.128 主机设置 编辑/etc/sysconfig/network文件,使得主机名永久有效 [[email protected] ~]# vim /etc/sysconfig/network HOSTNAME=node2.9527du.com 使用hos

heartheat+drbd高可用存储

DRBD数据同步 By:大官人 一.DRBD简介 DRBD的全称为:DistributedReplicatedBlock Device(DRBD)分布式块设备复制,DRBD是由内核模块和相关脚本而构成,用以构建高可用性的集群.其实现方式是通过网络来镜像整个设备.你可以把它看作是一种网络RAID.它允许用户在远程机器上建立一个本地块设备的实时镜像. 二.DRBD是如何工作的呢? (DRBDPrimary)负责接收数据,把数据写到本地磁盘并发送给另一台主机(DRBD Secondary).另一个主机

CentOS 6.5环境实现corosync+pacemaker实现DRBD高可用

DRBD (Distributed Replicated Block Device)分布式复制块设备,它是 Linux 平台上的分散式储存系统,通常用于高可用性(high availability, HA)集群中.DRBD 类似磁盘阵列的RAID 1(镜像),只不过 RAID 1 一般配置在一台服务器内,而 DRBD 是通过网络. DRBD Resource:DRBD所具有的几种属性: resource name:可以使用除空白字符外的任意ACSII表中的字符: drbd设备:drbd的设备的访