0.引入
corosync作为HA方案中成员管理层(membership layer),负责集群成员管理、通信方式(单播、广播、组播)等功能,pacemaker作为CRM层。在利用corosync+pacemaker 主备模式实践中,遇到一个问题,即脑裂问题。何谓脑裂: 在HA集群中,节点间通过心跳线进行网络通信,一旦心跳网络异常。导致成员互不相认,各自作为集群中的DC,这样资源同时会在主、备两节点启动。脑裂是corosync还是pacemaker导致的呢?一开始我认为是corosync,原因在于心跳端导致corosync不能正常通信。后来发现在pacemaker官网有找到脑裂(split-brain)的方案。pacemaker作为crm,主责是管理资源,还有一个作用是选择leader。
1.方案
在[1](http://drbd.linbit.com/users-guide-emb/s-configure-split-brain-behavior.html)文中给出了一种解决方法。本文中讨论另一种方法,即为pacemaker配置抢占资源。原理在于,pacemaker 可以定义资源的执行顺序。如果将独占资源放在最前面,后面的资源的启动则依赖与它,成也独占资源,败也独占资源。当心跳网络故障时候,谁先抢占到该资源,该节点就接管服务资源,提供服务。这种方案必须解决两个问题,一是必须定义一个抢占资源,二是自定义pacemaker RA,去抢夺资源。
2. 定义抢占资源
本文利用互斥锁来实现独占资源。具体由python实现一个简单的web服务,提供lock,unlock,updatelock服务。
__author__ = ‘ZHANGTIANJIONG629‘ import BaseHTTPServer import threading import time lock_timeout_seconds = 8 lock = threading.Lock() lock_client_ip = "" lock_time = 0 class LockService(BaseHTTPServer.BaseHTTPRequestHandler): def do_GET(self): ‘‘‘define url route‘‘‘ pass def lock(self, client_ip): global lock_client_ip global lock_time # if lock is free if lock.acquire(): lock_client_ip = client_ip lock_time = time.time() self.send_response(200, ‘ok‘) self.close_connection return # if current client hold lock,updte lock time elif lock_client_ip == client_ip: lock_time = time.time() self.send_response(200, ‘ok,update‘) self.close_connection return else: # lock timeout,grab lock if time.time() - lock_time > lock_timeout_seconds: lock_client_ip = client_ip; lock_time = time.time() self.send_response(200, ‘ok,grab lock‘) self.close_connection return else: self.send_response(403, ‘lock is hold by other‘) self.close_connection def update_lock(self, client_ip): global lock_client_ip global lock_time if lock_client_ip == client_ip: lock_time = time.time() self.send_response(200, ‘ok,update‘) self.close_connection return else: self.send_response(403, ‘lock is hold by other‘) self.close_connection return def unlock(self, client_ip): global lock_client_ip global lock_time if lock.acquire(): lock.release() self.send_response(200, ‘ok,unlock‘) self.close_connection return elif lock_client_ip == client_ip: lock.release() lock_time = 0 lock_client_ip = ‘‘ self.send_response(200, ‘ok,unlock‘) self.close_connection return else: self.send_response(403, ‘lock is hold by other‘) self.close_connection return if __name__ == ‘__main__‘: http_server = BaseHTTPServer.HTTPServer((‘127.0.0.1‘, ‘88888‘), LockService) http_server.serve_forever()
下一篇介绍自定义RA脚本。
时间: 2024-11-05 20:49:03