参考文档
http://www.cnblogs.com/breg/p/5728237.html
开始环境是单节点，存储数据一段时间后发现需要集群高可用环境，幸亏etcd支持在线扩容
1，修改单节点配置并重启etcd
[[email protected] etcd]# cat /etc/etcd/etcd.conf
ETCD_NAME=k8s1
ETCD_DATA_DIR="/data/etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_LISTEN_PEER_URLS="http://172.17.3.20:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.17.3.20:2380"
ETCD_INITIAL_CLUSTER="k8s1=http://172.17.3.20:2380"
备注后三行是新增，后重启etcd
2，注册新节点
注册新节点
[[email protected] etcd]# curl http://127.0.0.1:2379/v2/members -XPOST -H "Content-Type: application/json" -d ‘{"peerURLs": ["http://172.17.3.7:2380"]}‘
{"id":"dd224433fd05e450","name":"","peerURLs":["http://172.17.3.7:2380"],"clientURLs":[]}
注意只注册未启动新节点时集群状态是不健康的
[[email protected] etcd]# curl http://172.17.3.20:2379/v2/members
{"members":[{"id":"869f0c691c5458a3","name":"k8s1","peerURLs":["http://172.17.3.20:2380"],"clientURLs":["http://0.0.0.0:2379"]},
{"id":"dd224433fd05e450","name":"","peerURLs":["http://172.17.3.7:2380"],"clientURLs":[]}]}
[[email protected] etcd]# etcdctl cluster-health
member 869f0c691c5458a3 is unhealthy: got unhealthy result from http://0.0.0.0:2379
member dd224433fd05e450 is unreachable: no available published client urls
cluster is unhealthy
3，启动新节点
[[email protected] data]# cat /etc/etcd/etcd.conf
ETCD_NAME=k8s2
ETCD_DATA_DIR="/data/etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_LISTEN_PEER_URLS="http://172.17.3.7:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.17.3.7:2380"
ETCD_INITIAL_CLUSTER="k8s1=http://172.17.3.20:2380,k8s2=http://172.17.3.7:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
这是新节点配置，后启动新节点
4，检测新节点
[[email protected] etcd]# etcdctl cluster-health
member 869f0c691c5458a3 is healthy: got healthy result from http://0.0.0.0:2379
member dd224433fd05e450 is healthy: got healthy result from http://0.0.0.0:2379
cluster is healthy
5，重复上面操作添加新节点
添加第二个新节点后效果
[[email protected] etcd]# curl http://172.17.3.20:2379/v2/members
{"members":[{"id":"29e27bbd848a2e50","name":"k8s3","peerURLs":["http://172.17.3.8:2380"],"clientURLs":["http://0.0.0.0:2379"]},
{"id":"869f0c691c5458a3","name":"k8s1","peerURLs":["http://172.17.3.20:2380"],"clientURLs":["http://0.0.0.0:2379"]},{"id":"dd224433fd05e450","name":"k8s2","peerURLs":
["http://172.17.3.7:2380"],"clientURLs":["http://0.0.0.0:2379"]}]}
[[email protected] etcd]# etcdctl cluster-health
member 29e27bbd848a2e50 is healthy: got healthy result from http://0.0.0.0:2379
member 869f0c691c5458a3 is healthy: got healthy result from http://0.0.0.0:2379
member dd224433fd05e450 is healthy: got healthy result from http://0.0.0.0:2379
cluster is healthy
6，最后修改所有节点配置为一致
[[email protected] etcd]# cat /etc/etcd/etcd.conf
ETCD_NAME=k8s1
ETCD_DATA_DIR="/data/etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_LISTEN_PEER_URLS="http://172.17.3.20:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.17.3.20:2380"
ETCD_INITIAL_CLUSTER="k8s1=http://172.17.3.20:2380,k8s2=http://172.17.3.7:2380,k8s3=http://172.17.3.8:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
7，更新访问etcd集群参数kube-apiserver与flanneld
KUBE_ETCD_SERVERS="--etcd-servers=http://172.17.3.20:2379,http://172.17.3.7:2379,http://172.17.3.8:2379"
8，集群配置文件备份脚本
[[email protected] etcd_backup]# cat /data/scripts/backupetcd.sh
#!/bin/bash
date_time=`date +%Y%m%d`
etcdctl backup --data-dir /data/etcd/ --backup-dir /data/etcd_backup/${date_time}
find /data/etcd_backup/ -ctime +7 -exec rm -r {} \;
9，故障排查
注意各节点时钟相差过大导致集群建立不起来，所以需要先做时钟同步，默认1s内时差才能成功
注意如果其中有etcd节点启动不起来，可以etcdctl rember delete 后重新添加，删除时清空/data/etcd数据，注意至少要有一份数据保存，这样才能同步到其他节点