配置Etcd集群和TLS认证

由于后续准备在内网开发和测试环境采用二进制方式部署K8S相关组件，并考虑各组件的高可用性和安全性问题,本节介绍etcd服务的集群及tls配置。

一、安装环境介绍

二、Etcd二进制软件包下载地址:
https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz

三、安装与配置etcd组件
1、删除rpm版本的软件包、设置各自的主机名及时间

# yum -y remove etcd
# hostnamectl  set-hostname vm1
# timedatectl set-timezone Asia/Shanghai
# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.115.5 vm1
192.168.115.6 vm2
192.168.115.7 vm3
# ntpdate -u pool.ntp.org

2、关闭防火墙、配置秘钥信任

# systemctl stop firewalled
# systemctl disable firewalled
# ssh-keygen
# ssh-copy-id -i /root/.ssh/id_rsa.pub  [email protected]
# ssh-copy-id -i /root/.ssh/id_rsa.pub  [email protected]
# date && ssh vm2  date  && ssh vm3 date

3、将etcd软件包上传并解压到/usr/local/bin目录

# cd /usr/local/src/
# tar -zxvpf etcd-v3.3.2-linux-amd64.tar.gz
# cp etcd-v3.3.2-linux-amd64/{etcd,etcdctl} /usr/local/sbin/
# chmod +x /usr/local/sbin/etcd*
# scp -rp /usr/local/sbin/etcd* vm2:/usr/local/sbin/
# scp -rp /usr/local/sbin/etcd* vm3:/usr/local/sbin/

4、准备配置文件
Vm1:

# cat /etc/etcd.conf
name: infra0
data-dir: /data/etcd
listen-client-urls: http://192.168.115.5:2379,http://127.0.0.1:2379
advertise-client-urls: http://192.168.115.5:2379,http://127.0.0.1:2379
listen-peer-urls: http://192.168.115.5:2380
initial-advertise-peer-urls: http://192.168.115.5:2380
initial-cluster: infra0=http://192.168.115.5:2380,infra1=http://192.168.115.6:2380,infra2=http://192.168.115.7:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new

Vm2:

# cat /etc/etcd.conf
name: infra1
data-dir: /data/etcd
listen-client-urls: http://192.168.115.6:2379,http://127.0.0.1:2379
advertise-client-urls: http://192.168.115.6:2379,http://127.0.0.1:2379
listen-peer-urls: http://192.168.115.6:2380
initial-advertise-peer-urls: http://192.168.115.6:2380
initial-cluster: infra0=http://192.168.115.5:2380,infra1=http://192.168.115.6:2380,infra2=http://192.168.115.7:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new

VM3:

# cat /etc/etcd.conf
name: infra2
data-dir: /data/etcd
listen-client-urls: http://192.168.115.7:2379,http://127.0.0.1:2379
advertise-client-urls: http://192.168.115.7:2379,http://127.0.0.1:2379
listen-peer-urls: http://192.168.115.7:2380
initial-advertise-peer-urls: http://192.168.115.7:2380
initial-cluster: infra0=http://192.168.115.5:2380,infra1=http://192.168.115.6:2380,infra2=http://192.168.115.7:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new

5、启动etcd集群并测试

# mkdir -p /data/etcd
# nohup etcd --config-file=/etc/etcd.conf &
# export ETCDCTL_API=2
# etcdctl cluster-health
# etcdctl member list
# export ETCDCTL_API=3
# etcdctl --write-out=table --endpoints=192.168.115.5:2379 member list

四、配置etcd tls
1、下载cfssl工具

# mkdir ~/bin
# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
# mv cfssl_linux-amd64 /usr/local/bin/cfssl
# mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
# mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfss-certinfo
# chmod +x /usr/local/bin/cfssl*

2、生成证书

# mkdir ssl
# cd ssl
# cat bulid-key.sh
echo ‘{"CN":"CA","key":{"algo":"rsa","size":2048}}‘ | cfssl gencert -initca - | cfssljson -bare ca -
echo ‘{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","server auth","client auth"]}}}‘ > ca-config.json
export ADDRESS=192.168.115.5,192.168.115.6,192.168.115.7,vm1,vm2,vm3
export NAME=server
echo ‘{"CN":"‘$NAME‘","hosts":[""],"key":{"algo":"rsa","size":2048}}‘ | cfssl gencert -config=ca-config.json -ca=ca.pem -ca-key=ca-key.pem -hostname="$ADDRESS" - | cfssljson -bare $NAME
export ADDRESS=
export NAME=client
echo ‘{"CN":"‘$NAME‘","hosts":[""],"key":{"algo":"rsa","size":2048}}‘ | cfssl gencert -config=ca-config.json -ca=ca.pem -ca-key=ca-key.pem -hostname="$ADDRESS" - | cfssljson -bare $NAME
# sh bulid-key.sh

# ll
total 44
-rw-r--r-- 1 root root  732 Apr  3 05:13 build-ca.sh
-rw-r--r-- 1 root root  112 Apr  3 05:13 ca-config.json
-rw-r--r-- 1 root root  883 Apr  3 05:13 ca.csr
-rw------- 1 root root 1675 Apr  3 05:13 ca-key.pem
-rw-r--r-- 1 root root 1119 Apr  3 05:13 ca.pem
-rw-r--r-- 1 root root  928 Apr  3 05:13 client.csr
-rw------- 1 root root 1675 Apr  3 05:13 client-key.pem
-rw-r--r-- 1 root root 1180 Apr  3 05:13 client.pem
-rw-r--r-- 1 root root  928 Apr  3 05:13 server.csr
-rw------- 1 root root 1679 Apr  3 05:13 server-key.pem
-rw-r--r-- 1 root root 1220 Apr  3 05:13 server.pem

4、将相关的文件复制到etc节点上

# mkdir -p /etc/ssl/etcd/
# cp ./*.pem  /etc/ssl/etcd/
# scp -rp /etc/ssl/etcd/ vm2:/etc/ssl/
# scp -rp /etc/ssl/etcd/ vm3:/etc/ssl/

5、配置etcd启动加载相关证书
Vm1:

# etcd --name=infra0 --data-dir=/data/etcd --listen-client-urls=https://192.168.115.5:2379,https://127.0.0.1:2379 --advertise-client-urls=https://192.168.115.5:2379,https://127.0.0.1:2379 --listen-peer-urls=https://192.168.115.5:2380 --initial-advertise-peer-urls=https://192.168.115.5:2380 --initial-cluster=infra0=https://192.168.115.5:2380,infra1=https://192.168.115.6:2380,infra2=https://192.168.115.7:2380 --initial-cluster-token=etcd-cluster-token --initial-cluster-state=new --cert-file=/etc/ssl/etcd/server.pem --key-file=/etc/ssl/etcd/server-key.pem --peer-cert-file=/etc/ssl/etcd/server.pem --peer-key-file=/etc/ssl/etcd/server-key.pem --trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-client-cert-auth=true --client-cert-auth=true

vm2：

# etcd --name=infra1 --data-dir=/data/etcd --listen-client-urls=https://192.168.115.6:2379,https://127.0.0.1:2379 --advertise-client-urls=https://192.168.115.6:2379,https://127.0.0.1:2379 --listen-peer-urls=https://192.168.115.6:2380 --initial-advertise-peer-urls=https://192.168.115.6:2380 --initial-cluster=infra0=https://192.168.115.5:2380,infra1=https://192.168.115.6:2380,infra2=https://192.168.115.7:2380 --initial-cluster-token=etcd-cluster-token --initial-cluster-state=new --cert-file=/etc/ssl/etcd/server.pem --key-file=/etc/ssl/etcd/server-key.pem --peer-cert-file=/etc/ssl/etcd/server.pem --peer-key-file=/etc/ssl/etcd/server-key.pem --trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-client-cert-auth=true --client-cert-auth=true

Vm3:

# etcd --name=infra2 --data-dir=/data/etcd --listen-client-urls=https://192.168.115.7:2379,https://127.0.0.1:2379 --advertise-client-urls=https://192.168.115.7:2379,https://127.0.0.1:2379 --listen-peer-urls=https://192.168.115.7:2380 --initial-advertise-peer-urls=https://192.168.115.7:2380 --initial-cluster=infra0=https://192.168.115.5:2380,infra1=https://192.168.115.6:2380,infra2=https://192.168.115.7:2380 --initial-cluster-token=etcd-cluster-token --initial-cluster-state=new --cert-file=/etc/ssl/etcd/server.pem --key-file=/etc/ssl/etcd/server-key.pem --peer-cert-file=/etc/ssl/etcd/server.pem --peer-key-file=/etc/ssl/etcd/server-key.pem --trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-client-cert-auth=true --client-cert-auth=true

6、验证

# export ETCDCTL_API=2
# etcdctl --cert-file=/etc/ssl/etcd/client.pem   --key-file=/etc/ssl/etcd/client-key.pem  --ca-file=/etc/ssl/etcd/ca.pem --endpoints=https://192.168.115.5:2379,https://192.168.115.6:2379,https://192.168.115.7:2379 cluster-health
# export ETCDCTL_API=3
# etcdctl --write-out=table --cert=/etc/ssl/etcd/client.pem --key=/etc/ssl/etcd/client-key.pem --cacert=/etc/ssl/etcd/ca.pem --endpoints=https://192.168.115.5:2379,https://192.168.115.6:2379,https://192.168.115.7:2379
member list

6、配置自启动脚本

# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/data/etcd/
EnvironmentFile=-/etc/etcd.conf
User=root
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/sbin/etcd --name=infra0 --data-dir=/data/etcd --listen-client-urls=https://192.168.115.5:2379,https://127.0.0.1:2379 --advertise-client-urls=https://192.168.115.5:2379,https://127.0.0.1:2379 --listen-peer-urls=https://192.168.115.5:2380 --initial-advertise-peer-urls=https://192.168.115.5:2380 --initial-cluster=infra0=https://192.168.115.5:2380,infra1=https://192.168.115.6:2380,infra2=https://192.168.115.7:2380 --initial-cluster-token=etcd-cluster-token --initial-cluster-state=new --cert-file=/etc/ssl/etcd/server.pem --key-file=/etc/ssl/etcd/server-key.pem --peer-cert-file=/etc/ssl/etcd/server.pem --peer-key-file=/etc/ssl/etcd/server-key.pem --trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-trusted-ca-file=/etc/ssl/etcd/ca.pem --peer-client-cert-auth=true --client-cert-auth=true"
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
# systemctl daemon-reload


参考：
https://coreos.com/os/docs/latest/generate-self-signed-certificates.html

原文地址：http://blog.51cto.com/ylw6006/2095871

时间： 2024-08-30 00:16:55

配置Etcd集群和TLS认证的相关文章

配置Etcd集群

简单的配置Etcd集群 3.0 3个节点IP 192.168.134.140 192.168.134.141 192.168.134.142 3.1 下载etcd curl -L  https://github.com/coreos/etcd/releases/download/v2.3.6/etcd-v2.3.6-linux-amd64.tar.gz -o etcd-v2.3.6-linux-amd64.tar.gz 3.2 tar xzvf etcd-v2.3.6-linux-amd64.t

Etcd集群安装配置

本次测试集群为2各节点 一. Etcd集群安装配置 安装包:etcd-3.3.11-2.el7.centos.x86_64.rpm 配置文件: #[Member] #ETCD_CORS="" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" #ETCD_WAL_DIR="" ETCD_LISTEN_PEER_URLS="http://192.168.218.146:2380" ETCD_LI

kubeadm配置高可用etcd集群

操作系统为ubuntu18 kubernetes版本为v1.15.1 k8s默认在控制平面节点上的kubelet管理的静态pod中运行单个成员的etcd集群,但这不是高可用的方案. etcd高可用集群至少需要三个成员组成. etcd默认端口为2379,2380,三个节点的这两个端口都要能通. 可以在kubeadm配置文件更改默认端口. 这个实验有五个服务器. 我开的腾讯云香港服务器做的实验,网速很快,ssh稳定. 百度云没测. 阿里云测试不给力. 推荐腾讯云. k8s1: master1? k8

Kubernetes（K8s）安装部署过程（三）--创建高可用etcd集群

这里的etcd集群复用我们测试的3个节点,3个node都要安装并启动,注意修改配置文件 1.TLS认证文件分发:etcd集群认证用,除了本机有,分发到其他node节点 scp ca.pem kubernetes-key.pem kubernetes.pem [email protected]10.10.90.106:/etc/kubernetes/ssl scp ca.pem kubernetes-key.pem kubernetes.pem [email protected]10.10.90.

k8s之二进制安装etcd集群

前言 kubeadm安装的集群,默认etcd是一个单机的容器化的etcd,并且k8s和etcd通信没有经过ssl加密和认证,这点是需要改造的.所以首先我们需要先部署一个三节点的etcd集群,二进制部署,systemd守护进程,并且需要生成ca证书 ETCD集群详情 主机 IP 节点名称 etcd的名称 主机01 192.168.56.200 MM etcd1 主机02 192.168.56.201 SS01 etcd2 主机03 192.168.56.202 SS02 etcd3 master上

部署k8s ssl集群实践4:部署etcd集群

参考文档:https://github.com/opsnull/follow-me-install-kubernetes-cluster感谢作者的无私分享.集群环境已搭建成功跑起来.文章是部署过程中遇到的错误和详细操作步骤记录.如有需要对比参考,请按照顺序阅读和测试. 4.1下载和分发二进制安装包 [[email protected] kubernetes]# wget https://github.com/coreos/etcd/releases/download/v3.3.7/etcd-v3

kubernetes 集群安装etcd集群，带证书

install etcd 准备证书 https://www.kubernetes.org.cn/3096.html 在master1需要安装CFSSL工具,这将会用来建立 TLS certificates. export CFSSL_URL="https://pkg.cfssl.org/R1.2" wget "${CFSSL_URL}/cfssl_linux-amd64" -O /usr/local/bin/cfssl wget "${CFSSL_URL}

etcd集群搭建（高可用）

一.etcd介绍: ETCD 是一个高可用的分布式键值数据库,可用于服务发现.ETCD 采用 raft 一致性算法,基于 Go 语言实现.etcd作为一个高可用键值存储系统,天生就是为集群化而设计的.由于Raft算法在做决策时需要多数节点的投票,所以etcd一般部署集群推荐奇数个节点,推荐的数量为3.5或者7个节点构成一个集群. 二.特点: 实际上,etcd作为一个受到Zookeeper与doozer启发而催生的项目,除了拥有与之类似的功能外,更具有以下4个特点{![引自Docker官方文档]}

部署etcd集群

部署etcd集群 第一步:先拉取etcd二进制压缩包 wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz //解压压缩包 tar zxvf etcd-v3.3.2-linux-amd64.tar.gz 第二步:建立一个文件,分别存放bin文件,cfg配置文件,ssl验证文件 mkdir /opt/kubernetes/{bin,cfg,ssl} //然后将etcd,