Kubernetes群集之：二进制部署单etcd，多节点集群

Kubernetes集群部署

1.官方提供的三种部署方式
2.Kubernetes平台环境规划
3.自签SSL证书
4.Etcd数据库群集部署
5.Node安装Docker
6.Flannel容器集群网络部署
7.部署Master组件
8.部署Node组件
9.部署一个测试示例
10.部署Web UI（Dashboard）
11.部署集群内部DNS解析服务（CoreDNS）

官方提供的三种部署方式：

minikube：

Minikube是一个工具，可以在本地快速运行单点的Kubernetes，仅用于尝试Kubernetes或日常开发的用户使用
部署地址：https://kubernetes.io/docs/setup/minikube/

kubeadm：

Kubeadm也是一个工具，提供kubeadm init和kubeadm join，用于快速部署Kubernetes集群
部署地址：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

二进制包：

推荐，从官方下载发行版的二进制包，手动部署每个组件包，组成Kubernetes集群
下载地址：https://github.com/kubernetes/kubernetes/releases

要解决服务发现的问题，需要下面三大支柱，缺一不可

1.一个强一致性，高可用的服务存储目录

基于Ralf算法的etcd天生就是这样一个强一致性，高可用的服务存储目录

2.一秒注册服务和健康服务健康状况的机制

用户可以在etcdz中注册服务，并且对注册的服务配置key TTL，定时保持服务的心跳以达到监控健康状态的效果

3.一种查找和连接服务的机制

通过在etcd指定的主题下注册的服务业能在对应的主题下查到，为了确保连接，我们可以在每个服务机器上都部署一个proxy模式的etcd，这样就可以确保访问etcd集群的服务都能够互相连接

Demo：二进制部署多节点，单etcd群集

环境准备：

相关软件包及文档：

链接：https://pan.baidu.com/s/1nn67GDs8BD6sQTeKH4Ii4w
提取码：vx7m

Mester：7-3：192.168.18.128 kube-apiserver kube-controller-manager kube-scheduler etcd

Node1：7-4：192.168.18.148 kubelet kube-proxy docekr flannel etcd

Node2：7-5：192.168.18.145 kubelet kube-proxy docekr flannel etcd

Mester7-3：

[[email protected] ~]# mkdir k8s
[[email protected] ~]# cd k8s/
[[email protected] k8s]# mkdir etcd-cert
[[email protected] k8s]# mv etcd-cert.sh etcd-cert
[[email protected] k8s]# ls
etcd-cert  etcd.sh
[[email protected] k8s]# vim cfssl.sh
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
[[email protected] k8s]# bash cfssl.sh
[[email protected] k8s]# ls /usr/local/bin/
cfssl  cfssl-certinfo  cfssljson

`定义CA证书`
cat > ca-config.json <<EOF
{
  "signing":{
    "default":{
      "expiry":"87600h"
    },
    "profiles":{
      "www":{
        "expiry":"87600h",
        "usages":[
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ]
      }
    }
  }
}
EOF

`实证书签名`
cat > ca-csr.json <<EOF
{
    "CN":"etcd CA",
    "key":{
        "algo":"rsa",
        "size":2048
    },
    "names":[
        {
            "C":"CN",
            "L":"Nanjing",
            "ST":"Nanjing"
        }
    ]
}
EOF

`生产证书，生成ca-key.pem  ca.pem`
[[email protected] k8s]# cd etcd-cert/
[[email protected] etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2020/01/15 11:26:22 [INFO] generating a new CA key and certificate from CSR
2020/01/15 11:26:22 [INFO] generate received request
2020/01/15 11:26:22 [INFO] received CSR
2020/01/15 11:26:22 [INFO] generating key: rsa-2048
2020/01/15 11:26:23 [INFO] encoded CSR
2020/01/15 11:26:23 [INFO] signed certificate with serial number 58994014244974115135502281772101176509863440005

`指定etcd三个节点之间的通信验证`
cat > server-csr.json <<EOF
{
    "CN": "etcd",
    "hosts": [
    "192.168.18.128",
    "192.168.18.148",
    "192.168.18.145"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "NanJing",
            "ST": "NanJing"
        }
    ]
}
EOF

`生成ETCD证书 server-key.pem   server.pem`
[[email protected] etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
2020/01/15 11:28:07 [INFO] generate received request
2020/01/15 11:28:07 [INFO] received CSR
2020/01/15 11:28:07 [INFO] generating key: rsa-2048
2020/01/15 11:28:07 [INFO] encoded CSR
2020/01/15 11:28:07 [INFO] signed certificate with serial number 153451631889598523484764759860297996765909979890
2020/01/15 11:28:07 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
上传以下三个压缩包进行解压：

[[email protected] etcd-cert]# ls
ca-config.json  etcd-cert.sh                          server-csr.json
ca.csr          etcd-v3.3.10-linux-amd64.tar.gz       server-key.pem
ca-csr.json     flannel-v0.10.0-linux-amd64.tar.gz    server.pem
ca-key.pem      kubernetes-server-linux-amd64.tar.gz
ca.pem          server.csr
[[email protected] etcd-cert]# mv *.tar.gz ../
[[email protected] etcd-cert]# cd ../
[[email protected] k8s]# ls
cfssl.sh   etcd.sh                          flannel-v0.10.0-linux-amd64.tar.gz
etcd-cert  etcd-v3.3.10-linux-amd64.tar.gz  kubernetes-server-linux-amd64.tar.gz
[[email protected] k8s]# tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
[[email protected] k8s]# ls etcd-v3.3.10-linux-amd64
Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md
[[email protected] k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p
[[email protected] k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/

`证书拷贝`
[[email protected] k8s]# cp etcd-cert/*.pem /opt/etcd/ssl/

`进入卡住状态等待其他节点加入`
[[email protected] k8s]# bash etcd.sh etcd01 192.168.18.128 etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
此时新打开一个7-3的远程连接终端：
[[email protected] ~]# ps -ef | grep etcd
root       3479   1780  0 11:48 pts/0    00:00:00 bash etcd.sh etcd01 192.168.18.128 etcd02=https://192.168.195.148:2380,etcd03=https://192.168.195.145:2380
root       3530   3479  0 11:48 pts/0    00:00:00 systemctl restart etcd
root       3540      1  1 11:48 ?        00:00:00 /opt/etcd/bin/etcd
--name=etcd01 --data-dir=/var/lib/etcd/default.etcd
--listen-peer-urls=https://192.168.18.128:2380
--listen-client-urls=https://192.168.18.128:2379,http://127.0.0.1:2379
--advertise-client-urls=https://192.168.18.128:2379
--initial-advertise-peer-urls=https://192.168.18.128:2380
--initial-cluster=etcd01=https://192.168.18.128:2380,etcd02=https://192.168.195.148:2380,etcd03=https://192.168.195.145:2380
--initial-cluster-token=etcd-cluster
--initial-cluster-state=new
--cert-file=/opt/etcd/ssl/server.pem
--key-file=/opt/etcd/ssl/server-key.pem
--peer-cert-file=/opt/etcd/ssl/server.pem
--peer-key-file=/opt/etcd/ssl/server-key.pem
--trusted-ca-file=/opt/etcd/ssl/ca.pem
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
root       3623   3562  0 11:49 pts/1    00:00:00 grep --color=auto etcd
`拷贝证书去其他节点`
[[email protected] k8s]# scp -r /opt/etcd/ [email protected]:/opt/
The authenticity of host ‘192.168.18.148 (192.168.18.148)‘ can‘t be established.
ECDSA key fingerprint is SHA256:mTT+FEtzAu4X3D5srZlz93S3gye8MzbqVZFDzfJd4Gk.
ECDSA key fingerprint is MD5:fa:5a:88:23:49:60:9b:b8:7e:4b:14:4b:3f:cd:96:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘192.168.18.148‘ (ECDSA) to the list of known hosts.
[email protected]‘s password:
etcd                                                       100%  518   426.8KB/s   00:00
etcd                                                       100%   18MB 105.0MB/s   00:00
etcdctl                                                    100%   15MB 108.2MB/s   00:00
ca-key.pem                                                 100% 1679     1.4MB/s   00:00
ca.pem                                                     100% 1265   396.1KB/s   00:00
server-key.pem                                             100% 1675     1.0MB/s   00:00
server.pem                                                 100% 1338   525.6KB/s   00:00
[[email protected] k8s]# scp -r /opt/etcd/ [email protected]:/opt/
The authenticity of host ‘192.168.18.145 (192.168.18.145)‘ can‘t be established.
ECDSA key fingerprint is SHA256:mTT+FEtzAu4X3D5srZlz93S3gye8MzbqVZFDzfJd4Gk.
ECDSA key fingerprint is MD5:fa:5a:88:23:49:60:9b:b8:7e:4b:14:4b:3f:cd:96:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘192.168.18.145‘ (ECDSA) to the list of known hosts.
[email protected]‘s password:
etcd                                                       100%  518   816.5KB/s   00:00
etcd                                                       100%   18MB  87.4MB/s   00:00
etcdctl                                                    100%   15MB 108.6MB/s   00:00
ca-key.pem                                                 100% 1679     1.3MB/s   00:00
ca.pem                                                     100% 1265   411.8KB/s   00:00
server-key.pem                                             100% 1675     1.4MB/s   00:00
server.pem                                                 100% 1338   639.5KB/s   00:00

`启动脚本拷贝其他节点`
[[email protected] k8s]# scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
[email protected]‘s password:
etcd.service                                               100%  923   283.4KB/s   00:00
[[email protected] k8s]# scp /usr/lib/systemd/system/etcd.service [email protected]:/usr/lib/systemd/system/
[email protected]‘s password:
etcd.service                                               100%  923   347.7KB/s   00:00

Node1：7-4

`修改`
[[email protected] ~]# systemctl stop firewalld.service
[[email protected] ~]# setenforce 0
[[email protected] ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.18.148:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.18.148:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.18.148:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.18.148:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.18.128:2380,etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

[[email protected] ~]# systemctl start etcd
[[email protected] ~]# systemctl status etcd
● etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
   Active: active (running) since 三 2020-01-15 17:53:24 CST; 5s ago
#状态为Active

Node2：7-5

`修改`
[[email protected] ~]# systemctl stop firewalld.service
[[email protected] ~]# setenforce 0
[[email protected] ~]# vim /opt/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.18.145:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.18.145:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.18.145:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.18.145:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.18.128:2380,etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

[[email protected] ~]# systemctl start etcd
[[email protected] ~]# systemctl status etcd
● etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
   Active: active (running) since 三 2020-01-15 17:55:24 CST; 5s ago
 #状态为Active

群集状态验证：

`回到7-3上输入以下命令：`
[[email protected] k8s]# cd etcd-cert/
[[email protected] etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.18.128:2379,https://192.168.18.148:2379,https://192.168.18.145:2379" cluster-health
member 9104d301e3b6da41 is healthy: got healthy result from https://192.168.18.148:2379
member 92947d71c72a884e is healthy: got healthy result from https://192.168.18.145:2379
member b2a6d67e1bc8054b is healthy: got healthy result from https://192.168.18.128:2379
cluster is healthy
`状态为healthy健康`

原文地址：https://blog.51cto.com/14464303/2467093

时间： 2024-11-07 14:21:48

Kubernetes群集之：二进制部署单etcd，多节点集群的相关文章

ELK开发部署（1）：安装Elasticsearch组建单服务器多节点集群

ELK简介 最近有个需求搭建一套日志系统用于集成几个业务系统的日志提供快速的检索功能,目前是用Log4net存数据库+Error级别发邮件方式,也算简单暴力好用,但历史日志的模糊查询确实很慢,所以使用ELK是比较理想的解决方案.三年前写过两篇文章log4net.NoSql +ElasticSearch 实现日志记录和log4net.redis+logstash+kibana+elasticsearch+redis 实现日志系统,感觉有点凌乱,这次借着项目需要重新整理一下.参考ELK不权威指南可以

ELK日志框架（1）：安装Elasticsearch组建单服务器多节点集群

原文:ELK日志框架(1):安装Elasticsearch组建单服务器多节点集群 ELK简介 最近有个需求搭建一套日志系统用于集成几个业务系统的日志提供快速的检索功能,目前是用Log4net存数据库+Error级别发邮件方式,也算简单暴力好用,但历史日志的模糊查询确实很慢,所以使用ELK是比较理想的解决方案.三年前写过两篇文章log4net.NoSql +ElasticSearch 实现日志记录和log4net.redis+logstash+kibana+elasticsearch+redis

kubeadm搭建kubernetes（v1.13.1）单节点集群

kubeadm是Kubernetes官方提供的用于快速部署Kubernetes集群的工具,本篇文章使用kubeadm搭建一个单master节点的k8s集群. 节点部署信息 节点主机名 节点IP 节点角色 操作系统 k8s-master 10.10.55.113 master centos7.6 k8s-node1 10.10.55.114 node centos7.6 节点说明 master:控制节点.kube-apiserver负责API服务,kube-controller-manager负责

二进制部署MySQL（运维技术交流群：926402931，欢迎大家一起来交流。）

目录 1.MySQL安装前准备 1.1.安装依赖包 1.2.安装cmake 1.3.创建用户 2.MySQL下载安装 2.1.创建软件下载目录 2.2.下载并上传到/server/tools 2.3.解压 2.4.安装 3.配置并启动 3.1.创建软链接 3.2.拷贝配置文件到/etc 3.3.初始化数据库 3.4.创建目录并授权 3.5.复制启动脚本到/etc/init.d/mysqld 3.6.启动数据库 3.7.配置环境变量 1.MySQL安装前准备 1.1.安装依赖包 yum insta

redis单节点集群

一.概念 redis是一种支持Key-Value等多种数据结构的存储系统.可用于缓存.事件发布或订阅.高速队列等场景.该数据库使用ANSI C语言编写,支持网络,提供字符串.哈希.列表.队列.集合结构直接存取,基于内存,可持久化. 二.redis的应用场景有哪些 1.会话缓存(最常用) 2.消息队列,比如支付 3.活动排行榜或计数 4.发布.订阅消息(消息通知) 5.商品列表.评论列表等 1.redis安装: # wget http://download.redis.io/releases/re

Hadoop学习笔记（二）设置单节点集群

本文描述如何设置一个单一节点的 Hadoop 安装,以便您可以快速执行简单的操作,使用 Hadoop MapReduce 和 Hadoop 分布式文件系统 (HDFS). 参考官方文档:Hadoop MapReduce Next Generation - Setting up a Single Node Cluster. Hadoop版本:Apache Hadoop 2.5.1 系统版本:CentOS 6.5,内核(uname -r):2.6.32-431.el6.x86_64 系统必备组件 支

Hadoop学习笔记（两）设置单节点集群

本文描写叙述怎样设置一个单一节点的 Hadoop 安装.以便您能够高速运行简单的操作,使用 Hadoop MapReduce 和 Hadoop 分布式文件系统 (HDFS). 參考官方文档:Hadoop MapReduce Next Generation - Setting up a Single Node Cluster. Hadoop版本号:Apache Hadoop 2.5.1 系统版本号:CentOS 6.5.内核(uname -r):2.6.32-431.el6.x86_64 系统必备

使用Rancher Server部署本地多节点K8S集群

当我第一次开始我的Kubernetes之旅时,我一直在寻找一种设置本地部署环境的方式.很多人常常会使用minikube或microk8s,这两者非常适合新手在单节点集群环境下进行操作.但当我已经了解了基础知识之后,这两者显然不太够用,我需要进一步寻找能够运行本地多节点集群.与生产环境更相似的平台.为此,我查阅了许多参考资料,最后我找到了Rancher Server.接下来,我要介绍我是如何设置我的本地K8S多节点集群的.  准备master节点和worker节点的虚拟机  上图显示了集群的架构,

Kubernetes二进制部署之单节点部署

K8s单节点二进制部署 K8s二进制部署搭建步骤: 1:自签ETCD证书 2:ETCD部署 3:Node安装docker 4:Flannel部署(先写入子网到etcd) ---------master---------- 5:自签APIServer证书 6:部署APIServer组件(token,csv) 7:部署controller-manager(指定apiserver证书)和scheduler组件 ----------node---------- 8:生成kubeconfig(bootst