上一篇我们已经搭建etcd高可用集群，参考：二进制搭建kubernetes多master集群【一、使用TLS证书搭建etcd集群】
此文将搭建flannel网络，目的使跨主机的docker能够互相通信，也是保障kubernetes集群的网络基础和保障，下面正式开始配置。
一、部署 Flannel
在所有集群节点都安装Flannel，下面的操作以etcd集群的三个节点为例都执行一遍
1、下载安装Flannel
wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz
tar -xzvf flannel-v0.10.0-linux-amd64.tar.gz
cp flannel-v0.10.0-linux-amd64/{flanneld,mk-docker-opts.sh} /usr/local/bin
2、向 etcd 写入网段信息
下面2条命令在etcd集群中任意一台执行一次即可，也是是创建一个flannel网段供docker分配使用
etcdctl --endpoints=https://192.168.20.210:2379,https://192.168.20.211:2379,https://192.168.20.212:2379 --ca-file=/etc/kubernetes/pki/etcd/ca.pem --cert-file=/etc/kubernetes/pki/etcd/kubernetes.pem --key-file=/etc/kubernetes/pki/etcd/kubernetes-key.pem mkdir /kubernetes/network
etcdctl --endpoints=https://192.168.20.210:2379,https://192.168.20.211:2379,https://192.168.20.212:2379 --ca-file=/etc/kubernetes/pki/etcd/ca.pem --cert-file=/etc/kubernetes/pki/etcd/kubernetes.pem --key-file=/etc/kubernetes/pki/etcd/kubernetes-key.pem mk /kubernetes/network/config ‘{"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}‘
3、创建system unit文件
[[email protected] ~]#cat > /etc/systemd/system/flanneld.service <<EOF [Unit] Description=Flanneld overlay address etcd agent After=network.target After=network-online.target Wants=network-online.target After=etcd.service Before=docker.service [Service] Type=notify ExecStart=/usr/local/bin/flanneld -etcd-cafile=/etc/kubernetes/pki/etcd/ca.pem -etcd-certfile=/etc/kubernetes/pki/etcd/kubernetes.pem -etcd-keyfile=/etc/kubernetes/pki/etcd/kubernetes-key.pem -etcd-endpoints=https://192.168.20.210:2379,https://192.168.20.211:2379,https://192.168.20.212:2379 \ -etcd-prefix=/kubernetes/network ExecStartPost=/usr/local/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker Restart=on-failure [Install] WantedBy=multi-user.target RequiredBy=docker.service EOF
mk-docker-opts.sh 脚本将分配给 flanneld 的 Pod 子网网段信息写入到 /run/flannel/docker 文件中，后续 docker 启动时使用这个文件中参数值设置 docker0 网桥。
flanneld 使用系统缺省路由所在的接口和其它节点通信，对于有多个网络接口的机器（如，内网和公网），可以用 -iface=enpxx 选项值指定通信接口。
4、查看flannel分配的子网信息
[[email protected] ~]# cat /run/flannel/docker DOCKER_OPT_BIP="--bip=172.30.94.1/24" DOCKER_OPT_IPMASQ="--ip-masq=true" DOCKER_OPT_MTU="--mtu=1450" DOCKER_NETWORK_OPTIONS=" --bip=172.30.94.1/24 --ip-masq=true --mtu=1450" [[email protected]-master1 ~]# cat /run/flannel/subnet.env FLANNEL_NETWORK=172.30.0.0/16 FLANNEL_SUBNET=172.30.94.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=false
/run/flannel/docker是flannel分配给docker的子网信息，/run/flannel/subnet.env包含了flannel整个大网段以及在此节点上的子网段。 5、启动flannel并且设置开机自启动
systemctl daemon-reload systemctl enable flanneld systemctl start flanneld
6、查看flannel网络是否生效
[[email protected] ~]# ifconfig
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.30.94.1 netmask 255.255.255.0 broadcast 172.30.94.255
inet6 fe80::42:1aff:fed2:a4b4 prefixlen 64 scopeid 0x20<link>
ether 02:42:1a:d2:a4:b4 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 648 (648.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.20.210 netmask 255.255.254.0 broadcast 192.168.21.255
inet6 fe80::ff2:187b:66fc:621b prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:c3:dc:a5 txqueuelen 1000 (Ethernet)
RX packets 3965867 bytes 619350597 (590.6 MiB)
RX errors 0 dropped 583 overruns 0 frame 0
TX packets 3159970 bytes 390102190 (372.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 172.30.94.0 netmask 255.255.255.255 broadcast 0.0.0.0
inet6 fe80::7827:fcff:fe4e:b5ff prefixlen 64 scopeid 0x20<link>
ether 7a:27:fc:4e:b5:ff txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 8 overruns 0 carrier 0 collisions 0
可以明显看到flannel1.1的网络信息，说明flannel网络已经正常。
二、配置docker支持flannel网络
1、所有node安装docker
关于安装docker，请参考：安装指定版本的docker
2、配置docker支持flannel网络，所有docker节点都操作
[[email protected] ~]# vi /etc/systemd/system/multi-user.target.wants/docker.service [Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker EnvironmentFile=/run/flannel/docker ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. #TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process # restart the docker process if it exits prematurely Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target
如上红色即为新增加支持flannel网络的配置
3、重启docker，使配置生效
systemctl daemon-reload systemctl restart docker
4、查看docker网络是否生效
i、启动一个容器（如有现有容器可以不run一个新的）
[[email protected]-master1 ~]# docker run -itd centos
d63ee9c72b5023fgfg36ld93j6723hd72jd1hsp2303kf7
ii、查看ip地址是否是flannel网络分配的网段
[[email protected]-master1 ~]# docker inspect -f ‘{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}‘ d63
172.30.94.2
发现容器使用了172.30.94.0/24网段。属于flannel，配置完成。
原文地址：https://www.cnblogs.com/harlanzhang/p/10118214.html
时间： 2024-10-13 08:43:41