k8s简单集群搭建

目录

  • 1 环境准备
  • 2. 实验过程

注意：在使用该文档之前，您应该要对k8s的组件有一些了解，我就不描述组件在集群中所担任的角色了, 如有需要请移步官网或其他博客。并且没有使用到证书

1 环境准备

1.1 基本环境

  • 操作系统
CentOS Linux release 7.4.1708 (Core)
  • 软件版本
Kubernetes v1.9.1  （后面提供tar包）
etcd Version: 3.2.18（直接yum 安装）(源码包地址:   https://github.com/coreos/etcd/releases)
flanneld Version: 0.7.1（直接yum安装）(源码包地址：https://github.com/coreos/flannel/releases)
docker Version: docker://1.13.1（直接yum安装）
  • IP部署
master：192.168.1.192 （kube-apiserver, kube-controller-manager, kube-scheduler, etcd, flannel(非必须)）
node1：192.168.1.193  （kubelet, kube-proxy, etcd, flannel）
node2：192.168.1.194   ( kubelet, kube-proxy, etcd, flannel）

1.2 初始化操作

  • /etc/hosts配置
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.192 master
192.168.1.193 node1
192.168.1.194 node2
# 将这个文件拷贝到所有的节点上（master, node1, node2）
  • 防火墙+selinux
systemctl stop firewalld  # 注意这里执行后 最好用 firewall-cmd --state再确认一下，如果是Not running表示关闭了
vim /etc/sysconfig/selinux
SELINUX=disabled
  • repo文件
cd /etc/yum.repos.d/
cat kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
# 清除yum 缓存
yum clean all
# 建立缓存
yum makecache
  • 重启系统

    reboot

2. 实验过程

2.1 安装etcd集群（master, node1,node2上都要执行）

  • yum安装etcd
yum -y install etcd
yum -y install docker
  • /etc/etcd/etcd.conf
# [member]
ETCD_NAME=infra1
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.1.192:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.1.192:2379"
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.1.192:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.1.192:2379"
  • etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd   --name ${ETCD_NAME}   --initial-advertise-peer-urls ${ETCD_INITIAL_ADVERTISE_PEER_URLS}   --listen-peer-urls ${ETCD_LISTEN_PEER_URLS}   --listen-client-urls ${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379   --advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS}   --initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN}   --initial-cluster infra1=http://192.168.1.192:2380,infra2=http://192.168.1.193:2380,infra3=http://192.168.1.194:2380   --initial-cluster-state new \  # 设置该选项,那么在--initial-cluster中就要指定所有etcd集群成员
  --data-dir=${ETCD_DATA_DIR}
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
  • 启动etcd
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd

2.2 配置master节点（master上执行）

  • 下载k8s 1.9.1版本包

    下载地址，这里面就包含了master和node上所需的组件

mkdir -p /root/local/bin && mkdir /etc/kubernetes（master和node上都要执行，下面就直接使用了）
# 将/root/local/bin追加到系统PATH路径后
cat /root/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
 . ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:/root/local/bin  #就是这里啦
export PATH

source /root/.bash_profile


# cd /usr/src
wget https://storage.googleapis.com/kubernetes-release/release/v1.9.1/kubernetes-server-linux-amd64.tar.gz
tar -xf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp -r kube-apiserver kube-controller-manager kubectl kube-scheduler /root/local/bin

配置kube-apiserver

  • 配置/etc/kubernetes/apiserver
cat /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
# The port on the local server to listen on.
# KUBE_API_PORT="--port=8080"

# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.1.192:2379,http://192.168.1.193:2379,http://192.168.1.194:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
#KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""
  • 配置/etc/kubernetes/config
cat /etc/kubernetes/config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service  #请注意到这里的说明，这个文件是所有节点都需要的，所以要分发到所有的节点对应的路径下
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://192.168.1.192:8080"
  • 配置kube-apiserver.service （/usr/lib/systemd/system/kube-apiserver.service）
cd /usr/lib/systemd/system/
cat kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
After=etcd.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/root/local/bin/kube-apiserver             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_ETCD_SERVERS             $KUBE_API_ADDRESS             $KUBE_API_PORT             $KUBELET_PORT             $KUBE_ALLOW_PRIV             $KUBE_SERVICE_ADDRESSES             $KUBE_ADMISSION_CONTROL             $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

配置kube-controller-manager

  • 配置/etc/kubernetes/controller-manager
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
  • 配置/usr/lib/systemd/system/kube-controller-manager.service
cat /usr/lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/root/local/bin/kube-controller-manager             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

配置kube-scheduler

  • 配置/etc/kubernetes/scheduler
###
# kubernetes scheduler config
# default config should be adequate
# Add your own!
#KUBE_SCHEDULER_ARGS="--loglevel=0"
KUBE_SCHEDULER_ARGS="--address=127.0.0.1"
  • 配置/usr/lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler Plugin
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/root/local/bin/kube-scheduler             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

启动服务

for i in ‘kube-apiserver,kube-controller-manager,kube-scheduler‘
do
    systemctl enabld $i
    systemctl start $i
done

2.3 配置docker私有仓库（在master上执行，当然也可以是其他的节点，只要能访问到就行）

因为没有配置TLS所以我们需要修改一下docker.service文件

  • 修改docker.service文件
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target rhel-push-plugin.socket registries.service
Wants=docker-storage-setup.service
Requires=docker-cleanup.timer

[Service]
Type=notify
NotifyAccess=all
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/libexec/docker:/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd-current           --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current           --default-runtime=docker-runc           --exec-opt native.cgroupdriver=systemd           --userland-proxy-path=/usr/libexec/docker/docker-proxy-current           --init-path=/usr/libexec/docker/docker-init-current           --seccomp-profile=/etc/docker/seccomp.json           --insecure-registry=192.168.1.192:5000 \  # 就是这里啦，来访问该IP:PROT时不用https，node1和node2的docker这里也是需要这样配置，后面就不赘述docker的配置了
          $OPTIONS           $DOCKER_STORAGE_OPTIONS           $DOCKER_NETWORK_OPTIONS           $ADD_REGISTRY           $BLOCK_REGISTRY           $INSECURE_REGISTRY    $REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
KillMode=process

[Install]
WantedBy=multi-user.target
  • 启动docker
systemctl enable docker
systemctl start docker
  • 私有仓库搭建

    docker search registry # 查找一下可用的registry镜像

[[email protected] ~]# docker search registry  #结果太多就列这么多吧
INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED
docker.io docker.io/registry The Docker Registry 2.0 implementation for... 2093 [OK]
docker.io docker.io/konradkleine/docker-registry-frontend Browse and modify your Docker registry in ... 194 [OK]
docker.io docker.io/hyper/docker-registry-web Web UI, authentication service and event r... 140 [OK]
docker.io docker.io/atcol/docker-registry-ui A web UI for easy private/local Docker Reg... 106 [OK]

# 我们直接用第一个
docker pull docker.io/registry
# 查看我们的镜像
docker images
[[email protected] ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/registry latest b2b03e9146e1 12 days ago 33.3 MB
# 启动registry镜像
mkdir -p /home/belle/docker_registry/
docker run -d -p 5000:5000 -v /home/belle/docker_registry/:/var/lib/registry registry
# 查看在运行的容器
[[email protected] ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e43cd73f7f76 registry "/entrypoint.sh /e..." 5 days ago Restarting (1) 46 seconds ago suspicious_sinoussi
# 下面就是将我们需要的镜像下载下来 然后push到我们的刚刚启动的容器中
# 下载一个之后实验会用到的镜像 hello-world-nginx
# 给docker.io/kitematic/hello-world-nginx 镜像打tag
docker tag docker.io/kitematic/hello-world-nginx  192.168.1.192:5000/hello-world-nginx
# push
The push refers to a repository [192.168.1.192:5000/hello-world-nginx]
5f70bf18a086: Preparing
5f70bf18a086: Preparing
5f70bf18a086: Preparing
b51acdd3ef48: Preparing
3f47ff454588: Preparing
f19fb69b288a: Preparing
b11278aeb507: Preparing
fb85701f3991: Preparing
15235e629864: Preparing
86882fc1175f: Preparing
fb85701f3991: Waiting
15235e629864: Waiting
86882fc1175f: Waiting
9e8c93c7ea7e: Preparing
e66f0ebc2eef: Preparing
6a15a6c08ef6: Preparing
461f75075df2: Preparing
9e8c93c7ea7e: Waiting
e66f0ebc2eef: Waiting
6a15a6c08ef6: Waiting
461f75075df2: Waiting
f19fb69b288a: Layer already exists
b11278aeb507: Layer already exists
fb85701f3991: Layer already exists
15235e629864: Layer already exists
86882fc1175f: Layer already exists
b51acdd3ef48: Layer already exists
3f47ff454588: Layer already exists
5f70bf18a086: Layer already exists
6a15a6c08ef6: Layer already exists
e66f0ebc2eef: Layer already exists
9e8c93c7ea7e: Layer already exists
461f75075df2: Layer already exists
latest: digest: sha256:583f0c9ca89415140fa80f70f8079f5138180a6dda2c3ff3920353b459e061a3 size: 3226
# 这个结果是因为我之前就已经Push过了

2.4 配置node节点

node1和node2上相似，因为有些地方需要修改ip指向,这里以node1为例

kubeconfig文件
cat /etc/kubernetes/kubeconfig
apiVersion: v1
clusters:
- cluster:
    server: http://192.168.1.192:8080
  name: myk8s
contexts:
- context:
    cluster: myk8s
    user: ""
  name: myk8s-context
current-context: myk8s-context
kind: Config
preferences: {}
users: []
准备所需组件
  • 将kubelet kube-proxy中master上分发到所有的node
cd /usr/src/kubernetes/server/bin/
scp kubelet kube-proxy [email protected]:/root/local/bin
scp kubelet kube-proxy [email protected]:/root/local/bin
配置kubelet
  • 配置/etc/kubernetes/kubelet
cat /etc/kubernetes/kubelet
###
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=192.168.1.193"

# The port for the info server to serve on
KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=node1"

# location of the api-server
##KUBELET_API_SERVER="--api-servers=http://127.0.0.1:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=192.168.1.192:5000/pause-amd64"

# Add your own!
KUBELET_ARGS="--cluster-dns=10.254.0.2 --cluster-domain=cluster.local --runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice --fail-swap-on=false --cgroup-driver=systemd --kubeconfig=/etc/kubernetes/kubeconfig"

注意：KUBELET_ARGS中 一定要指定--cluster-dns --cluster-domain 不然之后创建deployment后 pods永远是containercreating状态，并会报错：Warning MissingClusterDNS 13s (x9 over 12m) kubelet, node2 pod: "nginx-7ff779f954-j4t55_default(7dff9ffa-8afa-11e8-b3ac-000c290991f6)". kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.

如果--cgroup-driver=systemd那么要增加配置--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice

  • 配置kubelet.service
cd /usr/lib/systemcd/system
cat kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/root/local/bin/kubelet             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBELET_API_SERVER             $KUBELET_ADDRESS             $KUBELET_PORT             $KUBELET_HOSTNAME             $KUBE_ALLOW_PRIV             $KUBELET_ARGS             $KUBELET_POD_INFRA_CONTAINER  #注意这个参数，之前这里是没有配置的，做实验到创建pods的时候怎么都不成功，结果节点的kubelet报错提示连接google的pause-amd64超时, wtf 我明明配置文件中写的自己的私有镜像，为毛还要去连接google的，所以就直接加到启动参数了。特别提醒：重启后一定要用ps aux | grep kubelet 查看一下里面是否有这个参数()
Restart=on-failure
KillMode=process

[Install]
WantedBy=multi-user.target
配置kube-proxy
  • 配置kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/root/local/bin/kube-proxy             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
  • 配置/etc/kubernetes/proxy
cat /etc/kubernetes/proxy
###
# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS=""
配置flannel

flannel我在master，node1，node2上都安装了

  • 安装
yum -y install flannel
  • 修改/etc/sysconfig/flannel
# Flanneld configuration options  

# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="http://192.168.1.192:2379,http://192.168.1.193:2379,http://192.168.1.194:2379"
# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/kube-centos/network"
# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

mkdir -p /kube-centos/network

  • 配置flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service  #注意这里
Before=docker.service #注意这里

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure

[Install]
WantedBy=multi-user.target
WantedBy=docker.service
  • 在etcd中创建网络配置
# 在master上执行（任意安装了etcd节点的都可以）
[[email protected] ~]# mkdir -p /kube-centos/network
[[email protected] ~]# etcdctl --endpoints=http://192.168.1.192:2379,http://192.168.1.193:2379,http://192.168.1.194:2379 set /kube-centos/network/config ‘{"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}‘
  • 启动flanneld
systemctl enable flanneld
systemctl start flanneld

使用systemctl命令启动flanneld后，会自动执行./mk-docker-opts.sh -i生成如下两个文件环境变量文件

  • /run/flannel/subnet.env
  • /run/docker_opts.env

    Docker将会读取这两个环境变量文件作为容器启动参数。

    配置docker

    在/usr/lib/systemd/system/docker.service中添加

EnvironmentFile=-/run/flannel/docker
EnvironmentFile=-/run/flannel/subnet.env
  • docker.service文件为
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target rhel-push-plugin.socket registries.service
Wants=docker-storage-setup.service
Requires=docker-cleanup.timer

[Service]
Type=notify
NotifyAccess=all
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/run/flannel/docker  #增加
EnvironmentFile=-/run/flannel/subnet.env  #增加
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/libexec/docker:/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd-current           --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current           --default-runtime=docker-runc           --exec-opt native.cgroupdriver=systemd           --userland-proxy-path=/usr/libexec/docker/docker-proxy-current           --init-path=/usr/libexec/docker/docker-init-current           --seccomp-profile=/etc/docker/seccomp.json           --insecure-registry=192.168.1.192:5000           $OPTIONS           $DOCKER_STORAGE_OPTIONS           $DOCKER_NETWORK_OPTIONS           $ADD_REGISTRY           $BLOCK_REGISTRY           $INSECURE_REGISTRY    $REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
KillMode=process

[Install]
WantedBy=multi-user.target
启动服务
for i in ‘etcd flanneld kubelet kube-proxy docoker‘
do
    systemctl enable $i
    systemctl start $i
done
现在查询etcd中的内容
[[email protected] ~]# export ENDPOINTS=http://192.168.1.192:2379,http://192.168.1.193:2379,http://192.168.1.194:2379
[[email protected] ~]# etcdctl --endpoints=${ENDPOINTS} ls /kube-centos/network/subnets
/kube-centos/network/subnets/172.30.6.0-24
/kube-centos/network/subnets/172.30.61.0-24
/kube-centos/network/subnets/172.30.67.0-24
[[email protected] ~]# etcdctl --endpoints=${ENDPOINTS} get /kube-centos/network/config
{"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}
[[email protected] ~]# etcdctl --endpoints=${ENDPOINTS} get /kube-centos/network/subnets/172.30.6.0-24
{"PublicIP":"192.168.1.192","BackendType":"vxlan","BackendData":{"VtepMAC":"22:e4:55:e3:27:f6"}}
[[email protected] ~]# etcdctl --endpoints=${ENDPOINTS} get /kube-centos/network/subnets/172.30.61.0-24
{"PublicIP":"192.168.1.193","BackendType":"vxlan","BackendData":{"VtepMAC":"c6:d3:c4:4b:d1:66"}}
[[email protected] ~]# etcdctl --endpoints=${ENDPOINTS} get /kube-centos/network/subnets/172.30.67.0-24
{"PublicIP":"192.168.1.194","BackendType":"vxlan","BackendData":{"VtepMAC":"12:0c:91:23:08:83"}}
# 如果能看到以上的信息，说明flannel配置成功了

2.5 验证

在master上执行

  • 查看nodes的状态
[[email protected] ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 Ready <none> 22h v1.9.1  #是ready就OK了
node2 Ready <none> 22h v1.9.1
  • 测试集群

    创建一个nginx的service试一下集群是否可用

[[email protected] ~]# kubectl run nginx --replicas=2 --labels="run=load-balancer-example" --image=192.168.1.192:5000/hello-world-nginx --port=80
deployment "nginx" created
[[email protected] ~]# kubectl expose deployment nginx --type=NodePort --name=example-service
service "example-service" exposed
# 查看一下Pods创建结果
[[email protected] ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-7ff779f954-9sccd 1/1 Running 1 3h  #发现已经运行了  注意：这里的READY列是1/1，一个pod中可以有多个容器，而这里的pod中只有一个容器所有是1/1  一对多之后再写一些文档来说明
nginx-7ff779f954-pbps4 1/1 Running 0 17m
# 查看一下pods创建时的一些events
[[email protected] ~]# kubectl get pods nginx-7ff779f954-pbps4
Name: nginx
Namespace: default
CreationTimestamp: Thu, 19 Jul 2018 14:28:16 +0800
Labels: run=load-balancer-example
Annotations: deployment.kubernetes.io/revision=1
Selector: run=load-balancer-example
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable  #要注意这里的一些参数,如果desired和available不对等，可能就出了什么问题了
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
  Labels: run=load-balancer-example
  Containers:
   nginx:
    Image: 192.168.1.192:5000/hello-world-nginx  #从我们自建的仓库中拉取
    Port: 80/TCP
    Environment: <none>
    Mounts: <none>
  Volumes: <none>
Conditions:
  Type Status Reason
  ---- ------ ------
  Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-7ff779f954 (2/2 replicas created)
Events: <none>
Name: nginx-7ff779f954-pbps4
Namespace: default
Node: node1/192.168.1.193  #第一个节点
Start Time: Thu, 19 Jul 2018 17:44:22 +0800
Labels: pod-template-hash=3993359510
                run=load-balancer-example
Annotations: <none>
Status: Running
IP: 172.30.6.3
Controlled By: ReplicaSet/nginx-7ff779f954
Containers:
  nginx:
    Container ID: docker://df7230d3e002c53341fe559851a4d913b77aa9bff0f2d21c656ad1ed6f0bb86d
    Image: 192.168.1.192:5000/hello-world-nginx
    Image ID: docker-pullable://192.168.1.192:5000/[email protected]:583f0c9ca89415140fa80f70f8079f5138180a6dda2c3ff3920353b459e061a3
    Port: 80/TCP
    State: Running
      Started: Thu, 19 Jul 2018 17:45:55 +0800
    Ready: True
    Restart Count: 0
    Environment: <none>
    Mounts: <none>
Conditions:
  Type Status
  Initialized True
  Ready True
  PodScheduled True
Volumes: <none>
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: <none>
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal Scheduled 21m default-scheduler Successfully assigned nginx-7ff779f954-pbps4 to node1
  Normal Pulling 19m (x4 over 21m) kubelet, node1 pulling image "192.168.1.192:5000/hello-world-nginx"  # 成功拉取
  Normal Pulled 19m kubelet, node1 Successfully pulled image "192.168.1.192:5000/hello-world-nginx"
  Normal Created 19m kubelet, node1 Created container  # 已经创建
  Normal Started 19m kubelet, node1 Started container
# 查看一下端口情况
[[email protected] ~]# kubectl describe svc example-service
Name: example-service
Namespace: default
Labels: run=load-balancer-example
Annotations: <none>
Selector: run=load-balancer-example
Type: NodePort
IP: 10.254.72.252
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 30782/TCP  #下面我们使用这个端口访问以下
Endpoints: 172.30.6.2:80,172.30.6.3:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
# 下面还是在任意Node上执行的
[[email protected] ~]# netstat -tlunp | grep 30782
tcp6 1 0 :::30782 :::* LISTEN 38917/kube-proxy   #会发现node上已经监听了NodePort所指定的端口了
[[email protected] ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
df7230d3e002 192.168.1.192:5000/[email protected]:583f0c9ca89415140fa80f70f8079f5138180a6dda2c3ff3920353b459e061a3 "sh /start.sh" 17 hours ago Up 17 hours k8s_nginx_nginx-7ff779f954-pbps4_default_50953118-8b38-11e8-b3ac-000c290991f6_0
b5e225cd6abf 192.168.1.192:5000/pause-amd64 "/pause" 17 hours ago Up 17 hours k8s_POD_nginx-7ff779f954-pbps4_default_50953118-8b38-11e8-b3ac-000c290991f6_0
# 可以看到有两个容器在运行，一个是pod 一个是真正提供服务的hello-world-nginx

可以看到我们通过node的ip可以访问到我们容器里的内容的，我们想要的结果是一个容器宕掉之后马上又能重启一个继续提供访问，所以我们这里就直接删除一个pod

  • 容灾演练
# 从上面得我们有两个pods 分别是nginx-7ff779f954-9sccd 和 nginx-7ff779f954-pbps4
# 我们现在删除nginx-7ff779f954-9sccd  pod
[[email protected] ~]# kubectl delete pod nginx-7ff779f954-9sccd
pod "nginx-7ff779f954-9sccd" deleted
[[email protected] ~]# kubectl get pods  #会发现有自动创建了一个nginx-7ff779f954-clxf9 并且两个IP依然能够正常访问
NAME READY STATUS RESTARTS AGE
nginx-7ff779f954-clxf9 1/1 Running 0 4m
nginx-7ff779f954-pbps4 1/1 Running 0 34m

这次实验就到这里吧，这次各个组件的配置都比较简单，而且没有用到证书，所以就仅供"观赏"了。后面随着知识了解的深入会加一些优化配置在里面。

2.6 一些补充

  • 尝试停掉某一node上的hello-world-nginx容器（这里是node1为例）
[[email protected] ~]# docker stop df7230
df7230
[[email protected] ~]# docker ps # 再次查看容器启动的情况 发现有新的容器b06f2d805a8e已经产生
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b06f2d805a8e 192.168.1.192:5000/[email protected]:583f0c9ca89415140fa80f70f8079f5138180a6dda2c3ff3920353b459e061a3 "sh /start.sh" About an hour ago Up About an hour k8s_nginx_nginx-7ff779f954-pbps4_default_50953118-8b38-11e8-b3ac-000c290991f6_1
b5e225cd6abf 192.168.1.192:5000/pause-amd64 "/pause" 20 hours ago Up 20 hours
[[email protected] ~]# journalctl -f -t kubelet # 查看一下kubelet的日志信息（截取了一部分）
Jul 20 11:50:02 node1 kubelet[58163]: I0720 11:50:02.811077 58163 kuberuntime_manager.go:514] Container {Name:nginx Image:192.168.1.192:5000/hello-world-nginx Command:[] Args:[] WorkingDir: Ports:[{Name: HostPort:0 ContainerPort:80 Protocol:TCP HostIP:}] EnvFrom:[] Env:[] Resources:{Limits:map[] Requests:map[]} VolumeMounts:[] VolumeDevices:[] LivenessProbe:nil ReadinessProbe:nil Lifecycle:nil TerminationMessagePath:/dev/termination-log TerminationMessagePolicy:File ImagePullPolicy:Always SecurityContext:nil Stdin:false StdinOnce:false TTY:false} is dead, but RestartPolicy says that we should restart it.  # 这里就是stop后  发生的restart信息事件了（但有个问题是，虽然恢复了但是却花了好几分钟才恢复。。。）
Jul 20 11:50:02 node1 kubelet[58163]: I0720 11:50:02.811330 58163 kuberuntime_manager.go:758] checking backoff for container "nginx" in pod "nginx-7ff779f954-pbps4_default(50953118-8b38-11e8-b3ac-000c290991f6)"

这实验真是熬人。。 当我知道还有kubeadmin和minikube这样的工具可以安装集群是整个人几乎是崩溃的，不过自己从头到尾搭建一次对k8s的整个流程和模块之间的关系会更加的了解，也有利于之后的一些插件的添加，期待下一篇kube-dns的诞生。。。。

原文地址：https://www.cnblogs.com/zunwen/p/9497690.html

时间： 2024-10-09 10:55:03

k8s简单集群搭建的相关文章

.net core i上 K8S(一)集群搭建

1.前言 以前搭建集群都是使用nginx反向代理,但现在我们有了更好的选择——K8S.我不打算一上来就讲K8S的知识点,因为知识点还是比较多,我打算先从搭建K8S集群讲起,我也是在搭建集群的过程中熟悉了K8S的一些概念,希望对大家有所帮助.K8S集群的搭建难度适中,网上有很多搭建k8s的教程,我搭建的过程中或多或少遇到一些问题,现在就把我总结完的教程给大家总结一下.这里主要讲通过二进制包安装K8S 2.集群组件介绍 节点 ip 组件 master 192.168.8.201 etcd:存储集群节

k8s docker集群搭建

?Kubernetes介绍 1.背景介绍 云计算飞速发展 - IaaS - PaaS - SaaS Docker技术突飞猛进 - 一次构建,到处运行 - 容器的快速轻量 - 完整的生态环境 2.什么是kubernetes 首先,他是一个全新的基于容器技术的分布式架构领先方案.Kubernetes(k8s)是Google开源的容器集群管理系统(谷歌内部:Borg).在Docker技术的基础上,为容器化的应用提供部署运行.资源调度.服务发现和动态伸缩等一系列完整功能,提高了大规模容器集群管理的便捷性

zookeeper 简单集群搭建

1.从Apache开源官网下载(二进制版,源码版本根据需要下载)https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/ 2.下载二进制版apache-zookeeper-3.5.6-zookpeeper-bin 3.直接放到你想要安装的目录/home/virtul-cluster-zookeeper/ 4.配置zoo.cfgtickTime=2000initLimit=10syncLimit=5dataDir=/home/virtul-clu

Redis的安装配置及简单集群部署

最近针对中铁一局项目,跟事业部讨论之后需要我们的KF平台能够接入一些开源的数据库,于是这两天研究了一下Redis的原理. 1. Redis的数据存储原理及简述 1.1Redis简述 Redis是一个基于内存且支持持久化的key-value的NoSQL数据库,其中每个key和value都是使用对象表示的,具有以以下特征:多样数据类型.持久化.主从同步.它支持存储的value类型包括string(字符串).list(链表).hash(哈希).set(集合)和zset(有序集合).这些数据类型都支持p

linux 下heartbeat简单高可用集群搭建

Heartbeat 项目是 Linux-HA 工程的一个组成部分,它实现了一个高可用集群系统.通过Heartbeat我们可以实现双机热备,以实现服务的持续性. linux下基于heartbeat的简单web服务的高可用集群搭建 首先规划好两台主机作为heartbeat的双机热备,命名为node1.lvni.cc(主) ;node2.lvni.cc, node1的eth0IP :192.168.157.148  Vip eth0:0:192.168.157.149 node2的eth0IP :19

Redis集群搭建与简单使用【转】

Redis集群搭建与简单使用 安装环境与版本 用两台虚拟机模拟6个节点,一台机器3个节点,创建出3 master.3 salve 环境. redis 采用 redis-3.2.4 版本. 两台虚拟机都是 oracle linux 6.6 ,一台(IP:192.168.31.245),一台(IP:192.168.31.210) . 安装过程 1. 下载并解压 cd /root/software wget http://download.redis.io/releases/redis-3.2.4.t

Hadoop学习之路（四）Hadoop集群搭建和简单应用

概念了解 主从结构:在一个集群中,会有部分节点充当主服务器的角色,其他服务器都是从服务器的角色,当前这种架构模式叫做主从结构. 主从结构分类: 1.一主多从 2.多主多从 Hadoop中的HDFS和YARN都是主从结构,主从结构中的主节点和从节点有多重概念方式: 1.主节点 从节点 2.master slave 3.管理者 工作者 4.leader follower Hadoop集群中各个角色的名称: 服务 主节点 从节点 HDFS NameNode DataNode YARN Resource

简单开发k8s多集群后台

详细的说明如何利用kubernetes api创建一个支持多集群的k8s后台,这里主要实现调用kubernetes api获取查看service服务,pod服务.下一篇再说明如果操作一个service,以及注入容器console 相关的组件 Django (2.0.7) kubernetes (6.0.0) Django Rest Framework (3.8.2) Bootstrap 3.3.7 Jquery 2.1.1 pace 页面异步加载插件 新建一个基于django的k8s项目,命名为

和我一起打造个简单搜索之ElasticSearch集群搭建

我们所常见的电商搜索如京东,搜索页面都会提供各种各样的筛选条件,比如品牌.尺寸.适用季节.价格区间等,同时提供排序,比如价格排序,信誉排序,销量排序等,方便了用户去找到自己心里理想的商品. 站内搜索对于一个网站几乎是标配,只是搜索的强大与否的区别,有的网站只支持关键词模糊搜索,而淘宝,京东提供了精细的筛选条件,同时支持拼音搜索等更方便的搜索方式. 由于笔者在一家做网络文学的公司工作,所以实现就是以小说为商品的搜索,具体可以参考起点网小说的搜索. 如图所示,起点网的搜索提供了关键词搜索和排序条件以