kubernetes实战-基于redis和docker的留言簿案例
[置顶] kubernetes实战-基于redis和docker的留言簿案例
2015-06-22 19:41 7570人阅读 评论(5) 收藏 举报
本文章已收录于：
Docker知识库
分类：
docker（7） 
版权声明：本文为博主原创文章，未经博主允许不得转载。
一、介绍
本案例基于Kubernetes和Docker，其中包括
1、web前端
2、redis master
3、redis slave
其中web前端通过javascript redis api和redis master交互
kubernetes体系架构
二、配置
0、先决条件
Kubernetes 集群
1、启动redis master
使用replication controller确保只有一个pod在运行（当某个节点down了，rc会在另一个健康的node启动redis master），但可能会有数据丢失。
[[email protected] example]# kubectl create -f redis-master-controller.json
replicationcontrollers/redis-master
[[email protected] example]# kubectl get rc
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
redis-master master redis name=redis-master 1
验证master运行成功，如下展示了pod运行在centos2/192.168.1.112这台机器上
[[email protected] example]# kubectl get pods
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 55 seconds
master redis Running 55 seconds
SSH到centos2/192.168.1.112查看docker状态
[[email protected] yum.repos.d]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
91689ce56668 redis:latest "/entrypoint.sh redi 3 minutes ago Up 3 minutes k8s_master.52732b08_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_97e79b7b
38c3180813c3 gcr.io/google_containers/pause:0.8.0 "/pause" 3 minutes ago Up 3 minutes k8s_POD.49eee8c2_redis-master-svar7_default_5b6d5485-1894-11e5-b3ad-000c293c8c19_298e038f
注意：kubectl create执行后，如果镜像不存在，会执行docker pull，根据网络情况，下载中的pods 在kubertnetes UI上会显示pending状态
2、启动master service
一个kubernetes service会对一个或多个container进行负载均衡，这是通过我们上面redis-master中定义的labels元数据实现的，值得注意的是，在redis中只有一个master，但是我们依然为它创建一个service，这是因为这样我们就能使用一个elastic IP来路由到具体某一个master。
kubernetes集群中的service是通过container中的环境变量实现服务发现的，service基于pod label实现container的负载均衡。
在第一步中创建的pod包含了一个label“name=redis-master”，service的selector字段决定了service将流量转发给哪个pod，port和targetPort信息定义了service proxy运行在什么端口。
[[email protected] example]# kubectl create -f redis-master-service.json
services/redis-master
[[email protected] example]# kubectl get services
NAME LABELS SELECTOR IP(S) PORT(S)
redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP
上面的运行成功后，所有pods都能发现redis master运行在6379端口，从salve到master流量走向会有以下两步：
1、一个redis slave会连接到redis master service的port上
2、流量会从service节点上的port到targetPort
如果targetPort未指定，默认和port一致
3、启动replicated slave pod
虽然redis master是一个单独的pod，redis slaves是一个replicated pod，在Kubernetes中，一个Replication Controller负责管理一个replicated pod的多个实例，RC会自动拉起down掉的replica（可以通过杀死docker 进程方式简单测试）
[[email protected] example]# kubectl create -f redis-slave-controller.json
replicationcontrollers/redis-slave
[[email protected] example]# kubectl get rc
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
redis-master master redis name=redis-master 1
redis-slave slave kubernetes/redis-slave:v2 name=redis-slave 2
[[email protected] example]# kubectl get pods
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 41 minutes
master redis Running 41 minutes
redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 29 seconds
slave kubernetes/redis-slave:v2 Running 28 seconds
redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 29 seconds
slave kubernetes/redis-slave:v2 Running 28 seconds
可以看到一个master pod和两个slave pod
4、启动slave service
和master一样，我们希望有一个代理服务连接到redis slave，除了服务发现之外，slave service还为web app client提供了透明代理。
这次service 的selector是name=redis-slave，我们可以方便的使用kubectl get services -l "label=value"命令来定位这些服务
[[email protected] example]# kubectl create -f redis-slave-service.json
services/redis-slave
[[email protected] example]# kubectl get services
NAME LABELS SELECTOR IP(S) PORT(S)
redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP
redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP
5、创建frontend pod
这是一个简单的PHP 服务，用来和master service（写请求）或slave service（读请求）交互
[[email protected] example]# kubectl create -f frontend-controller.json
replicationcontrollers/frontend
[[email protected] example]# kubectl get rc
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
frontend php-redis kubernetes/example-guestbook-php-redis:v2 name=frontend 3
redis-master master redis name=redis-master 1
redis-slave slave kubernetes/redis-slave:v2 name=redis-slave 2
运行成功后，查看当前pod运行状态
[[email protected] example]# kubectl get pods
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
frontend-fr5z1 172.17.0.13 centos2/192.168.1.112 name=frontend Running 2 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes
frontend-gjx3t 172.17.0.14 centos2/192.168.1.112 name=frontend Running 2 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes
frontend-v608r 172.17.0.12 centos2/192.168.1.112 name=frontend Running 2 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 2 minutes
redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running 53 minutes
master redis Running 53 minutes
redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 12 minutes
slave kubernetes/redis-slave:v2 Running 12 minutes
redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 12 minutes
slave kubernetes/redis-slave:v2 Running 12 minutes
可以看到一个redis master，两个redis slave和三个frontend pods
6、创建guestbook service
和其他service一样，你可以创建一个service管理frontend pods
[[email protected] example]# kubectl create -f frontend-service.json
services/frontend
[[email protected] example]# kubectl get services
NAME LABELS SELECTOR IP(S) PORT(S)
frontend name=frontend name=frontend 10.254.154.111 80/TCP
redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP
redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP
我们可以通过frontend service（10.254.154.111）访问pods，但是这个IP明显是无法在外部访问的，下一节讲解如何在外部网络访问guestbook
7、外部网络访问guestbook
kubernetes 支持两种访问暴露一个服务到外部IP地址NodePorts 和LoadBalancers，https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/services.md#external-services
另外可以查看防火墙，找到service对应的端口，如下
[[email protected] example]# kubectl get pods,services
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE
frontend-fr5z1 172.17.0.13 centos2/192.168.1.112 name=frontend Running 22 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes
frontend-gjx3t 172.17.0.14 centos2/192.168.1.112 name=frontend Running 22 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes
frontend-v608r 172.17.0.12 centos2/192.168.1.112 name=frontend Running 22 minutes
php-redis kubernetes/example-guestbook-php-redis:v2 Running 22 minutes
redis-master-svar7 172.17.0.9 centos2/192.168.1.112 name=redis-master Running About an hour
master redis Running About an hour
redis-slave-31tkb 172.17.0.10 centos2/192.168.1.112 name=redis-slave Running 32 minutes
slave kubernetes/redis-slave:v2 Running 32 minutes
redis-slave-uk8nu 172.17.0.11 centos2/192.168.1.112 name=redis-slave Running 32 minutes
slave kubernetes/redis-slave:v2 Running 32 minutes
NAME LABELS SELECTOR IP(S) PORT(S)
frontend name=frontend name=frontend 10.254.154.111 80/TCP
kubernetes component=apiserver,provider=kubernetes <none> 10.254.0.2 443/TCP
kubernetes-ro component=apiserver,provider=kubernetes <none> 10.254.0.1 80/TCP
redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP
redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP
[[email protected] example]# kubectl get services
NAME LABELS SELECTOR IP(S) PORT(S)
frontend name=frontend name=frontend 10.254.154.111 80/TCP
kubernetes component=apiserver,provider=kubernetes <none> 10.254.0.2 443/TCP
kubernetes-ro component=apiserver,provider=kubernetes <none> 10.254.0.1 80/TCP
redis-master name=redis-master name=redis-master 10.254.154.90 6379/TCP
redis-slave name=redis-slave name=redis-slave 10.254.159.145 6379/TCP
发现redis-master是在10.254.154.90上，登录到centos2上，执行iptables-save，发现其中有这样一条规则
-A KUBE-PORTALS-HOST -d 10.254.154.90/32 -p tcp -m comment --comment "default/redis-master:" -m tcp --dport 6379 -j DNAT --to-destination 192.168.1.112:49038
说明，centos2本机的49038端口映射到master container内的6379了，当然我们就能执行在本机访问redis了
[[email protected] yum.repos.d]# redis-cli -p 49038
127.0.0.1:49038> set a b
OK
127.0.0.1:49038> get a
"b"
8、使用curl简单测试
提交数据
curl "localhost:8000/index.php?cmd=set&key=messages&value=jay_sais_hi"
查询数据
curl "localhost:8000/index.php?cmd=get&key=messages"
附本案例用到的6个.json文件
1、redis-master-controller.json
{
"kind":"ReplicationController",
"apiVersion":"v1beta3",
"metadata":{
"name":"redis-master",
"labels":{
"name":"redis-master"
}
},
"spec":{
"replicas":1,
"selector":{
"name":"redis-master"
},
"template":{
"metadata":{
"labels":{
"name":"redis-master"
}
},
"spec":{
"containers":[
{
"name":"master",
"image":"redis",
"ports":[
{
"containerPort":6379
}
]
}
]
}
}
}
}
2、redis-master-service.json
{
"kind":"Service",
"apiVersion":"v1beta3",
"metadata":{
"name":"redis-master",
"labels":{
"name":"redis-master"
}
},
"spec":{
"ports": [
{
"port":6379,
"targetPort":6379
}
],
"selector":{
"name":"redis-master"
}
}
}
3、redis-slave-controller.json
{
"kind":"ReplicationController",
"apiVersion":"v1beta3",
"metadata":{
"name":"redis-slave",
"labels":{
"name":"redis-slave"
}
},
"spec":{
"replicas":2,
"selector":{
"name":"redis-slave"
},
"template":{
"metadata":{
"labels":{
"name":"redis-slave"
}
},
"spec":{
"containers":[
{
"name":"slave",
"image":"kubernetes/redis-slave:v2",
"ports":[
{
"containerPort":6379
}
]
}
]
}
}
}
}
4、redis-slave-service.json
{
"kind":"Service",
"apiVersion":"v1beta3",
"metadata":{
"name":"redis-slave",
"labels":{
"name":"redis-slave"
}
},
"spec":{
"ports": [
{
"port":6379
}
],
"selector":{
"name":"redis-slave"
}
}
}
5、frontend-controller.json
{
"kind":"ReplicationController",
"apiVersion":"v1beta3",
"metadata":{
"name":"frontend",
"labels":{
"name":"frontend"
}
},
"spec":{
"replicas":3,
"selector":{
"name":"frontend"
},
"template":{
"metadata":{
"labels":{
"name":"frontend"
}
},
"spec":{
"containers":[
{
"name":"php-redis",
"image":"kubernetes/example-guestbook-php-redis:v2",
"ports":[
{
"containerPort":80
}
]
}
]
}
}
}
}
6、frontend-service.json
{
"kind":"Service",
"apiVersion":"v1beta3",
"metadata":{
"name":"frontend",
"labels":{
"name":"frontend"
}
},
"spec":{
"ports": [
{
"port":80
}
],
"selector":{
"name":"frontend"
}
}
}