Tensorflow serving with Kubernetes

1.Build docker image

由于自己build镜像总是遇到问题，此处暂时借用dockerhub上的一个镜像 docker.io/mochin/tensorflow-serving

将此镜像push到k8s所在集群的docker registry中

2.编写yaml

官方例子中给出了一个yaml不过有些地方不对，或者不适用这个dockerimage(可能是因为0.4.0的版本)

做出了一些修改

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: inception-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: inception-server
    spec:
      containers:
      - name: inception-container
        image: registry.lenovo.com:8080/tensorflow-serving:0.4.0
        command:
        - /bin/sh
        - -c
        args:
        - serving/bazel-bin/tensorflow_serving/example/inception_inference --port=9900 serving/inception-export
        ports:
        - containerPort: 9900
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: inception-service
  name: inception-service
spec:
  ports:
  - port: 9900
    targetPort: 9900
  selector:
    app: inception-server
  #type: LoadBalancer
  externalIPs:
    - xxx.xxx.xxx.xxx

3. 提交到k8s

kubectl create -f xxx.yaml

4. 查看状态

#>kubectl get service
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
inception-service xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx   9900/TCP 8m

#>kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
inception-deployment 2 2 2 2 3h

5. 进行测试

运行一个docker container去访问部署的服务

docker run --rm -it tensorflow-serving:latest /bin/bash

#>cd serving

#> ./bazel-bin/tensorflow_serving/example/inception_client --server=10.100.208.54:9900 --image=./tensorflow/tensorflow/examples/label_image/data/grace_hopper.jpg &> log &

7.863746 : military uniform
6.911478 : bow tie, bow-tie, bowtie
6.642433 : mortarboard
5.758826 : suit, suit of clothes
5.614463 : academic gown, academic robe, judge‘s robe

6. 负载均衡配置

TODO