【tensorflow2.0】使用tensorflow-serving部署模型

TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。

例如：通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。

通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。

通过 tensorflow-serving 可以加载模型后提供网络接口API服务，通过任意编程语言发送网络请求都可以获取模型预测结果。

通过 tensorFlow for Java接口，可以在Java或者spark(scala)中调用tensorflow模型进行预测。

我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法

〇，tensorflow serving模型部署概述

使用 tensorflow serving 部署模型要完成以下步骤。

  • (1) 准备protobuf模型文件。
  • (2) 安装tensorflow serving。
  • (3) 启动tensorflow serving 服务。
  • (4) 向API服务发送请求，获取预测结果。

可通过以下colab链接测试效果《tf_serving》： https://colab.research.google.com/drive/1vS5LAYJTEn-H0GDb1irzIuyRB8E3eWc8

%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras import *

一，准备protobuf模型文件

我们使用tf.keras 训练一个简单的线性回归模型，并保存成protobuf文件。

import tensorflow as tf
from tensorflow.keras import models,layers,optimizers

## 样本数量
n = 800

## 生成测试用数据集
X = tf.random.uniform([n,2],minval=-10,maxval=10)
w0 = tf.constant([[2.0],[-1.0]])
b0 = tf.constant(3.0)

Y = [email protected] + b0 + tf.random.normal([n,1],
    mean = 0.0,stddev= 2.0) # @表示矩阵乘法,增加正态扰动

## 建立模型
tf.keras.backend.clear_session()
inputs = layers.Input(shape = (2,),name ="inputs") #设置输入名字为inputs
outputs = layers.Dense(1, name = "outputs")(inputs) #设置输出名字为outputs
linear = models.Model(inputs = inputs,outputs = outputs)
linear.summary()

## 使用fit方法进行训练
linear.compile(optimizer="rmsprop",loss="mse",metrics=["mae"])
linear.fit(X,Y,batch_size = 8,epochs = 100)  

tf.print("w = ",linear.layers[1].kernel)
tf.print("b = ",linear.layers[1].bias)

## 将模型保存成pb格式文件
export_path = "./data/linear_model/"
version = "1"       #后续可以通过版本号进行模型版本迭代与管理
linear.save(export_path+version, save_format="tf")
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
inputs (InputLayer)          [(None, 2)]               0
_________________________________________________________________
outputs (Dense)              (None, 1)                 3
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0
_________________________________________________________________
Epoch 1/100
100/100 [==============================] - 0s 2ms/step - loss: 273.0472 - mae: 13.9096
Epoch 2/100
100/100 [==============================] - 0s 2ms/step - loss: 250.0846 - mae: 13.3155
Epoch 3/100
100/100 [==============================] - 0s 2ms/step - loss: 228.0106 - mae: 12.7211
Epoch 4/100
100/100 [==============================] - 0s 2ms/step - loss: 208.5060 - mae: 12.1514
Epoch 5/100
100/100 [==============================] - 0s 2ms/step - loss: 188.6825 - mae: 11.5647
Epoch 6/100
100/100 [==============================] - 0s 2ms/step - loss: 170.6377 - mae: 10.9862
Epoch 7/100
100/100 [==============================] - 0s 2ms/step - loss: 153.1913 - mae: 10.4133
Epoch 8/100
100/100 [==============================] - 0s 2ms/step - loss: 137.3440 - mae: 9.8525
Epoch 9/100
100/100 [==============================] - 0s 2ms/step - loss: 122.1956 - mae: 9.2907
Epoch 10/100
100/100 [==============================] - 0s 2ms/step - loss: 108.5923 - mae: 8.7439
Epoch 11/100
100/100 [==============================] - 0s 2ms/step - loss: 94.8144 - mae: 8.1773
Epoch 12/100
100/100 [==============================] - 0s 2ms/step - loss: 83.0037 - mae: 7.6339
Epoch 13/100
100/100 [==============================] - 0s 2ms/step - loss: 71.8595 - mae: 7.1003
Epoch 14/100
100/100 [==============================] - 0s 2ms/step - loss: 61.8016 - mae: 6.5690
Epoch 15/100
100/100 [==============================] - 0s 2ms/step - loss: 52.5519 - mae: 6.0456
Epoch 16/100
100/100 [==============================] - 0s 2ms/step - loss: 44.4070 - mae: 5.5431
Epoch 17/100
100/100 [==============================] - 0s 2ms/step - loss: 37.0890 - mae: 5.0457
Epoch 18/100
100/100 [==============================] - 0s 2ms/step - loss: 30.6758 - mae: 4.5701
Epoch 19/100
100/100 [==============================] - 0s 2ms/step - loss: 25.1626 - mae: 4.1214
Epoch 20/100
100/100 [==============================] - 0s 2ms/step - loss: 20.3433 - mae: 3.6880
Epoch 21/100
100/100 [==============================] - 0s 2ms/step - loss: 16.3199 - mae: 3.2814
Epoch 22/100
100/100 [==============================] - 0s 2ms/step - loss: 13.1249 - mae: 2.9330
Epoch 23/100
100/100 [==============================] - 0s 2ms/step - loss: 10.4714 - mae: 2.6117
Epoch 24/100
100/100 [==============================] - 0s 2ms/step - loss: 8.5397 - mae: 2.3433
Epoch 25/100
100/100 [==============================] - 0s 2ms/step - loss: 7.0484 - mae: 2.1351
Epoch 26/100
100/100 [==============================] - 0s 2ms/step - loss: 6.1226 - mae: 1.9872
Epoch 27/100
100/100 [==============================] - 0s 2ms/step - loss: 5.6070 - mae: 1.9047
Epoch 28/100
100/100 [==============================] - 0s 2ms/step - loss: 5.2954 - mae: 1.8510
Epoch 29/100
100/100 [==============================] - 0s 2ms/step - loss: 5.0835 - mae: 1.8137
Epoch 30/100
100/100 [==============================] - 0s 2ms/step - loss: 4.9148 - mae: 1.7841
Epoch 31/100
100/100 [==============================] - 0s 2ms/step - loss: 4.7715 - mae: 1.7581
Epoch 32/100
100/100 [==============================] - 0s 2ms/step - loss: 4.6395 - mae: 1.7303
Epoch 33/100
100/100 [==============================] - 0s 2ms/step - loss: 4.5205 - mae: 1.7106
Epoch 34/100
100/100 [==============================] - 0s 1ms/step - loss: 4.4232 - mae: 1.6903
Epoch 35/100
100/100 [==============================] - 0s 2ms/step - loss: 4.3417 - mae: 1.6738
Epoch 36/100
100/100 [==============================] - 0s 1ms/step - loss: 4.2691 - mae: 1.6579
Epoch 37/100
100/100 [==============================] - 0s 2ms/step - loss: 4.2078 - mae: 1.6470
Epoch 38/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1606 - mae: 1.6381
Epoch 39/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1203 - mae: 1.6292
Epoch 40/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0847 - mae: 1.6230
Epoch 41/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0589 - mae: 1.6182
Epoch 42/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0382 - mae: 1.6141
Epoch 43/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0188 - mae: 1.6109
Epoch 44/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0089 - mae: 1.6098
Epoch 45/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9979 - mae: 1.6075
Epoch 46/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9891 - mae: 1.6055
Epoch 47/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9848 - mae: 1.6053
Epoch 48/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9806 - mae: 1.6044
Epoch 49/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9752 - mae: 1.6037
Epoch 50/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9739 - mae: 1.6038
Epoch 51/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9712 - mae: 1.6024
Epoch 52/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9690 - mae: 1.6024
Epoch 53/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9685 - mae: 1.6021
Epoch 54/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9667 - mae: 1.6021
Epoch 55/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6009
Epoch 56/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6019
Epoch 57/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6016
Epoch 58/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6012
Epoch 59/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9655 - mae: 1.6018
Epoch 60/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6016
Epoch 61/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6010
Epoch 62/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6017
Epoch 63/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6021
Epoch 64/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6019
Epoch 65/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6027
Epoch 66/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6013
Epoch 67/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9645 - mae: 1.6019
Epoch 68/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6024
Epoch 69/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 70/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6022
Epoch 71/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9626 - mae: 1.6022
Epoch 72/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9632 - mae: 1.6015
Epoch 73/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6023
Epoch 74/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6017
Epoch 75/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6003
Epoch 76/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6022
Epoch 77/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9624 - mae: 1.6023
Epoch 78/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6019
Epoch 79/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6019
Epoch 80/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6018
Epoch 81/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9649 - mae: 1.6025
Epoch 82/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9631 - mae: 1.6021
Epoch 83/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6020
Epoch 84/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6020
Epoch 85/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6014
Epoch 86/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6017
Epoch 87/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
Epoch 88/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6016
Epoch 89/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6016
Epoch 90/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9636 - mae: 1.6019
Epoch 91/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6029
Epoch 92/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6026
Epoch 93/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6014
Epoch 94/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9623 - mae: 1.6019
Epoch 95/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 96/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9641 - mae: 1.6017
Epoch 97/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6027
Epoch 98/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6024
Epoch 99/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6021
Epoch 100/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
w =  [[1.99997306]
 [-1.01220131]]
b =  [2.88236618]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Assets written to: ./data/linear_model/1/assets
# 查看保存的模型文件
!ls {export_path+version}

assets saved_model.pb variables

# 查看模型文件相关信息
!saved_model_cli show --dir {export_path+str(version)} --all
MetaGraphDef with tag-set: ‘serve‘ contains the following SignatureDefs:

signature_def[‘__saved_model_init_op‘]:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs[‘__saved_model_init_op‘] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def[‘serving_default‘]:
  The given SavedModel SignatureDef contains the following input(s):
    inputs[‘inputs‘] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: serving_default_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs[‘outputs‘] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0413 05:10:30.262132 140384690243456 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.

Defined Functions:
  Function Name: ‘__call__‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None

  Function Name: ‘_default_save_signature‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)

  Function Name: ‘call_and_return_all_conditional_losses‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None

二，安装 tensorflow serving

安装 tensorflow serving 有2种主要方法：通过Docker镜像安装，通过apt安装。

通过Docker镜像安装是最简单，最直接的方法，推荐采用。

Docker可以理解成一种容器，其上面可以给各种不同的程序提供独立的运行环境。

一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.

无需算法工程师同学动手安装，以下安装过程仅供参考。

不同操作系统机器上安装Docker的方法可以参照以下链接。

Windows: https://www.runoob.com/docker/windows-docker-install.html

MacOs: https://www.runoob.com/docker/macos-docker-install.html

CentOS: https://www.runoob.com/docker/centos-docker-install.html

安装Docker成功后，使用如下命令加载 tensorflow/serving 镜像到Docker中

docker pull tensorflow/serving

三，启动 tensorflow serving 服务

!docker run -t --rm -p 8501:8501     -v "/Users/.../data/linear_model/"     -e MODEL_NAME=linear_model     tensorflow/serving & >server.log 2>&1

四，向API服务发送请求

可以使用任何编程语言的http功能发送请求，下面示范linux的 curl 命令发送请求，以及Python的requests库发送请求。

!curl -d ‘{"instances": [1.0, 2.0, 5.0]}‘     -X POST http://localhost:8501/v1/models/linear_model:predict
{
    "predictions": [[3.06546211], [5.01313448]
    ]
}
import json,requests

data = json.dumps({"signature_name": "serving_default", "instances": [[1.0, 2.0], [5.0,7.0]]})
headers = {"content-type": "application/json"}
json_response = requests.post(‘http://localhost:8501/v1/models/linear_model:predict‘,
        data=data, headers=headers)
predictions = json.loads(json_response.text)["predictions"]
print(predictions)

参考：

开源电子书地址：https://lyhue1991.github.io/eat_tensorflow2_in_30_days/

GitHub 项目地址：https://github.com/lyhue1991/eat_tensorflow2_in_30_days

原文地址：https://www.cnblogs.com/xiximayou/p/12690757.html

时间： 2024-10-08 18:03:30

【tensorflow2.0】使用tensorflow-serving部署模型的相关文章

YOLOV3——PyTorch训练TensorFlowLite部署模型转换

主要思路 将训练好的.pt文件转换为keras的.h5文件,再将.h5文件转换为.tflite文件. 步骤: 1.环境:PyTorch1.0以上(其他版本也可以,主要是适配下面的开源代码) .pt转weights下载后在文件内添加pt_to_weights.py. from models import * model = Darknet("cfg/yolov3-obj.cfg")#你用于训练的cfg文件 load_darknet_weights(model,"weights/

Tensorflow serving的编译

Tensorflow serving提供了部署tensorflow生成的模型给线上服务的方法,包括模型的export,load等等. 安装参考这个 https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md ? ? 但是由于被qiang的问题 (googlesource无法访问) https://github.com/tensorflow/serving/issues/6 需要修改一下 W

tensorflow serving 之minist_saved_model.py解读

最近在学习tensorflow serving,但是就这样平淡看代码可能觉得不能真正思考,就想着写个文章看看,自己写给自己的,就像自己对着镜子演讲一样,写个文章也像自己给自己讲课,这样思考的比较深,学到的也比较多,有错欢迎揪出, minist_saved_model.py 是tensorflow的第一个例子,里面有很多serving的知识,还不了解,现在看.下面是它的入口函数,然后直接跳转到main if __name__ == '__main__': tf.app.run() 在main函数里

记录二：tensorflow2.0写MNIST手写体

最近学习神经网络,tensorflow,看了好多视频,查找了好多资料,感觉东西都没有融入自己的思维中.今天用tensorflow2.0写了一个MNIST手写体的版本,记录下学习的过程. 复现手写体识别的基本步骤:准备数据,处理数据,搭建模型,迭代训练模型,使用模型. 一.全连接模型 1.导入数据集(此次编写运行均在notebook) 先引入所需模块,在用tf.keras下载数据 2.分析图片的大小 1)用shape查看数量大小:2)通过指定[0][1]来查看具体图像尺寸,lable存储的是ima

『TensorFlow2.0正式版教程』极简安装TF2.0正式版（CPU&GPU）教程

0 前言 TensorFlow 2.0,今天凌晨,正式放出了2.0版本. 不少网友表示,TensorFlow 2.0比PyTorch更好用,已经准备全面转向这个新升级的深度学习框架了. ? 本篇文章就带领大家用最简单地方式安装TF2.0正式版本(CPU与GPU),由我来踩坑,方便大家体验正式版本的TF2.0. 废话不多说现在正式开始教程. 1 环境准备 我目前是在Windows10上面,使用conda管理的python环境,通过conda安装cuda与cudnn(GPU支持),通过pip安装的t

tensorflow2.0新特性

Tensorflow2.0相比于以往版本,有着极大的区别:最明显的区别可以用三字词来概括:更简单,更易用,更强大. 接下来让我们一起见证下不一样的地方吧! 一.使用tf.data加载数据 使用tf.data创建的输入管道读取训练数据:支持从内存(Numpy)方便地输入数据: 二.使用tf.keras构建,训练和验证模型,或使用Premade来验证模型 可以直接标准的打包模型(逻辑回归,随机森林),也可以直接使用(tf.estimator API) 如果不想从头训练模型,可以使用迁移学习来训练一个

colab上基于tensorflow2.0的BERT中文多分类

bert模型在tensorflow1.x版本时,也是先发布的命令行版本,随后又发布了bert-tensorflow包,本质上就是把相关bert实现封装起来了. tensorflow2.0刚刚在2019年10月左右发布,谷歌也在积极地将之前基于tf1.0的bert实现迁移到2.0上,但近期看还没有完全迁移完成,所以目前还没有基于tf2.0的bert安装包面世,因为近期想基于现有发布的模型做一个中文多分类的事情,所以干脆就弄了个基于命令行版本的.过程中有一些坑,随之记录下来. 1. colab:因为

【tensorflow2.0】处理结构化数据-titanic生存预测

1.准备数据 import numpy as np import pandas as pd import matplotlib.pyplot as plt import tensorflow as tf from tensorflow.keras import models,layers dftrain_raw = pd.read_csv('./data/titanic/train.csv') dftest_raw = pd.read_csv('./data/titanic/test.csv')

【tensorflow2.0】处理图片数据-cifar2分类

1.准备数据 cifar2数据集为cifar10数据集的子集,只包括前两种类别airplane和automobile. 训练集有airplane和automobile图片各5000张,测试集有airplane和automobile图片各1000张. cifar2任务的目标是训练一个模型来对飞机airplane和机动车automobile两种图片进行分类. 我们准备的Cifar2数据集的文件结构如下所示. 在tensorflow中准备图片数据的常用方案有两种,第一种是使用tf.keras中的Ima