docker pull 和 push 的性能分析

0. 测试环境

基本架构：没有docker index，所有操作发生在daemon和registry之间

镜像：

# docker images --tree
Warning: ‘--tree‘ is deprecated, it will be removed soon. See usage.
├─9247a3750813 Virtual Size: 85.1 MB
│ └─a92b1d9f0ca5 Virtual Size: 1.078 GB Tags: 10.180.156.6:5000/bigtest2:latest

网络环境：docker daemon和registry之间为千兆网络，带宽足够大。

1. docker push 日志分析及registry put对比测试

在客户端做一次docker push的时候，观察registry端日志，刨去无关的一些日志后，如下：

10.180.156.11 - - [02/Apr/2015:03:25:55 +0000] "GET /v2/ HTTP/1.1" 404 233 "-" "Go 1.1 package http"
10.180.156.11 - - [02/Apr/2015:03:25:55 +0000] "GET /v1/_ping HTTP/1.1" 200 2 "-" "Go 1.1 package http"
02/Apr/2015:03:25:55 +0000 DEBUG: args = {‘namespace‘: ‘library‘, ‘repository‘: u‘bigtest2‘}
02/Apr/2015:03:25:55 +0000 ERROR: <?xml version="1.0" encoding="UTF-8"?>
10.180.156.11 - - [02/Apr/2015:03:25:55 +0000] "PUT /v1/repositories/bigtest2/ HTTP/1.1" 200 2 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:25:55 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
02/Apr/2015:03:25:55 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
02/Apr/2015:03:25:55 +0000 DEBUG: api_error: Image not found
10.180.156.11 - - [02/Apr/2015:03:25:55 +0000] "GET /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/json HTTP/1.1" 404 28 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:25:55 +0000 DEBUG: api_error: Image not found
10.180.156.11 - - [02/Apr/2015:03:25:55 +0000] "GET /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/json HTTP/1.1" 404 28 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:25:55 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
10.180.156.11 - - [02/Apr/2015:03:25:56 +0000] "PUT /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/json HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
===3秒===
02/Apr/2015:03:25:57 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
===34秒===
10.180.156.11 - - [02/Apr/2015:03:26:31 +0000] "PUT /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/layer HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:26:31 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
10.180.156.11 - - [02/Apr/2015:03:26:31 +0000] "PUT /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/checksum HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:26:31 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
10.180.156.11 - - [02/Apr/2015:03:26:32 +0000] "PUT /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/json HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
===6秒===
02/Apr/2015:03:26:37 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
===325秒===
10.180.156.11 - - [02/Apr/2015:03:32:02 +0000] "PUT /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/layer HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:32:02 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
10.180.156.11 - - [02/Apr/2015:03:32:03 +0000] "PUT /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/checksum HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:32:03 +0000 DEBUG: args = {‘tag‘: u‘latest‘, ‘namespace‘: ‘library‘, ‘repository‘: u‘bigtest2‘}
02/Apr/2015:03:32:03 +0000 DEBUG: [put_tag] namespace=library; repository=bigtest2; tag=latest
10.180.156.11 - - [02/Apr/2015:03:32:03 +0000] "PUT /v1/repositories/bigtest2/tags/latest HTTP/1.1" 200 4 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:03:32:03 +0000 DEBUG: args = {‘images‘: True, ‘namespace‘: ‘library‘, ‘repository‘: u‘bigtest2‘}
10.180.156.11 - - [02/Apr/2015:03:32:03 +0000] "PUT /v1/repositories/bigtest2/images HTTP/1.1" 204 - "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"

可以发现，对于一个2层的镜像，整个push的时间可以分成几段

操作|耗时|百分比
|:—-|:—|:—
put json1|3s|0.8%
put layer1|34s|9.2%
put json2|6s|1.6%
put layer2|325s|87.6%
总耗时|371s|100%

也就是说，在做put layer1和put layer2时消耗了96.8%的时间。那么，纯的对registry服务做同样文件的PUT操作，会消耗多少时间呢？

# time curl -X PUT  -H "Transfer-Encoding: chunked" --data-binary @layer-a92b1 "http://10.180.156.6:5000/v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/layer"
true
real    0m44.571s
user    0m0.302s
sys 0m1.341s

# time curl -X PUT  -H "Transfer-Encoding: chunked" --data-binary @layer-9247a "http://10.180.156.6:5000/v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/layer"
true
real    0m4.277s
user    0m0.029s
sys 0m0.130s

2. docker pull 日志分析及registry get对比测试

类似上个节，继续做pull的日志分析

10.180.156.11 - - [02/Apr/2015:08:54:16 +0000] "GET /v2/ HTTP/1.1" 404 233 "-" "Go 1.1 package http"
10.180.156.11 - - [02/Apr/2015:08:54:16 +0000] "GET /v1/_ping HTTP/1.1" 200 2 "-" "Go 1.1 package http"
02/Apr/2015:08:54:16 +0000 DEBUG: args = {‘namespace‘: ‘library‘, ‘repository‘: u‘bigtest2‘}
10.180.156.11 - - [02/Apr/2015:08:54:16 +0000] "GET /v1/repositories/bigtest2/images HTTP/1.1" 200 152 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:54:16 +0000 DEBUG: args = {‘namespace‘: u‘library‘, ‘repository‘: u‘bigtest2‘}
02/Apr/2015:08:54:16 +0000 DEBUG: [get_tags] namespace=library; repository=bigtest2
10.180.156.11 - - [02/Apr/2015:08:54:16 +0000] "GET /v1/repositories/library/bigtest2/tags HTTP/1.1" 200 78 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:54:16 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
10.180.156.11 - - [02/Apr/2015:08:54:16 +0000] "GET /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/ancestry HTTP/1.1" 200 136 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:54:16 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
10.180.156.11 - - [02/Apr/2015:08:54:17 +0000] "GET /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/json HTTP/1.1" 200 619 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:54:17 +0000 DEBUG: args = {‘image_id‘: u‘9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f‘}
===6秒===
10.180.156.11 - - [02/Apr/2015:08:54:23 +0000] "GET /v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/layer HTTP/1.1" 200 37124367 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:54:24 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
===42秒===
02/Apr/2015:08:55:06 +0000 DEBUG: api_error: Image not found
10.180.156.11 - - [02/Apr/2015:08:55:06 +0000] "GET /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/json HTTP/1.1" 404 28 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:55:06 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
10.180.156.11 - - [02/Apr/2015:08:55:06 +0000] "GET /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/json HTTP/1.1" 200 1172 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"
02/Apr/2015:08:55:07 +0000 DEBUG: args = {‘image_id‘: u‘a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5‘}
===2秒===
10.180.156.11 - - [02/Apr/2015:08:56:08 +0000] "GET /v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/layer HTTP/1.1" 200 420084375 "-" "docker/1.5.0 go/go1.4.1 git-commit/a8a31ef kernel/3.13.0-32-generic os/linux arch/amd64"

可以发现，对于一个2层的镜像，整个pull的时间可以分成几段

操作|耗时|百分比
|:—-|:—|:—
get json1|6s|5.1%
get layer1|42s|36.2%
get json2|0s|0%
get layer2|2s|1.7%
总耗时|116s|100%

对比直接从registry上get测试

# time curl -X GET  -H "Transfer-Encoding: chunked" -o layer-9247a-down "http://10.180.156.6:5000/v1/images/9247a37508136a5fe916f4f87bfa2aa47f2edf8fc1a1ce787445b606d8fdbf9f/layer"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 35.4M  100 35.4M    0     0  25.3M      0  0:00:01  0:00:01 --:--:-- 25.3M

real    0m1.409s
user    0m0.036s
sys 0m0.233s

# time curl -X GET  -H "Transfer-Encoding: chunked" -o layer-a92b1-down "http://10.180.156.6:5000/v1/images/a92b1d9f0ca59cefd024a5f2f2d23e2d617b3e67cc2363c94998006fa87e60e5/layer"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  400M  100  400M    0     0  19.2M      0  0:00:20  0:00:20 --:--:-- 24.5M

real    0m20.771s
user    0m0.381s
sys 0m2.501s

3. 结论

从单个操作看，docker push/pull的50%以上的耗时在docker daemon，做分层、打tar包、checksum等。在这方面registry层的优化已经意义不大。

时间： 2024-10-08 12:59:15

docker pull 和 push 的性能分析的相关文章

docker-registry的定制和性能分析

docker-index Web UI Meta-data 元数据存储(附注.星级.公共库清单) 访问认证 token管理 docker-registry 存储镜像.以及镜像层的家族谱系 没有用户账户数据 不知道用户的账户和安全性 把安全和认证委托给docker-hub来做,用token来保证传递安全 不需要重新发明轮子,支持多种存储后端 没有本地数据库 后端存储 因为镜像最终是以tar.gz的方式静态存储在服务端 适用于对象存储而不是块存储 registry存储驱动 官方支持的驱动有文件.亚马

解决 Docker pull 出现的net/http: TLS handshake timeout 的一个办法

docker pull 时发生以下错误: 原因:不可描述(政府屏蔽了?) 解决思路:百度搜了下net/http: TLS handshake timeout 出现一个这个结果比较满意 http://dockone.io/article/876?utm_source=tuicool&utm_medium=referral 我不用官方的dockhub了,转而使用国内的仓库daocloud $ echo "DOCKER_OPTS=\"\$DOCKER_OPTS --registry-

Android TraceView 最权威的性能分析工具

TraceView是什么 Traceview是android平台配备一个很好的性能分析的工具.它可以通过图形化的方式让我们了解我们要跟踪的程序的性能,并且能具体到method. Traceview的作用 查看跟踪代码的执行时间,分析哪些是耗时操作 可以用于跟踪方法的调用,尤其是Android Framework层的方法调用关系 如何使用TraceView 使用TraceView主要有两种方式: 最简单的方式就是直接打开DDMS,选择一个进程,然后按上面的"Start Method Profili

正确使用Android性能分析工具&mdash;&mdash;TraceView

前面唠叨 最近公司app中有些列表在滑动的时候会有卡顿现象,我就开始着手解决这些问题,解决问题之前首先要分析列表滑动的性能瓶颈在什么地方.因为之前不会正确使用TraceView这个工具,主要是看不懂TraceView界面下方数据指标的值代表什么意思-以前我用StopWatch类来分析性能,现在觉得弱爆了-不过有些地方StopWatch工具类还是很简单好用的~ 网上可以找了很多博客来介绍这个工具的使用方法,很多都是讲解了一些一些就会的方法,讲一个大概,包括StackOverFlow上我也没有找到很

如何使用Android中TraceView性能分析工具

现来看一下整个界面的图,整个界面包括上下两部分,上面是你测试的进程中每个线程的执行情况,每个线程占一行:下面是每个方法执行的各个指标的值 上面一部分是你测试进程的中每个线程运行的时间线,下图中可以可以看到,主要只有一个main线程在执行,因为我滑动了一下列表,main线程(UI线程)正在进行绘制View呢~ 附相关视频教程Android应用开发视频教程 然后我点击了序号为133的一个方法io.bxbxbai.android.examples.activity.ExpandableLayoutMa

Android 常用的性能分析工具详解：GPU呈现模式, TraceView, Systrace, HirearchyViewer（转）

此篇将重点介绍几种常用的Android性能分析工具: 一.Logcat 日志 选取Tag＝ActivityManager,可以粗略地知道界面Displaying的时间消耗.当我们打开一个Activity的时候,log会打印一串log如下: I/ActivityManager﹕ Displayed xxx.xxx.xxx/TestActivity: +1s272ms (total +3s843ms) 第一个时间表示系统接受到打开的intent到TestActivity界面显示出来的时间1.272秒

docker pull 私有镜像

错误演示 [[email protected] jdk8]# curl http://10.20.2.29:5000/v2/_catalog {"repositories":["docker.io/jdk","docker.io/nginx"]} [[email protected] jdk8]# docker pull jdk Using default tag: latest Trying to pull repository docker.

性能分析 函数粒度 函数里的一条语句 汇编 反编译 机器指令

在Linux下做性能分析3:perf - 知乎 https://zhuanlan.zhihu.com/p/22194920 Linux Perf 性能分析工具及火焰图浅析 - 知乎 https://zhuanlan.zhihu.com/p/54276509 perf record -a -g -e cycles -e cs #系统整体采样 查看指定进程 redis-server perf report --pid 7070 mysqld perf report --pid 5634 Sample

docker：搭建ELK 开源日志分析系统

ELK 是由三部分组成的一套日志分析系统, Elasticsearch: 基于json分析搜索引擎,Elasticsearch是个开源分布式搜索引擎,它的特点有:分布式,零配置,自动发现,索引自动分片, 索引副本机制,restful风格接口,多数据源,自动搜索负载等. Logstash:动态数据收集管道,Logstash是一个完全开源的工具,它可以对你的日志进行收集.分析,并将其存储供以后使用 Kibana:可视化视图,将elasticsearh所收集的data通过视图展现.kibana 是一个