docker一键部署hadoop心得(二)

今天在运行MapReduce程序时,虽然wordcount实例运行成功了,但后面出现了重新使用历史服务器失败的错误

17/12/22 13:33:19 INFO ipc.Client: Retrying connect to server: hadoop-slave1/172.18.0.11:45463. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)17/12/22 13:33:20 INFO ipc.Client: Retrying connect to server: hadoop-slave1/172.18.0.11:45463. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)17/12/22 13:33:21 INFO ipc.Client: Retrying connect to server: hadoop-slave1/172.18.0.11:45463. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)17/12/22 13:33:28 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server17/12/22 13:33:30 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:31 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:32 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:33 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:34 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:35 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:36 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:37 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:38 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:39 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:40 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server17/12/22 13:33:41 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:42 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:43 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:44 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:45 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:46 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:47 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:48 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:49 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:50 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:50 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server17/12/22 13:33:51 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:52 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:53 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:54 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:55 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:56 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:57 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:58 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:33:59 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)17/12/22 13:34:00 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)java.io.IOException: java.net.ConnectException: Call From hadoop-master/172.18.0.10 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused    at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:344)    at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:429)    at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:601)    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:323)    at org.apache.hadoop.mapreduce.Job$1.run(Job.java:320)    at java.security.AccessController.doPrivileged(Native Method)    at javax.security.auth.Subject.doAs(Subject.java:422)    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)    at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:320)    at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:616)    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1323)    at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)    at java.lang.reflect.Method.invoke(Method.java:498)    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)    at java.lang.reflect.Method.invoke(Method.java:498)    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: java.net.ConnectException: Call From hadoop-master/172.18.0.10 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)    at org.apache.hadoop.ipc.Client.call(Client.java:1480)    at org.apache.hadoop.ipc.Client.call(Client.java:1413)    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)    at com.sun.proxy.$Proxy15.getJobReport(Unknown Source)    at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)    at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)    at java.lang.reflect.Method.invoke(Method.java:498)    at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:325)    ... 24 moreCaused by: java.net.ConnectException: Connection refused    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:615)    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)    at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376)    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)    at org.apache.hadoop.ipc.Client.call(Client.java:1452)    ... 32 more?

原文地址:https://www.cnblogs.com/lijinze-tsinghua/p/8331528.html

时间: 2025-01-07 21:42:17

docker一键部署hadoop心得(二)的相关文章

RabbitMQ集群 Docker一键部署

以下内容来自网络转载 步骤1. 安装docker 以centos7为例,https://docs.docker.com/engine/installation/linux/centos/ 步骤2. 创建节点互联基础 #创建Docker私有网络 docker network create rabbitmqnet 步骤3. 创建节点 #使用Docker Networking,分别创建三个节点 docker run -d --name=rabbitmq1 -p 5672:5672 -p 15672:1

Docker 下部署hadoop集群

原文地址:https://www.cnblogs.com/leolzi/p/12166121.html

Docker部署Hadoop集群

一.主机规划 3台主机:1个master.2个slaver/worker ip地址使用docker默认的分配地址: master: 主机名: hadoop2.ip地址: 172.17.0.2 slaver1: 主机名: hadoop3.ip地址: 172.17.0.3 主机名: hadoop4.ip地址: 172.17.0.4 二.软件安装 1.在docker中安装centos镜像,并启动centos容器,安装ssh.--详见"docker上安装centos镜像"一文. 2.通过ssh

Docker Compose 一键部署Nginx代理Tomcat集群

Docker Compose 一键部署Nginx代理Tomcat集群 目录结构 [[email protected] ~]# tree compose_nginx_tomcat/ compose_nginx_tomcat/ ├── docker-compose.yml ├── mysql │   ├── conf │   │   └── my.cnf │   └── data ├── nginx │   ├── Dockerfile │   ├── nginx-1.12.1.tar.gz │  

8天入门docker系列 —— 第五天 使用aspnetcore小案例熟悉容器互联和docker-compose一键部署

原文:8天入门docker系列 -- 第五天 使用aspnetcore小案例熟悉容器互联和docker-compose一键部署 这一篇继续完善webnotebook,如果你读过上一篇的内容,你应该知道怎么去挂载webnotebook日志和容器的远程访问,但是这些还远不够,webnotebook 总要和一些数据库打交道吧,比如说mysql,mongodb,redis,通常情况下这些存储设备要么是以容器的方式承载,要么是由DBA在非容器环境下统一管理. 一:webnotebook连接容器redis

docker环境安装及springboot+docker在idea上实现一键部署远程服务器

以下均是在虚拟机上操作的环境 一.docker安装 1.检查内核版本,必须是3.10及以上(docker要求centos系统的内核版本高于3.10) uname ‐r #如果系统版本较低,则使用yum update更新内核 2.安装docker yum install docker 3.输入y确认安装 4.启动docker [[email protected] ~]# systemctl start docker [[email protected] ~]# docker ‐v Docker v

如何在Docker中部署DzzOffice

一.一些背景 之前研究Docker很久了,并且在公司内部实际使用起来了,目前分两种场景使用Docker 1.作为PAAS,提供一致,统一的编译/测试环境: 2.作为虚拟机,直接分配给新来的开发人员使用. 使用Docker一段时间之后,准备在公司内部使用Docker + Hadoop(HDFS)组件内部一个私有云,统一管理公司和产品/项目组内部的代码.文档. 我选择先从文档管理开始,在底层架子搭建好之后,需要给内部人员提供一个好的"门面",在没有这个门面之前,我们大量使用SSH+Comm

使用docker-compose 一键部署你的分布式调用链跟踪框架skywalking

原文:使用docker-compose 一键部署你的分布式调用链跟踪框架skywalking 一旦你的程序docker化之后,你会遇到各种问题,比如原来采用的本地记日志的方式就不再方便了,虽然你可以挂载到宿主机,但你使用 --scale 的话,会导致 记录日志异常,所以最好的方式还是要做日志中心化,另一个问题,原来一个请求在一个进程中的痉挛失败,你可以在日志中巡查出调用堆栈,但是docker化之后, 原来一个进程的东西会拆成几个微服务,这时候最好就要有一个分布式的调用链跟踪,类似于wcf中的sv

宜信开源|Davinci一键部署:如何三句代码跑起Davinci

导读:之前喜欢Davinci的小伙伴儿在安装部署Davinci遇见问题时需要在github issue区等待技术人员的解答.现在不用怕啦,社区热心用户白菜君帮我们支持了docker-composer一键启动,以后只需寥寥几行代码,Davinci就能舒畅的run起来了.还等什么,赶紧部署起来吧~ 敲重点 Davinci Docker原部署教程在这里: https://github.com/edp963/davinci-docker 里面会不定时更新 记得收藏啊!! 下面是部署教程: 一.环境要求