hadoop动态添加datanode启动失败的经验

动态添加datanode节点，主机名node14.cn
shell>hadoop-daemon.sh start datanode
shell>jps #查看datanode进程是否已启动
发现DataNode进程启动后立即消失，查询日志发现一下记录：

2018-04-15 00:08:43,158 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2018-04-15 00:08:43,168 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode []
2018-04-15 00:08:43,673 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2018-04-15 00:08:43,837 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2018-04-15 00:08:43,837 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started
2018-04-15 00:08:43,839 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS is node11.cn:9000
2018-04-15 00:08:44,138 WARN org.apache.hadoop.fs.FileSystem: "node11.cn:9000" is a deprecated filesystem name. Use "hdfs://node11.cn:9000/" instead.
2018-04-15 00:08:44,196 INFO org.apache.hadoop.hdfs.DFSUtil: Starting Web-server for hdfs at: http://node11.cn:9001
2018-04-15 00:08:44,266 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2018-04-15 00:08:44,273 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.namenode is not defined
2018-04-15 00:08:44,293 INFO org.apache.hadoop.http.HttpServer2: Added global filter ‘safety‘ (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2018-04-15 00:08:44,298 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2018-04-15 00:08:44,298 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2018-04-15 00:08:44,298 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2018-04-15 00:08:44,374 INFO org.apache.hadoop.http.HttpServer2: Added filter ‘org.apache.hadoop.hdfs.web.AuthFilter‘ (class=org.apache.hadoop.hdfs.web.AuthFilter)
2018-04-15 00:08:44,377 INFO org.apache.hadoop.http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resource
s, pathSpec=/webhdfs/v1/*
2018-04-15 00:08:44,411 INFO org.apache.hadoop.http.HttpServer2: HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: node11.cn:9001
        at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:892)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:828)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:706)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:593)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:765)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:749)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1446)
                Caused by: java.net.BindException: Cannot assign requested address
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
        at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:887)
        ... 8 more
2018-04-15 00:08:44,414 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...
2018-04-15 00:08:44,415 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.

2018-04-15 00:08:44,415 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2018-04-15 00:08:44,415 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.

java.net.BindException: Port in use: node11.cn:9001
        at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:892)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:828)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:706)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:593)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:765)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:749)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1446)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1512)
Caused by: java.net.BindException: Cannot assign requested address
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:887)
        ... 8 more
2018-04-15 00:08:44,423 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2018-04-15 00:08:44,426 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node14.cn/192.168.74.114
************************************************************/

解决方式：
删除dfs目录下的内容重新执行一下命令即可
shell>rm -rf dfs/
shell>hadoop-daemon.sh start datanode
shell>yarn-daemon.sh start nodemanager

刷新nanenode节点
shell>hdfs dfsadmin -refreshNodes
shell>start-balancer.sh
新增datanode成功，
将数据分发到新增datanode节点主机上
shell>hadoop balancer -threshold 10 #50控制磁盘使用率的参数，数值越小，各个节点磁盘使用率越均衡

原文地址：http://blog.51cto.com/maoxiaoxiong/2103543

时间： 2024-12-27 16:22:04

hadoop动态添加datanode启动失败的经验的相关文章

Hadoop动态添加/删除节点（datanode和tacktracker）

总的来说,正确的做法是优先通过配置文件,再在具体机器上进行相应进程的启动/停止操作. 网上一些资料说在调整配置文件的时候,优先使用主机名而不是IP进行配置. 总的来说添加/删除DataNode和TaskTracker的方法非常相似,只是操作的配置项和使用的命令有微小差异. 1. DataNode 1.0 配置文件 在master/namenode下修改配置文件conf/mapred-site.xml. 关键参数dfs.hosts和dfs.hosts.exclude. 注意:不同hadoop版本的

hadoop datanode启动失败

问题导读: 1.Hadoop出现问题时,该如何入手查看问题?2.datanode无法启动,我们该怎么解决?3.如何动态加入DataNode或TaskTracker? 一.问题描述当我多次格式化文件系统时,如 [email protected]:/usr/local/hadoop-1.0.2# bin/hadoop namenode -format 复制代码 会出现datanode无法启动,查看日志,发现错误为: 2012-04-20 20:39:46,501 ERROR org.apache.h

[Nutch]Hadoop动态增加DataNode节点和TaskTracker节点

在之前的博文有对分布式模式做负载均衡设置,那么这次我们就来说明一下如果动态增加DataNode结点和TaskTracker结点. 1. 在host1上增加host4 使用如下命令: vi conf/slaves 如下: 2. 将host1上的公钥加入到host4中 使用如下命令: ssh-copy-id -i .ssh/id_rsa.pub [email protected] 如下: 将host1的公钥添加到host4中,可以免密码登录.如下: 3. 将hadoop拷贝到host4上 使用如下命

datanode启动失败

当我动态添加一个hadoop从节点的之后,出现了一个问题: [[email protected] current]# hadoop-daemon.sh start datanode starting datanode, logging to /usr/local/hadoop1.1/libexec/../logs/hadoop-root-datanode-hadoop.out [[email protected] ~]# jps jps命令发现没有datanode启动,所以去它提示的路径下查看了

hdfs datanode 启动失败

hadoop-root-datanode-ubuntu.log中: 2015-03-12 23:52:33,671 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:9000. Exiting.java.io.IOExc

hadoop问题总结:datanode启动后,在web50070端口发现不到datanode节点(能力工场)

直接上问题:这两天为了试验,安装了两套集群: (1)32位hadoop1集群(5个节点); (2)64位hadoop2集群(6个节点) 两个集群中都遇到过这样的问题:在namenode正常启动hadoop集群后,查看datanode是正常的显示存在进程,但是在web界面中查看的时候,发现数据节点全部属于宕机,或者就是干脆直接没有datanode.还有种情况,datanode启动后,jps查看是在的,但是一会再去查看,发现挂掉了.还有就是,存储空间显示占用100% 其实这两个集群问题是一样的,都是

[Nutch]Hadoop动态删除DataNode节点和TaskTracker节点

在之前的博文有说明动态添加一个结点,本次就说明如何动态删除一个结点. 在上一篇博文有讲解如何限制一个结点的连接,要动态删除一个结点,可以在这个基础之上进行配置. 1. 在host1上配置dfs.hosts.exclude 在dfs.hosts.exclude指定的文件中添加host4: 再执行如下命令: hadoop dfsadmin -refreshNodes 然后用如下命令进行查看: hadoop dfsadmin -report 如下: 2. web界面 从web管理界面可以看到,目前只有

启动Hadoop时，DataNode启动后一会儿自动消失的解决方法

查看slaver1/2的logs,发现 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:9000java.io.IOException: Incompatible clusterIDs in /u

Hadoop 数据节点DataNode异常

转载自 :http://blog.csdn.net/gis_101/article/details/52679914 1.问题背景 操作系统:CentOS7 64位 Hadoop版本:Hadoop2.7.3 JDK:1.8.0_65 2.问题描述 使用start-all.sh脚本或者先后执行start-dfs.sh.start-yarn.sh脚本启动Hadoop后,通过jps命令检测DataNode节点进程发现有DataNode进程,数秒后DataNode进程莫名其妙"消失"了,只有J