Hadoop启动没有namenode进程

Hadoop问题：启动hadoop时报namenode未初始化：java.io.IOException: NameNode is not formatted.
1、启动Hadoop

[email protected]:~/hadoop-1.0.4/bin$ ./start-all.sh 
starting namenode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-namenode-ubuntu.out
localhost: starting datanode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-datanode-ubuntu.out
localhost: starting secondarynamenode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-secondarynamenode-ubuntu.out
starting jobtracker, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-jobtracker-ubuntu.out
localhost: starting tasktracker, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-tasktracker-ubuntu.out

2、访问localhost:50070失败，说明namenode启动失败
3、查看namenode启动日志

[email protected]:~/hadoop-1.0.4/bin$ cd ../logs
[email protected]:~/hadoop-1.0.4/logs$ view hadoop-ubuntu-namenode-ubuntu.log
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubuntu/127.0.1.1
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by ‘hortonfo‘ on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
2013-01-24 07:05:46,936 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2013-01-24 07:05:46,945 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
2013-01-24 07:05:46,945 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2013-01-24 07:05:46,945 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started
2013-01-24 07:05:47,053 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
2013-01-24 07:05:47,058 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-01-24 07:05:47,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered.
2013-01-24 07:05:47,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source NameNode registered.
2013-01-24 07:05:47,092 INFO org.apache.hadoop.hdfs.util.GSet: VM type       = 32-bit
2013-01-24 07:05:47,092 INFO org.apache.hadoop.hdfs.util.GSet: 2% max memory = 19.33375 MB
2013-01-24 07:05:47,092 INFO org.apache.hadoop.hdfs.util.GSet: capacity      = 2^22 = 4194304 entries
2013-01-24 07:05:47,092 INFO org.apache.hadoop.hdfs.util.GSet: recommended=4194304, actual=4194304
2013-01-24 07:05:47,140 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=ubuntu
2013-01-24 07:05:47,140 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2013-01-24 07:05:47,140 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true
2013-01-24 07:05:47,143 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.block.invalidate.limit=100
2013-01-24 07:05:47,143 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
2013-01-24 07:05:47,154 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean
2013-01-24 07:05:47,169 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times 
2013-01-24 07:05:47,174 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: NameNode is not formatted.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:330)
    at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
2013-01-24 07:05:47,175 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: NameNode is not formatted.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:330)
    at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)

其中"2013-01-24 07:05:47,174 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: NameNode is not formatted."一行显示，namenode未初始化。
4、初始化namenode，却提示是否重新初始化namenode，于是输入Y。

[email protected]:~/hadoop-1.0.4$ bin/hadoop namenode -format
13/01/24 07:05:08 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubuntu/127.0.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by ‘hortonfo‘ on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
Re-format filesystem in /home/ubuntu/hadoop-1.0.4/tmp/dfs/name ? (Y or N) y
Format aborted in /home/ubuntu/hadoop-1.0.4/tmp/dfs/name
13/01/24 07:05:12 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/

5、初始化后，重新启动hadoop，localhost:50070仍然访问失败。查看namenode启动日志，依然报namenode没有初始化的错误。
6、于是删除core-site.xml配置文件中配置的tmp目录下的所有文件；
将hadoop所有服务停止；

再次启动hadoop，访问localhost:50070成功。

[email protected]:~/hadoop-1.0.4/tmp$ rm -rf *
[email protected]:~/hadoop-1.0.4/tmp$ cd ../bin
[email protected]:~/hadoop-1.0.4/bin$ ./stop-all.sh 
stopping jobtracker
localhost: stopping tasktracker
no namenode to stop
localhost: stopping datanode
localhost: stopping secondarynamenode
[email protected]:~/hadoop-1.0.4/bin$ hadoop namenode -format
13/01/24 07:10:45 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubuntu/127.0.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by ‘hortonfo‘ on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
13/01/24 07:10:46 INFO util.GSet: VM type       = 32-bit
13/01/24 07:10:46 INFO util.GSet: 2% max memory = 19.33375 MB
13/01/24 07:10:46 INFO util.GSet: capacity      = 2^22 = 4194304 entries
13/01/24 07:10:46 INFO util.GSet: recommended=4194304, actual=4194304
13/01/24 07:10:46 INFO namenode.FSNamesystem: fsOwner=ubuntu
13/01/24 07:10:46 INFO namenode.FSNamesystem: supergroup=supergroup
13/01/24 07:10:46 INFO namenode.FSNamesystem: isPermissionEnabled=true
13/01/24 07:10:46 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
13/01/24 07:10:46 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
13/01/24 07:10:46 INFO namenode.NameNode: Caching file names occuring more than 10 times 
13/01/24 07:10:46 INFO common.Storage: Image file of size 112 saved in 0 seconds.
13/01/24 07:10:46 INFO common.Storage: Storage directory /home/ubuntu/hadoop-1.0.4/tmp/dfs/name has been successfully formatted.
13/01/24 07:10:46 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/
[email protected]:~/hadoop-1.0.4/bin$ ./start-all.sh 
starting namenode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-namenode-ubuntu.out
localhost: starting datanode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-datanode-ubuntu.out
localhost: starting secondarynamenode, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-secondarynamenode-ubuntu.out
starting jobtracker, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-jobtracker-ubuntu.out
localhost: starting tasktracker, logging to /home/ubuntu/hadoop-1.0.4/libexec/../logs/hadoop-ubuntu-tasktracker-ubuntu.out

7、原来是修改了配置文件中的tmp目录后没有对hdfs做初始化，导致启动hadoop时报namenode没有初始化的错误。

时间： 2024-12-30 03:46:04

Hadoop启动没有namenode进程的相关文章

启动hadoop，没有启动namenode进程。log4j:ERROR setFile(null,true) call faild.

解决办法: cd /home/hadoop/hadoop-env/hadoop-1.0.1/log less hadoop-hadoop-namenode-localhost.localdomain.log  --查看该日志,发现是权限问题 chmod 777 hadoop-hadoop-namenode-localhost.localdomain.log --root用户下操作,赋权限 关闭hadoop后在启动hadoop,发现虽然没有报错,但还是没有启动namenode进程. 则还是进入ha

Hadoop启动不了Namenode进程，出现IllegalArgumentException异常

这次的问题乍看起来很奇怪,在本机完成配置启动Hadoop的时候,首先,我们需要对namenode进行format,但是在执行了命令之后,却出现了如下的异常:FATAL namenode.NameNode: Exception in namenode join java.lang.IllegalArgumentException: URI has an authority component.别的不管,就冲着这个 authority,我义无反顾地在 format命令前加上了 sudo,结果发现--

hadoop启动后jps查不到namenode的解决办法

最近由于项目需要开始接触hadoop,在配置伪分布式启动后,jps查询进程发现少了namenode,而DataNode却存在. 下面是我的core-site.xml和hdfs-site.xml配置: core-site.xml <configuration> <property> <name>hadoop.tmp.dir</name> <value>file:/usr/local/hadoop/tmp</value> <desc

hadoop启动后jps没有namenode(转)

hadoop启动后jps没有namenode 一般都是由于两次或两次以上格式化NameNode造成的,有两种方法可以解决: 1.删除DataNode的所有资料 2.修改每个DataNode的namespaceID(位于/home/hdfs/data/current/VERSION文件中)或修改NameNode的namespaceID(位于/home/hdfs/name/current/VERSION文件中), 目的是两者一致. 但是查看后,两者的ID是一样的, 于是查看/usr/local/ha

启动hadoop报ERROR org.apache.hadoop.hdfs.server.namenode.FSImage: Failed to load image from FSImageFile

不知道怎么回事,今天在启动集群时通过jps查看进程时始终有一个standby namenode进程无法启动.查看日志时报的是不能加载fsimage文件.日志截图如下: 日志报的很明显了是不能加载元数据信息,解决方案: 解决办法: 1.手动copy namenode(active)所在的那台服务器上XXX/dfs/name/current/下的所有文件到namenode(standby) 所在的那台服务器的对应文件夹下. 2. 重新格式化namenode(active),然后再把格式化后的元数据复

hadoop启动守护进程报JAVA_HOME is not set and could not be found

hadoop启动守护进程 sbin/start-dfs.sh 报如下错误:JAVA_HOME is not set and could not be found 解决办法(JAVA_HOME修改为具体的路径):vim etc/hadoop/hadoop-env.sh

启动hadoop集群的时候只能启动一个namenode，另一个报错There appears to be a gap in the edit log. We expected txid 6, but got txid 10.

背景:昨晚11点40几分,终于各个集群组件都启动成功了,然后心满意足的去睡觉了,但是今早再起来再去启动的时候就出现了namenode的问题,然后就开始了查找原因的艰辛历程. 查看报错的log日志: 2019-04-07 13:22:57,746 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimagejava.io.IOException: There appea

Hadoop 2.0 NameNode HA和Federation实践

参考链接:Hadoop 2.0 NameNode HA和Federation实践 Posted on 2012/12/10 一.背景 天云趋势在2012年下半年开始为某大型国有银行的历史交易数据备份及查询提供基于Hadoop的技术解决方案,由于行业的特殊性,客户对服务的可用性有着非常高的要求,而HDFS长久以来都被单点故障的问题所困扰,直到Apache Hadoop在2012年5月发布了2.0的alpha版本,其中MRv2还很不成熟,可HDFS的新功能已经基本可用,尤其是其中的的High Ava

hadoop启动start-all.sh出错（master: ssh: connect to host master port 22: Connection refused）

当我们打开系统输入hadoop启动命令start-all.sh时出现以下错误: [[email protected] ~]# start-all.sh starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-namenode-master.out master: ssh: connect to host master port 22: Network is unreachable master: ss