解决Hadoop启动报错:File /opt/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

今天启动hadoop时,发现datanode启动不了,查看日志发现出现以下的错误:

java.io.IOException: File /opt/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)

at org.apache.hadoop.ipc.Client.call(Client.java:740)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at com.sun.proxy.$Proxy2.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at com.sun.proxy.$Proxy2.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2937)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2819)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2102)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2288)

发现原因:原来是前天格式化了hdfs,每次格式化时(namenode format)会重新创建一个namenodeId,而dfs.data.dir参数配置的目录中包含的是上次format创建的id,和dfs.name.dir参数配置的目录中的id不一致。namenode format清空了namenode下的数据,但是没有清空datanode下的数据,导致启动时失败。

解决方法:我是重新创建dfs.data.dir指定的文件夹,然后修改成该文件夹,如原本hdfs-site.xml中dfs.data.dir的参数是/opt/hadoop/data,我在/opt/hadoop下创建了data1的文件夹,然后将dfs.data.dir的参数改为:/opt/hadoop/data1,最后重新格式化hdfs (hadoop namenode -format),问题解决.

时间: 2024-08-26 03:10:35

解决Hadoop启动报错:File /opt/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1的相关文章

用java运行Hadoop程序报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.

用java运行Hadoop例程报错:org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.所写代码如下: package com.pcitc.hadoop; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.h

解决JVM启动报错:Unrecognized VM option '+HeapDumpOnOutOfMemeryError'

今天再搞一些OutOfMemery的相关知识探索,我想在JVM遇到OOM错误的时候,能够打印出heap dump,以便事后用Eclipse Memory Analyzer Tool(MAT)等内存分析工具分析内存的占用情况.我使用了JDK1.6.0_37和JDK1.7.0_60版本进行试验,到网上找了下,知道-XX:+HeapDumpOnOutOfMemoryError可以让JVM在探测到内存OOM的时候打印dump.但是在JVM启动参数添加这个参数的时候,JVM启动失败: Unrecogniz

修改grub解决计算机启动报错:ERROR 17

修改grub解决计算机启动报错:ERROR 17 原本计算机有C,D,E,F,G.C盘windows系统盘,G盘安装了Centos.后来由于需要,就压缩卷的方式从D盘压缩出一个H盘.然后再次启动计算机的时候就遇到了错误:ERROR 17 原因: 就在于先安装windows后安装Linux这样的话,grub就由位于G盘的Centos引导,现在该分区之前又划分出一个新的逻辑分区,就导致磁盘系统符号向后移了一位,因此grub失败. 解决方案: 1,刻录一个Linux安装U盘,进入BIOS修改启动项,从

linux查看与修改交换内存配置(解决zabbix-agent启动报错)

问题 zabbix-agent在一台centos6.5上启动报错: cannot allocate shared memory of size 949056: [28] No space left on device cannot allocate shared memory for collector 检查 # sysctl -a | grep shm kernel.shmmax = 33554432 kernel.shmall = 2097152 kernel.shmmni = 4096 k

hadoop启动报错:localhost: ssh: Could not resolve hostname localhost

hadoop启动journalnode时报错:localhost: ssh: Could not resolve hostname localhost: Temporary failure in name resolution 解决办法:将高亮部分补充到/etc/profile中,然后source生效 export JAVA_HOME=/opt/module/jdk1.8.0_144 export HADOOP_HOME=/opt/ha/hadoop-2.7.4 export HADOOP_CO

解决Eclipse启动报错Failed to create the Java Virtual Machine

电脑:2G内存,WIN7 32位. 启动adt-bundle-windows-x86-20140702\eclipse\eclipse.exe时,报错[Failed to create the Java Virtual Machine].点确定后,Eclipse自动关闭 解决方法如下: 修改Eclipse目录下的配置文件[eclipse.iniadt-bundle-windows-x86-20140702\eclipse\eclipse.ini] 将下面两个值改为原来的0.5倍,然后保存,重新启

解决Eclipse启动报错【Failed to create the Java Virtual Machine】

电脑:2G内存,WIN7 32位. 启动adt-bundle-windows-x86-20140702\eclipse\eclipse.exe时,报错[Failed to create the Java Virtual Machine].点确定后,Eclipse自动关闭 解决方法如下: 修改Eclipse目录下的配置文件[eclipse.iniadt-bundle-windows-x86-20140702\eclipse\eclipse.ini] 将下面两个值改为原来的0.5倍,然后保存,重新启

MySQL 启动报错:File ./mysql-bin.index not found (Errcode: 13)

一 说明: 前几天,试着复制了一个mysql目录到另一台机器上面,然后更改权限并启动,但是一直报错 Starting MySQL. ERROR! The server quit without updating PID file (/data/mysql/AY14020816093477605eZ.pid). 二 排查: 上面只能看到mysql启动失败,具体的原因,需要查看数据库目录下的.err文件,查看.err文件,内容如下: 140726 00:18:10 mysqld_safe mysql

解决zabbix启动报错

当启动zabbix_server的时,报错:zabbix is not running ,the information displayed may not be current 解决办法:查看zabbix服务器日志/var/log/zabbix_server.log tail -f /var/log/zabbix_server.log 如果发现 "connection to database ... to many connections",这是修改mysql的配置文件 /etc/m