hadoop中执行命令时发生错误

通过一下方式找到错误的原因，开启hadoop的调试信息

[[email protected] bin]# export HADOOP_ROOT_LOGGER=DEBUG,console

这样在执行命令时，可以通过error字样定位执行命令时产生错误的原因

[[email protected] bin]# ./hadoop fs -mkdir test
14/10/08 11:17:55 DEBUG util.Shell: setsid exited with exit code 0
14/10/08 11:17:56 DEBUG conf.Configuration: parsing URL jar:file:/usr/local/hadoop-2.5.1/share/hadoop/common/hadoop-common-2.5.1.jar!/core-default.xml
14/10/08 11:17:56 DEBUG conf.Configuration: parsing input stream sun.net.www.pr[email protected]
14/10/08 11:17:56 DEBUG conf.Configuration: parsing URL file:/usr/local/hadoop-2.5.1/etc/hadoop/core-site.xml
14/10/08 11:17:56 DEBUG conf.Configuration: parsing input stream [email protected]
14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
14/10/08 11:17:56 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
14/10/08 11:17:56 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/10/08 11:17:56 DEBUG security.Groups: Creating new Groups object
14/10/08 11:17:56 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/10/08 11:17:56 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /usr/local/hadoop-2.5.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14‘ not found (required by /usr/local/hadoop-2.5.1/lib/native/libhadoop.so.1.0.0)
14/10/08 11:17:56 DEBUG util.NativeCodeLoader: java.library.path=/usr/local/hadoop-2.5.1/lib/native
14/10/08 11:17:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/10/08 11:17:56 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/10/08 11:17:56 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/10/08 11:17:56 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/10/08 11:17:56 DEBUG security.UserGroupInformation: hadoop login
14/10/08 11:17:56 DEBUG security.UserGroupInformation: hadoop login commit
14/10/08 11:17:56 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/10/08 11:17:56 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false
14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false
14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false
14/10/08 11:17:56 DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path =
14/10/08 11:17:57 DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
14/10/08 11:17:57 DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=or[email protected]55a1c291
14/10/08 11:17:57 DEBUG ipc.Client: getting client out of cache: [email protected]
14/10/08 11:17:57 DEBUG shortcircuit.DomainSocketFactory: Both short-circuit local reads and UNIX domain socket are disabled.
14/10/08 11:17:57 DEBUG ipc.Client: The ping interval is 60000 ms.
14/10/08 11:17:57 DEBUG ipc.Client: Connecting to yts/192.168.3.82:9000
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: starting, having connections 1
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root sending #0
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root got value #0
14/10/08 11:17:57 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 143ms
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root sending #1
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root got value #1
14/10/08 11:17:57 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms
mkdir: `test‘: No such file or directory
14/10/08 11:17:57 DEBUG ipc.Client: stopping client from cache: [email protected]
14/10/08 11:17:57 DEBUG ipc.Client: removing client from cache: [email protected]
14/10/08 11:17:57 DEBUG ipc.Client: stopping actual client because no more references remain: [email protected]
14/10/08 11:17:57 DEBUG ipc.Client: Stopping client
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: closed
14/10/08 11:17:57 DEBUG ipc.Client: IPC Client (124407148) connection to yts/192.168.3.82:9000 from root: stopped, remaining connections 0

时间： 2024-10-05 04:19:23

hadoop中执行命令时发生错误的相关文章

使用wget命令时发生错误

用的是centos6.5, 当我使用命今 sudo wget https://cmake.org/files/v3.6/cmake-3.6.1.tar.gz 下载个cmake的包时, 发生了这样的错误 --2016-09-01 06:53:39-- https://cmake.org/files/v3.6/cmake-3.6.1.tar.gz Resolving cmake.org... 66.194.253.19 Connecting to cmake.org|66.194.253.19|:4

Eclipse中Maven Install时发生错误

问题描述 要把一个本地包保存进本地maven库中, 所以对该project执行了run as => Maven Install, 结果报下面的错误. 解决办法 1. 通过命令窗口手动创建这两个文件夹. 2. 再执行一次Maven Install就可以了. 原文地址:https://www.cnblogs.com/cheese320/p/8778020.html

C# .Net FrameWork3.5中异步HTTP请求时，由于安全协议的问题System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)方法抛出“基础连接已经关闭: 发送时发生错误”的解决办法

现象描述: C# .Net FrameWork3.5中异步HTTP请求时,由于安全协议的问题System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)方法抛出“基础连接已经关闭: 发送时发生错误”. 原因分析: 大多数情况下是由于客户端所设置的HTTP访问请求的安全协议不符合服务器端的安全协议要求.比如,NASA提供瓦片服务的http://worldwind25.arc.nasa.gov/wms?service=WMS&v

转载：Linux下执行SVN命令时提示错误：Valid UTF-8 data

在Linux下执行svn add *时出现如下错误: svn:  Valid UTF-8  data(hex: 4b)followed by invalid UTF-8 sequence(hex:  fc 63 68  65) 出现这个错误是因为svn库里有文件的名字不是utf-8编码的,这种情况对于中文来说很常见.比如在自己的windows上建了一个中文名字的文件,就会使这种情况. 几经周折,才找到解决办法: 首先,执行命令: ls * | file -/dev/stdin:  ISO-885

cmd中mysql主键id自增，在添加信息时发生错误，再次成功添加时，id已经跳过错误的信息继续自增。

id 自增,在往这个表里添加信息时 发生错误,再次添加 id数值已经跳过之前 原文地址:https://www.cnblogs.com/Zhao159461/p/10764117.html

执行 $ Gulp 时发生了什么 —— 基于 Gulp 的前端集成解决方案（二）

前言 文章 在windows下安装gulp —— 基于 Gulp 的前端集成解决方案(一) 中,已经完成对 gulp 的安装,由于是window环境,文中特意提到了可以通过安装 gitbash 来代替 window 系统的 CMD 命令行工具.本节主要围绕 如何使用 Gulp 完成一个预编译 SASS 文件的任务 来逐步熟悉 Gulp . 本文地址:http://www.cnblogs.com/leonkao/p/4611102.html Gulp 概述 Gulp 是一个构建工具,通过自动化处理

Gnome Terminal,Xshell等终端模拟器中执行命令出现乱码问题解决

一.前言 Xshell跟Gnome Terminal相比,两者都是终端模拟器(在Xshell中也可以执行简单的内置命令,如"cd","ls"等),地位相同. 二.原理分析 涉及到乱码,那么需要了解编码解码过程.在终端模拟器中执行命令,通信过程示意图如图1所示. 图1 在以上通信过程中,在"命令执行单元"处发生了一系列的编码解码过程,在"终端模拟器"处也发生了一系列的编码解码过程,此外,我们常常创建SSH连接,从而建立一个远端S

WCF问题集锦：基础连接已经关闭 接收时发生错误

今天,进行单元测试时,当执行到第二个测试类时--之前从来没有错误的测试居然报了:基础连接已经关闭 接收时发生错误.想想今天唯一改动的是在测试代码中加了下列两个执行体: [TestFixtureSetUp] public void Set() { CommonFuns.SetTime(); } [TestFixtureTearDown] public void Restore() { CommonFuns.RestoreTime(); } 其中,Set用来设置系统当前时间为制定值,Restore用

excel2003出现“向程序发送命令时出现错误”解决方法

电脑已经预装了office2010,因为想要学习 <Excel 图标之道>,安装了Excel2003,刚刚安装好,打开出现“向程序发送命令时出现错误”,虽然点击确定就可以用了,可总是觉得不爽,网上找了相关方法, 1. 在资源管理器中进入“C:\Documents and Settings\Administrator\Application Data\Microsoft\Templates”目录,删除其中的“Normal.dot”公用模板文件(可能还有个隐藏的编辑中的文件,一并删除),再重新启动