错误遇到的情形： hive整合hbase，hive的数据表 load，select，insert一切正常 通过hive往hbase关联表插入数据的时候报错，错误内容如下： 2016-04-18 14:00:34,721 Stage-0 map = 0%, reduce = 0% 2016-04-18 14:00:56,491 Stage-0 map = 100%, reduce = 0% Ended Job = job_1460958898158_0001 with errors Error during job, obtaining debugging information... Job Tracking URL: http://master.hadoop:8088/proxy/application_1460958898158_0001/ Examining task ID: task_1460958898158_0001_m_000000 (and more) from job job_1460958898158_0001 Task with the most failures(4): ----- Task ID: task_1460958898158_0001_m_000000 URL: http://master.hadoop:8088/taskdetails.jsp?jobid=job_1460958898158_0001&tipid=task_1460958898158_0001_m_000000 ----- Diagnostic Messages for this Task: Error: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:424) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) ... 9 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38) ... 14 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103) ... 17 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 22 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:373) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:166) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:441) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360) at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98) ... 22 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:315) ... 34 more FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched: Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec 解决办法：hive-env.sh 加入 export HADOOP_HEAPSIZE=1024 export HIVE_AUX_JARS_PATH=/opt/hadoop/hive/lib/hive-hbase-handler-0.10.0-cdh4.2.2.jar,/opt/hadoop/hive/lib/hbase-0.94.2-cdh4.2.2.jar,/opt/hadoop/hive/lib/zookeeper-3.4.5-cdh4.2.2.jar hive-site.xml 加入 hive.aux.jars.path file:///opt/hadoop/hive/lib/hive-hbase-handler-0.10.0-cdh4.2.2.jar,file:///opt/hadoop/hive/lib/hbase-0.94.2-cdh4.2.2-security.jar,file:///opt/hadoop/hive/lib/zookeeper-3.4.5-cdh4.2.2.jar 再次测试，解决！
时间： 2024-10-10 18:02:07