在hive上执行如下sql：
select a.product_id, a.merchant_id, b.category_id from pm_info a join product b on (a.product_id=b.id) where a.is_deleted=0 and a.product_is_gift=0 and a.can_sale=1 and a.can_show=1 and b.is_deleted=0 and b.PRODUCT_IS_GIFT=0;
出现错误，错误如下：
java.io.IOException: Failed to run job : org.apache.hadoop.security.AccessControlException: User pms cannot submit applications to queue root.default at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:300) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292) at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562) at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548) at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:448) at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1352) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1138) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Job Submission failed with exception 'java.io.IOException(Failed to run job : org.apache.hadoop.security.AccessControlException: User pms cannot submit applications to queue root.default)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
原因分析：执行的SQL语句含有where条件，join连接，就会执行MapReduce任务去执行，1号店新集群的MapReduce含有两个队列，一个是root.default队列，一个是pms队列；如果不指定提交到哪个队列，hadoop就会将此Job提交到默认的root.default队列中，但是集群设定了job由MapReduce的pms队列受理，所以提交到root.default队列的任务通通被打回
解决方案（指定Job提交到哪个MapReduce的队列，在shell中执行）：
hive -e "set mapred.job.queue.name=pms;select a.product_id, a.merchant_id, b.category_id from pm_info a join product b on (a.product_id=b.id) where a.is_deleted=0 and a.product_is_gift=0 and a.can_sale=1 and a.can_show=1 and b.is_deleted=0 and b.PRODUCT_IS_GIFT=0;"
Job Submission failed with exception 'java.io.IOException
时间： 2024-11-11 14:48:38