基于HBase Hadoop 分布式集群环境下的MapReduce程序开发

　　HBase分布式集群环境搭建成功后，连续4、5天实验客户端Map/Reduce程序开发，这方面的代码网上多得是，写个测试代码非常容易，可是真正运行起来可说是历经挫折。下面就是我最终调通并让程序在集群上运行起来的一些经验教训。

　　一、首先说一下我的环境：

　　1，集群的环境配置请见这篇博文。

　　2，开发客户机环境：操作系统是CentOS6.5，JDK版本是1.7.0-60，开发工具是Eclipse（原始安装是从google的ADT网站下载的ADT专用开发环境，后来加装了Java企业开发的工具，启动Flash还是ADT的：））。

　　3，etc/hosts文件添加了集群的所有主机。

　　二、必要准备工作：

　　1，java类库准备：从集群中scp复制了Hadoop/share下所有文件，hbase/lib目录下的所有文件到开发机的/home/username/lib目录下备用(其中username是开发用户名称)，打开eclipse，选择菜单window->Preferences,

在左侧树形菜单中导航至Java--Build Path--User Libraries，我添加了3个User Library，“hadoop-common”，“hadoop-yarn”，“hbase”

然后分别添加如下（这是User Libraries的导出文件，可以在上面的界面中导入）的jar包。

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<eclipse-userlibraries version="2">
    <library name="hadoop-common" systemlibrary="false">
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/hadoop-auth-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/hadoop-annotations-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/hadoop-common-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-logging-1.1.3.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-compress-1.4.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/guava-11.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-collections-3.2.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/slf4j-api-1.7.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/slf4j-log4j12-1.7.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/zookeeper-3.4.5-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-configuration-1.6.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-math3-3.1.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/activation-1.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/asm-3.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/avro-1.7.5-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-beanutils-1.7.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-beanutils-core-1.8.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-cli-1.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-codec-1.4.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-digester-1.8.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-el-1.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-httpclient-3.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-io-2.4.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-lang-2.6.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/commons-net-3.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/httpclient-4.2.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/httpcore-4.2.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jackson-core-asl-1.8.8.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jackson-jaxrs-1.8.8.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jackson-xc-1.8.8.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jasper-compiler-5.5.23.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jasper-runtime-5.5.23.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/java-xmlbuilder-0.4.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jaxb-api-2.2.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jaxb-impl-2.2.3-1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jersey-core-1.9.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jersey-json-1.9.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jersey-server-1.9.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jets3t-0.9.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jettison-1.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jetty-6.1.26.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jetty-util-6.1.26.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jsch-0.1.42.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jsp-api-2.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/jsr305-1.3.9.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/junit-4.8.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/log4j-1.2.17.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/mockito-all-1.8.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/paranamer-2.3.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/protobuf-java-2.5.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/servlet-api-2.5.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/snappy-java-1.0.4.1.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/stax-api-1.0-2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/xmlenc-0.52.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/common/lib/xz-1.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce/lib/netty-3.6.2.Final.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/hdfs/hadoop-hdfs-2.3.0-cdh5.0.2.jar"/>
    </library>
    <library name="hadoop-yarn" systemlibrary="false">
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-core-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-common-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-shuffle-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-api-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-client-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-common-2.3.0-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.3.0-cdh5.0.2.jar"/>
    </library>
    <library name="hbase" systemlibrary="false">
        <archive path="/home/gzg/javadev/lib/hbase/hbase-client-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-common-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-hadoop2-compat-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-hadoop-compat-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-it-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-prefix-tree-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-protocol-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-testing-util-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-thrift-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/protobuf-java-2.5.0.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-server-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/hbase-shell-0.96.1.1-cdh5.0.2.jar"/>
        <archive path="/home/gzg/javadev/lib/hbase/htrace-core-2.01.jar"/>
    </library>
</eclipse-userlibraries>

注：这个很有可能并不是运行Hbase Map/Reduce任务需要的最精简的类库，恕我没有逐一测试。

　　2，新建一个Java/Application，我的名字就叫hbase1，然后在项目project视图中点击右键，选择“Build Path”--“Add Libraries...”--选择“User Library”，然后选中上面添加的三个User Library。

　　3，复制集群的三个配置文件到项目的src目录下，这三个配置文件是：

core-site.xml  hdfs-site.xml  yarn-site.xml

　　三、编写代码，这里基本按照我所遇挫折过程逐步完善代码（缺少类库之类的就略掉了）：

　　 1，遭遇空指针错误--reverseDNS

主文件内容：

/**
 * @author gzg
 *
 */
public class Test1  {

	/**
	 * @param args
	 * @throws IOException
	 * @throws ZooKeeperConnectionException
	 * @throws MasterNotRunningException
	 */
	Configuration conf=null;
	public static void main(String[] args) {
		try {
			new TestHBaseMapred().run(false);
		} catch(yh.utils.system.CommandError cep){
			cep.printStackTrace();
		} catch (ClassNotFoundException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}
}

　TestHBaseMapRed.java内容：

public class TestHBaseMapred {
	public void run(boolean toFile) throws ClassNotFoundException, IOException, InterruptedException{
		org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();　　　　　　　　　//设置zookeeper的地址s
		conf.set("hbase.zookeeper.quorum", "zk1,zk2,zk3");　　　　　　　　　//此设置好像没用
		conf.set("hadoop.job.user", "hadoop");
		org.apache.hadoop.mapreduce.Job job=null;
		String  tmpTable = "_mrtest1";
		yh.utils.ElapsedTimeMeasure etm=yh.utils.ElapsedTimeMeasure.createAndStart();
		try {

			job = Job.getInstance(conf);
			job.setJobName("count meter");

			if(toFile==false){
				HBaseUtil.dropTable(conf, tmpTable);
				HBaseUtil.createTable(conf, tmpTable, "c");
				job.setOutputFormatClass(org.apache.hadoop.hbase.mapreduce.TableOutputFormat.class);
				job.getConfiguration().set(org.apache.hadoop.hbase.mapreduce.TableOutputFormat.OUTPUT_TABLE, tmpTable);
				//
				job.setReducerClass(TblReduce.class);
			}else{
				job.setOutputFormatClass(org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.class);
				Path op = new Path("result");
				org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.setOutputPath(job, op);
				//
				job.setReducerClass(TxtReduce.class);
			}			

			Scan scan = new Scan();
			scan.addFamily(Bytes.toBytes("c"));
			org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob("dzdl_t1",scan,TblMap.class ,Text.class,LongWritable.class,job);
			job.waitForCompletion(true);
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		System.out.println("time elapsed :"+etm.stop().toString());

	}
	public static class TblMap extends org.apache.hadoop.hbase.mapreduce.TableMapper<Text, LongWritable>{

		@Override
		protected void map(ImmutableBytesWritable key,Result result,org.apache.hadoop.mapreduce.Mapper.Context  ctx) throws IOException, InterruptedException{
			Cell c=result.getColumnLatestCell(Bytes.toBytes("c"), Bytes.toBytes("col3"));
			String col= Bytes.toString(CellUtil.cloneQualifier(c));
			Text kv = new Text();
			kv.set("count_result");
			LongWritable lv = new LongWritable();
			lv.set(1);
			ctx.write(kv, lv);
			//org.apache.log4j.Logger.getRootLogger().info(kv+" : 1");
		}
	}

	public static class  TblReduce extends org.apache.hadoop.hbase.mapreduce.TableReducer<Text,LongWritable,ImmutableBytesWritable>{
		@Override
		protected void reduce(Text key,Iterable<LongWritable> values,Context ctx) throws IOException, InterruptedException{
			long t =0;
			for(LongWritable v:values){
				t+=v.get();
			}
			byte[] ok=Bytes.toBytes("count_result");
			Put p=new Put(ok);

			p.add(Bytes.toBytes("c"),key.copyBytes(),Bytes.toBytes(t));
		p.add(Bytes.toBytes("c"),Bytes.toBytes("14112896"),Bytes.toBytes(14112896));
			ctx.write(new ImmutableBytesWritable(ok),p);
		}
	}

	public static class TxtReduce extends org.apache.hadoop.mapreduce.lib.reduce.LongSumReducer<Text>{
		@Override
		public void reduce(Text key,Iterable<LongWritable> values,Context ctx) throws IOException, InterruptedException{
			long t =0;
			for(LongWritable v:values){
				t+=v.get();
			}
			//Put p=new Put(key.copyBytes());

			//p.add(Bytes.toBytes("c"),key.copyBytes(),Bytes.toBytes(t));
			//ctx.write(new ImmutableBytesWritable(key.copyBytes()),p);
			LongWritable l = new LongWritable();
			l.set(t);
			ctx.write(key,l);

		}
	}
}

此时运行，会报如下错误：

Exception in thread "main" java.lang.NullPointerException
	at org.apache.hadoop.net.DNS.reverseDns(DNS.java:92)
	at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:218)
	at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:184)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
	at hbase1.TestHBaseMapred.run(TestHBaseMapred.java:82)
	at hbase1.Test1.main(Test1.java:69)

看错误，不知道为什么需要调用DNS服务，hosts文件好像没有作用， 搜索很久，终于在一个老外的问题里找到答案，conf添加如下设置解决。

conf.set("hbase.nameserver.address","nn1,nn2");

添加后，运行仍会报下面的错无，但是好像并不影响程序继续运行：

2014-08-21 14:05:04,158 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn1/192.168.5.111 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2: Name or service not known]; remaining name ‘111.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,170 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,171 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn3/192.168.5.113 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘113.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,172 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,173 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,173 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn1/192.168.5.111 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘111.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,174 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,174 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,175 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,175 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,199 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,200 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn2/192.168.5.112 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘112.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,201 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,201 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn2/192.168.5.112 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘112.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,202 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn3/192.168.5.113 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘113.5.168.192.in-addr.arpa‘
2014-08-21 14:05:04,539 INFO [org.apache.hadoop.mapreduce.JobSubmitter] - number of splits:15

　　 2，本地运行的问题。解决了上面的问题，程序终于可以运行了，可是我在rm的web界面中却看不到我提交的任务，再次求助bing，终于找到症结所在，需要添加设置如下：

                //此项必须设置，否则程序只是在本地客户端上运行，通过rm的浏览器管理界面看不到！
		conf.set("mapreduce.framework.name", "yarn");

　　  3，添加了上面语句后，新的问题又来了，运行程序出现新的错误：

2014-08-21 14:15:08,887 INFO [org.apache.hadoop.mapreduce.Job] - The url to track the job: http://rm2:23188/proxy/application_1408378794442_0014/
2014-08-21 14:15:08,888 INFO [org.apache.hadoop.mapreduce.Job] - Running job: job_1408378794442_0014
2014-08-21 14:15:21,466 INFO [org.apache.hadoop.mapreduce.Job] - Job job_1408378794442_0014 running in uber mode : false
2014-08-21 14:15:21,469 INFO [org.apache.hadoop.mapreduce.Job] -  map 0% reduce 0%
2014-08-21 14:15:34,988 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000013_0, Status : FAILED
2014-08-21 14:15:35,023 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000011_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:39,485 INFO [org.apache.hadoop.mapreduce.Job] -  map 7% reduce 0%
2014-08-21 14:15:39,490 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000003_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:15:39,497 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000006_0, Status : FAILED

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:39,505 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000001_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:40,663 INFO [org.apache.hadoop.mapreduce.Job] -  map 0% reduce 0%
2014-08-21 14:15:40,736 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000004_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:42,783 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000013_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:44,921 INFO [org.apache.hadoop.mapreduce.Job] -  map 7% reduce 0%
2014-08-21 14:15:44,925 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000005_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

2014-08-21 14:15:46,019 INFO [org.apache.hadoop.mapreduce.Job] -  map 0% reduce 0%
2014-08-21 14:15:46,066 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:15:46,070 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000002_0, Status : FAILED

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:46,073 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000009_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:46,076 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000014_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:47,086 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000007_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:15:59,146 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000006_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:00,237 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000003_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:01,312 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000013_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:16:01,329 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000004_1, Status : FAILED

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:03,396 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000008_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:16:03,401 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000000_1, Status : FAILED

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:04,488 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000009_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:16:04,493 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000001_1, Status : FAILED

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:05,530 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000010_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:06,546 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000012_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:09,672 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000006_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:10,935 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000014_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:19,239 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000012_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more
2014-08-21 14:16:19,243 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000003_2, Status : FAILED

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:19,246 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:20,307 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000004_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:21,326 INFO [org.apache.hadoop.mapreduce.Job] - Task Id : attempt_1408378794442_0014_m_000011_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:196)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class hbase1.TestHBaseMapred$TblMap not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
	... 8 more

2014-08-21 14:16:23,352 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 100%
2014-08-21 14:16:24,378 INFO [org.apache.hadoop.mapreduce.Job] - Job job_1408378794442_0014 failed with state FAILED due to: Task failed task_1408378794442_0014_m_000013
Job failed as tasks failed. failedMaps:1 failedReduces:0

2014-08-21 14:16:24,584 INFO [org.apache.hadoop.mapreduce.Job] - Counters: 13
	Job Counters
		Failed map tasks=31
		Killed map tasks=13
		Launched map tasks=44
		Other local map tasks=29
		Data-local map tasks=15
		Total time spent by all maps in occupied slots (ms)=1665094
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=832547
		Total vcore-seconds taken by all map tasks=832547
		Total megabyte-seconds taken by all map tasks=1278792192
	Map-Reduce Framework
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
*** measure elapsed time,stoped at 2014-08-21 14:16:24:588
time elapsed :0days 0hours 1minutes 29seconds 879milliseconds[89.879seconds]

试着设置 job.setJarByClass(Test1.class); 无效。此问题困扰了我两天，难道只能上传到集群集群上运行吗？这样也太不方便了，N久后，终于我有幸找到了小鱼儿的博客 的这篇文章MapReduce提交作业常见问题，终于是找到了解决方法。我采用的是文中所提第三种方法，在程序开始运行前自己首先打包自己，就是调用的是写好的shell文件，这里也贴出来，以下两个文件在和src同级的dist目录中

[[email protected] dist]$ cat manifest.mf
Manifest-Version: 1.0
Class-Path: . $HADOOP_HOME/lib hbase1_lib/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2.ja
 r hbase1_lib/hadoop-auth-2.3.0-cdh5.0.2.jar hbase1_lib/hadoop-annotat
 ions-2.3.0-cdh5.0.2.jar hbase1_lib/hadoop-common-2.3.0-cdh5.0.2.jar h
 base1_lib/commons-logging-1.1.3.jar hbase1_lib/commons-compress-1.4.1
 .jar hbase1_lib/guava-11.0.2.jar hbase1_lib/commons-collections-3.2.1
 .jar hbase1_lib/slf4j-api-1.7.5.jar hbase1_lib/slf4j-log4j12-1.7.5.ja
 r hbase1_lib/zookeeper-3.4.5-cdh5.0.2.jar hbase1_lib/commons-configur
 ation-1.6.jar hbase1_lib/commons-math3-3.1.1.jar hbase1_lib/activatio
 n-1.1.jar hbase1_lib/asm-3.2.jar hbase1_lib/avro-1.7.5-cdh5.0.2.jar h
 base1_lib/commons-beanutils-1.7.0.jar hbase1_lib/commons-beanutils-co
 re-1.8.0.jar hbase1_lib/commons-cli-1.2.jar hbase1_lib/commons-codec-
 1.4.jar hbase1_lib/commons-digester-1.8.jar hbase1_lib/commons-el-1.0
 .jar hbase1_lib/commons-httpclient-3.1.jar hbase1_lib/commons-io-2.4.
 jar hbase1_lib/commons-lang-2.6.jar hbase1_lib/commons-net-3.1.jar hb
 ase1_lib/httpclient-4.2.5.jar hbase1_lib/httpcore-4.2.5.jar hbase1_li
 b/jackson-core-asl-1.8.8.jar hbase1_lib/jackson-jaxrs-1.8.8.jar hbase
 1_lib/jackson-mapper-asl-1.8.8.jar hbase1_lib/jackson-xc-1.8.8.jar hb
 ase1_lib/jasper-compiler-5.5.23.jar hbase1_lib/jasper-runtime-5.5.23.
 jar hbase1_lib/java-xmlbuilder-0.4.jar hbase1_lib/jaxb-api-2.2.2.jar
 hbase1_lib/jaxb-impl-2.2.3-1.jar hbase1_lib/jersey-core-1.9.jar hbase
 1_lib/jersey-json-1.9.jar hbase1_lib/jersey-server-1.9.jar hbase1_lib
 /jets3t-0.9.0.jar hbase1_lib/jettison-1.1.jar hbase1_lib/jetty-6.1.26
 .jar hbase1_lib/jetty-util-6.1.26.jar hbase1_lib/jsch-0.1.42.jar hbas
 e1_lib/jsp-api-2.1.jar hbase1_lib/jsr305-1.3.9.jar hbase1_lib/junit-4
 .8.2.jar hbase1_lib/log4j-1.2.17.jar hbase1_lib/mockito-all-1.8.5.jar
  hbase1_lib/paranamer-2.3.jar hbase1_lib/protobuf-java-2.5.0.jar hbas
 e1_lib/servlet-api-2.5.jar hbase1_lib/snappy-java-1.0.4.1.jar hbase1_
 lib/stax-api-1.0-2.jar hbase1_lib/xmlenc-0.52.jar hbase1_lib/xz-1.0.j
 ar hbase1_lib/netty-3.6.2.Final.jar hbase1_lib/hadoop-hdfs-2.3.0-cdh5
 .0.2.jar hbase1_lib/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2_2.jar
 hbase1_lib/hadoop-mapreduce-client-core-2.3.0-cdh5.0.2.jar hbase1_lib
 /hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.2.jar hbase1_lib/hado
 op-mapreduce-client-common-2.3.0-cdh5.0.2.jar hbase1_lib/hadoop-mapre
 duce-client-shuffle-2.3.0-cdh5.0.2.jar hbase1_lib/hadoop-yarn-api-2.3
 .0-cdh5.0.2.jar hbase1_lib/hadoop-yarn-client-2.3.0-cdh5.0.2.jar hbas
 e1_lib/hadoop-yarn-common-2.3.0-cdh5.0.2.jar hbase1_lib/hadoop-yarn-a
 pplications-distributedshell-2.3.0-cdh5.0.2.jar hbase1_lib/hbase-clie
 nt-0.96.1.1-cdh5.0.2.jar hbase1_lib/hbase-common-0.96.1.1-cdh5.0.2.ja
 r hbase1_lib/hbase-hadoop2-compat-0.96.1.1-cdh5.0.2.jar hbase1_lib/hb
 ase-hadoop-compat-0.96.1.1-cdh5.0.2.jar hbase1_lib/hbase-it-0.96.1.1-
 cdh5.0.2.jar hbase1_lib/hbase-prefix-tree-0.96.1.1-cdh5.0.2.jar hbase
 1_lib/hbase-protocol-0.96.1.1-cdh5.0.2.jar hbase1_lib/hbase-shell-0.9
 6.1.1-cdh5.0.2.jar hbase1_lib/hbase-testing-util-0.96.1.1-cdh5.0.2.ja
 r hbase1_lib/hbase-thrift-0.96.1.1-cdh5.0.2.jar hbase1_lib/protobuf-j
 ava-2.5.0_2.jar hbase1_lib/htrace-core-2.01.jar hbase1_lib/hbase-serv
 er-0.96.1.1-cdh5.0.2.jar
Main-Class: hbase1.Test1

[[email protected] dist]$ cat makejar
cd "$(dirname $0)"
rm -f hbase1.jar
jar cvfm hbase1.jar manifest.mf  -C "../bin/" .

同时，修改了TestHBaseMapred.run方法签名，传递jar包路径，新的run方法如下：

public void run(String appJarFile,boolean toFile) throws ClassNotFoundException, IOException, InterruptedException{
		//org.apache.hadoop.conf.Configuration conf = yh.utils.hbase.HBaseUtil.getConfiguration("zk1,zk2,zk3");
		org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
		conf.set("hbase.zookeeper.quorum", "zk1,zk2,zk3");
		conf.set("hadoop.job.user", "hadoop");
		conf.set("mapreduce.framework.name", "yarn");

		conf.set("hbase.nameserver.address","nn1,nn2");

		org.apache.hadoop.mapreduce.Job job=null;
		String  tmpTable = "_mrtest1";
		yh.utils.ElapsedTimeMeasure etm=yh.utils.ElapsedTimeMeasure.createAndStart();
		try {

			job = Job.getInstance(conf);
			job.setJobName("count meter");
			//job.setJarByClass(Test1.class);
			job.setJar(appJarFile);

			if(toFile==false){
				HBaseUtil.dropTable(conf, tmpTable);
				HBaseUtil.createTable(conf, tmpTable, "c");
				job.setOutputFormatClass(org.apache.hadoop.hbase.mapreduce.TableOutputFormat.class);
				job.getConfiguration().set(org.apache.hadoop.hbase.mapreduce.TableOutputFormat.OUTPUT_TABLE, tmpTable);
				//
				job.setReducerClass(TblReduce.class);
			}else{
				job.setOutputFormatClass(org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.class);
				Path op = new Path("result");
				org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.setOutputPath(job, op);
				//
				job.setReducerClass(TxtReduce.class);
			}			

			Scan scan = new Scan();
			scan.addFamily(Bytes.toBytes("c"));
			org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob("dzdl_t1",scan,TblMap.class ,Text.class,LongWritable.class,job);
			job.waitForCompletion(true);
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		System.out.println("time elapsed :"+etm.stop().toString());

	}

　　 4，在选择写入文件的Reduce方法是，同样遇到权限问题，我采用的解决方法是给所有用户授予了读写权限，这里注意/tmp也要授权，运行时用到

[[email protected] hbase]$ hdfs dfs -chmod 777 /user
[[email protected] hbase]$ hdfs dfs -chmod 777 /tmp

　　 另外，最后还有一点注意，设置log4j.properties文件中的级别为INFO，不要设置为WARN，否则看不到很多信息，不利于发现错误原因。

# Configure logging for testing: optionally with log file
#log4j.rootLogger=WARN, stdout
log4j.rootLogger=INFO, stdout
# log4j.rootLogger=WARN, stdout, logfile

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - %m%n

log4j.appender.logfile=org.apache.log4j.FileAppender
log4j.appender.logfile.File=target/spring.log
log4j.appender.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.logfile.layout.ConversionPattern=%d %p [%c] - %m%n

　　四、At last，终于大功告成，看看map/reduce的运行过程

*** measure elapsed time,started at 2014-08-21 14:48:35:880
2014-08-21 14:48:36,601 WARN [org.apache.hadoop.util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-08-21 14:48:36,682 INFO [org.apache.hadoop.conf.Configuration.deprecation] - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2014-08-21 14:48:42,227 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:zookeeper.version=3.4.5-cdh5.0.2--1, built on 06/09/2014 16:09 GMT
2014-08-21 14:48:42,227 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:host.name=211.137.170.246
2014-08-21 14:48:42,227 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.version=1.7.0_60
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.vendor=Oracle Corporation
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.home=/usr/java/jdk1.7.0_60/jre
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.class.path=/home/gzg/javadev/lib/hadoop/mapreduce/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/hadoop-auth-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/hadoop-annotations-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/hadoop-common-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-logging-1.1.3.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-compress-1.4.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/guava-11.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-collections-3.2.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/zookeeper-3.4.5-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-configuration-1.6.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-math3-3.1.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/activation-1.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/asm-3.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/avro-1.7.5-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-cli-1.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-codec-1.4.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-digester-1.8.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-el-1.0.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-httpclient-3.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-io-2.4.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-lang-2.6.jar:/home/gzg/javadev/lib/hadoop/common/lib/commons-net-3.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/httpclient-4.2.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/httpcore-4.2.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/gzg/javadev/lib/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/gzg/javadev/lib/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/gzg/javadev/lib/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/gzg/javadev/lib/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/gzg/javadev/lib/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/gzg/javadev/lib/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/gzg/javadev/lib/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/gzg/javadev/lib/hadoop/common/lib/jersey-core-1.9.jar:/home/gzg/javadev/lib/hadoop/common/lib/jersey-json-1.9.jar:/home/gzg/javadev/lib/hadoop/common/lib/jersey-server-1.9.jar:/home/gzg/javadev/lib/hadoop/common/lib/jets3t-0.9.0.jar:/home/gzg/javadev/lib/hadoop/common/lib/jettison-1.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/jetty-6.1.26.jar:/home/gzg/javadev/lib/hadoop/common/lib/jetty-util-6.1.26.jar:/home/gzg/javadev/lib/hadoop/common/lib/jsch-0.1.42.jar:/home/gzg/javadev/lib/hadoop/common/lib/jsp-api-2.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/jsr305-1.3.9.jar:/home/gzg/javadev/lib/hadoop/common/lib/junit-4.8.2.jar:/home/gzg/javadev/lib/hadoop/common/lib/log4j-1.2.17.jar:/home/gzg/javadev/lib/hadoop/common/lib/mockito-all-1.8.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/paranamer-2.3.jar:/home/gzg/javadev/lib/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/gzg/javadev/lib/hadoop/common/lib/servlet-api-2.5.jar:/home/gzg/javadev/lib/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/gzg/javadev/lib/hadoop/common/lib/stax-api-1.0-2.jar:/home/gzg/javadev/lib/hadoop/common/lib/xmlenc-0.52.jar:/home/gzg/javadev/lib/hadoop/common/lib/xz-1.0.jar:/home/gzg/javadev/lib/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/gzg/javadev/lib/hadoop/hdfs/hadoop-hdfs-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-app-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-core-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-common-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/mapreduce2/hadoop-mapreduce-client-shuffle-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-api-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-client-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-common-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.3.0-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-client-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-common-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-hadoop2-compat-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-hadoop-compat-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-it-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-prefix-tree-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-protocol-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-testing-util-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-thrift-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/protobuf-java-2.5.0.jar:/home/gzg/javadev/lib/hbase/hbase-server-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/hbase-shell-0.96.1.1-cdh5.0.2.jar:/home/gzg/javadev/lib/hbase/htrace-core-2.01.jar:/home/gzg/javadev/hbase1/bin:/usr/java/jdk1.7.0_60/jre/lib:/usr/lib64
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.io.tmpdir=/tmp
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:java.compiler=<NA>
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:os.name=Linux
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:os.arch=amd64
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:os.version=2.6.32-431.el6.x86_64
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:user.name=gzg
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:user.home=/home/gzg
2014-08-21 14:48:42,228 INFO [org.apache.zookeeper.ZooKeeper] - Client environment:user.dir=/home/gzg/javadev/hbase1
2014-08-21 14:48:42,230 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:42,254 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:42,261 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk2/192.168.5.122:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:42,276 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk2/192.168.5.122:2181, initiating session
2014-08-21 14:48:42,335 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk2/192.168.5.122:2181, sessionid = 0x1647ed72def6009d, negotiated timeout = 40000
2014-08-21 14:48:43,407 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:43,408 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:43,409 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk1/192.168.5.121:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:43,412 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk1/192.168.5.121:2181, initiating session
2014-08-21 14:48:43,421 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk1/192.168.5.121:2181, sessionid = 0x15478ef8bf2f016f, negotiated timeout = 40000
2014-08-21 14:48:53,961 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x15478ef8bf2f016f closed
2014-08-21 14:48:53,961 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:53,985 INFO [org.apache.hadoop.hbase.client.HBaseAdmin] - Started disable of _mrtest1
2014-08-21 14:48:54,051 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:54,053 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk3/192.168.5.123:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:54,057 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:54,067 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk3/192.168.5.123:2181, initiating session
2014-08-21 14:48:54,072 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk3/192.168.5.123:2181, sessionid = 0x174786daf57f01d0, negotiated timeout = 40000
2014-08-21 14:48:54,119 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x174786daf57f01d0 closed
2014-08-21 14:48:54,119 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:54,235 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:54,236 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:54,237 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk2/192.168.5.122:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:54,243 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk2/192.168.5.122:2181, initiating session
2014-08-21 14:48:54,281 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk2/192.168.5.122:2181, sessionid = 0x1647ed72def6009e, negotiated timeout = 40000
2014-08-21 14:48:54,325 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x1647ed72def6009e closed
2014-08-21 14:48:54,326 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:54,558 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:54,560 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk3/192.168.5.123:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:54,561 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:54,562 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk3/192.168.5.123:2181, initiating session
2014-08-21 14:48:54,568 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk3/192.168.5.123:2181, sessionid = 0x174786daf57f01d1, negotiated timeout = 40000
2014-08-21 14:48:54,584 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x174786daf57f01d1 closed
2014-08-21 14:48:54,584 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:54,894 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:54,899 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:54,900 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk3/192.168.5.123:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:54,901 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk3/192.168.5.123:2181, initiating session
2014-08-21 14:48:54,906 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk3/192.168.5.123:2181, sessionid = 0x174786daf57f01d2, negotiated timeout = 40000
2014-08-21 14:48:54,922 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x174786daf57f01d2 closed
2014-08-21 14:48:54,922 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:55,430 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x7f120aa6, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:55,431 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x7f120aa6 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:55,432 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk1/192.168.5.121:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:55,433 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk1/192.168.5.121:2181, initiating session
2014-08-21 14:48:55,439 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk1/192.168.5.121:2181, sessionid = 0x15478ef8bf2f0170, negotiated timeout = 40000
2014-08-21 14:48:55,456 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x15478ef8bf2f0170 closed
2014-08-21 14:48:55,457 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:55,459 INFO [org.apache.hadoop.hbase.client.HBaseAdmin] - Disabled _mrtest1
2014-08-21 14:48:55,638 INFO [org.apache.hadoop.hbase.client.HBaseAdmin] - Deleted _mrtest1
2014-08-21 14:48:55,638 INFO [org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation] - Closing master protocol: MasterService
2014-08-21 14:48:55,638 INFO [org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation] - Closing zookeeper sessionid=0x1647ed72def6009d
2014-08-21 14:48:55,658 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x1647ed72def6009d closed
2014-08-21 14:48:55,659 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:55,824 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=hconnection-0x30dd1e89, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:55,826 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0x30dd1e89 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:55,826 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk2/192.168.5.122:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:55,836 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk2/192.168.5.122:2181, initiating session
2014-08-21 14:48:55,855 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk2/192.168.5.122:2181, sessionid = 0x1647ed72def6009f, negotiated timeout = 40000
2014-08-21 14:48:56,473 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=180000 watcher=catalogtracker-on-hconnection-0x30dd1e89, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:56,474 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=catalogtracker-on-hconnection-0x30dd1e89 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:56,475 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk2/192.168.5.122:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:56,476 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk2/192.168.5.122:2181, initiating session
2014-08-21 14:48:56,482 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk2/192.168.5.122:2181, sessionid = 0x1647ed72def600a0, negotiated timeout = 40000
2014-08-21 14:48:56,500 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x1647ed72def600a0 closed
2014-08-21 14:48:56,501 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:56,502 INFO [org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation] - Closing master protocol: MasterService
2014-08-21 14:48:56,502 INFO [org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation] - Closing zookeeper sessionid=0x1647ed72def6009f
2014-08-21 14:48:56,506 INFO [org.apache.zookeeper.ZooKeeper] - Session: 0x1647ed72def6009f closed
2014-08-21 14:48:56,507 INFO [org.apache.zookeeper.ClientCnxn] - EventThread shut down
2014-08-21 14:48:56,508 INFO [root] - Table _mrtest1 created!
2014-08-21 14:48:56,909 INFO [org.apache.zookeeper.ZooKeeper] - Initiating client connection, connectString=zk2:2181,zk1:2181,zk3:2181 sessionTimeout=90000 watcher=hconnection-0x6a86ad83, quorum=zk2:2181,zk1:2181,zk3:2181, baseZNode=/hbase
2014-08-21 14:48:56,911 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server zk3/192.168.5.123:2181. Will not attempt to authenticate using SASL (unknown error)
2014-08-21 14:48:56,912 INFO [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0x6a86ad83 connecting to ZooKeeper ensemble=zk2:2181,zk1:2181,zk3:2181
2014-08-21 14:48:56,915 INFO [org.apache.zookeeper.ClientCnxn] - Socket connection established to zk3/192.168.5.123:2181, initiating session
2014-08-21 14:48:56,922 INFO [org.apache.zookeeper.ClientCnxn] - Session establishment complete on server zk3/192.168.5.123:2181, sessionid = 0x174786daf57f01d3, negotiated timeout = 40000
2014-08-21 14:48:56,965 INFO [org.apache.hadoop.hbase.mapreduce.TableOutputFormat] - Created table instance for _mrtest1
2014-08-21 14:49:07,111 INFO [org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider] - Failing over to rm2
2014-08-21 14:49:12,459 WARN [org.apache.hadoop.mapreduce.JobSubmitter] - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2014-08-21 14:49:25,381 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn1/192.168.5.111 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2: Name or service not known]; remaining name ‘111.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,385 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,386 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn3/192.168.5.113 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘113.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,387 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,387 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,388 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn1/192.168.5.111 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘111.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,389 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,389 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn4/192.168.5.114 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘114.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,390 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,390 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn6/192.168.5.116 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘116.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,413 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,414 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn2/192.168.5.112 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘112.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,415 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn5/192.168.5.115 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘115.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,415 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn2/192.168.5.112 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘112.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,416 ERROR [org.apache.hadoop.hbase.mapreduce.TableInputFormatBase] - Cannot resolve the host name for dn3/192.168.5.113 because of javax.naming.ConfigurationException: Unknown DNS server: nn1,nn2 [Root exception is java.net.UnknownHostException: nn1,nn2]; remaining name ‘113.5.168.192.in-addr.arpa‘
2014-08-21 14:49:25,595 INFO [org.apache.hadoop.mapreduce.JobSubmitter] - number of splits:15
2014-08-21 14:49:25,925 INFO [org.apache.hadoop.mapreduce.JobSubmitter] - Submitting tokens for job: job_1408378794442_0015
2014-08-21 14:49:26,170 INFO [org.apache.hadoop.yarn.client.api.impl.YarnClientImpl] - Submitted application application_1408378794442_0015
2014-08-21 14:49:26,219 INFO [org.apache.hadoop.mapreduce.Job] - The url to track the job: http://rm2:23188/proxy/application_1408378794442_0015/
2014-08-21 14:49:26,220 INFO [org.apache.hadoop.mapreduce.Job] - Running job: job_1408378794442_0015
2014-08-21 14:49:38,592 INFO [org.apache.hadoop.mapreduce.Job] - Job job_1408378794442_0015 running in uber mode : false
2014-08-21 14:49:38,594 INFO [org.apache.hadoop.mapreduce.Job] -  map 0% reduce 0%
2014-08-21 14:50:40,813 INFO [org.apache.hadoop.mapreduce.Job] -  map 7% reduce 0%
2014-08-21 14:50:47,911 INFO [org.apache.hadoop.mapreduce.Job] -  map 11% reduce 0%
2014-08-21 14:50:49,945 INFO [org.apache.hadoop.mapreduce.Job] -  map 13% reduce 0%
2014-08-21 14:51:03,145 INFO [org.apache.hadoop.mapreduce.Job] -  map 20% reduce 0%
2014-08-21 14:51:04,159 INFO [org.apache.hadoop.mapreduce.Job] -  map 20% reduce 4%
2014-08-21 14:51:10,265 INFO [org.apache.hadoop.mapreduce.Job] -  map 20% reduce 7%
2014-08-21 14:51:28,566 INFO [org.apache.hadoop.mapreduce.Job] -  map 24% reduce 7%
2014-08-21 14:51:30,598 INFO [org.apache.hadoop.mapreduce.Job] -  map 33% reduce 7%
2014-08-21 14:51:31,613 INFO [org.apache.hadoop.mapreduce.Job] -  map 33% reduce 9%
2014-08-21 14:51:34,656 INFO [org.apache.hadoop.mapreduce.Job] -  map 33% reduce 11%
2014-08-21 14:51:42,840 INFO [org.apache.hadoop.mapreduce.Job] -  map 40% reduce 11%
2014-08-21 14:51:46,912 INFO [org.apache.hadoop.mapreduce.Job] -  map 40% reduce 13%
2014-08-21 14:51:55,050 INFO [org.apache.hadoop.mapreduce.Job] -  map 47% reduce 13%
2014-08-21 14:51:56,074 INFO [org.apache.hadoop.mapreduce.Job] -  map 47% reduce 16%
2014-08-21 14:52:03,177 INFO [org.apache.hadoop.mapreduce.Job] -  map 53% reduce 16%
2014-08-21 14:52:08,241 INFO [org.apache.hadoop.mapreduce.Job] -  map 53% reduce 18%
2014-08-21 14:52:12,303 INFO [org.apache.hadoop.mapreduce.Job] -  map 60% reduce 18%
2014-08-21 14:52:14,343 INFO [org.apache.hadoop.mapreduce.Job] -  map 60% reduce 20%
2014-08-21 14:52:26,528 INFO [org.apache.hadoop.mapreduce.Job] -  map 64% reduce 20%
2014-08-21 14:52:27,551 INFO [org.apache.hadoop.mapreduce.Job] -  map 73% reduce 20%
2014-08-21 14:52:29,581 INFO [org.apache.hadoop.mapreduce.Job] -  map 73% reduce 22%
2014-08-21 14:52:32,621 INFO [org.apache.hadoop.mapreduce.Job] -  map 73% reduce 24%
2014-08-21 14:52:44,811 INFO [org.apache.hadoop.mapreduce.Job] -  map 80% reduce 24%
2014-08-21 14:52:47,848 INFO [org.apache.hadoop.mapreduce.Job] -  map 80% reduce 27%
2014-08-21 14:52:56,950 INFO [org.apache.hadoop.mapreduce.Job] -  map 84% reduce 27%
2014-08-21 14:52:57,966 INFO [org.apache.hadoop.mapreduce.Job] -  map 87% reduce 27%
2014-08-21 14:52:59,997 INFO [org.apache.hadoop.mapreduce.Job] -  map 87% reduce 29%
2014-08-21 14:53:42,487 INFO [org.apache.hadoop.mapreduce.Job] -  map 91% reduce 29%
2014-08-21 14:53:43,506 INFO [org.apache.hadoop.mapreduce.Job] -  map 93% reduce 29%
2014-08-21 14:53:45,540 INFO [org.apache.hadoop.mapreduce.Job] -  map 93% reduce 31%
2014-08-21 14:53:47,575 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 31%
2014-08-21 14:53:48,584 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 33%
2014-08-21 14:53:51,611 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 44%
2014-08-21 14:53:54,642 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 55%
2014-08-21 14:53:57,679 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 66%
2014-08-21 14:54:00,710 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 71%
2014-08-21 14:54:03,749 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 79%
2014-08-21 14:54:06,791 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 88%
2014-08-21 14:54:09,829 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 96%
2014-08-21 14:54:11,854 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 100%
2014-08-21 14:54:11,873 INFO [org.apache.hadoop.mapreduce.Job] - Job job_1408378794442_0015 completed successfully
2014-08-21 14:54:12,082 INFO [org.apache.hadoop.mapreduce.Job] - Counters: 61
	File System Counters
		FILE: Number of bytes read=324596614
		FILE: Number of bytes written=651096206
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=1105
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=15
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=0
	Job Counters
		Killed map tasks=6
		Launched map tasks=21
		Launched reduce tasks=1
		Data-local map tasks=15
		Rack-local map tasks=6
		Total time spent by all maps in occupied slots (ms)=5299964
		Total time spent by all reduces in occupied slots (ms)=413798
		Total time spent by all map tasks (ms)=2649982
		Total time spent by all reduce tasks (ms)=206899
		Total vcore-seconds taken by all map tasks=2649982
		Total vcore-seconds taken by all reduce tasks=206899
		Total megabyte-seconds taken by all map tasks=4070372352
		Total megabyte-seconds taken by all reduce tasks=423729152
	Map-Reduce Framework
		Map input records=14112896
		Map output records=14112896
		Map output bytes=296370816
		Map output materialized bytes=324596698
		Input split bytes=1105
		Combine input records=0
		Combine output records=0
		Reduce input groups=1
		Reduce shuffle bytes=324596698
		Reduce input records=14112896
		Reduce output records=1
		Spilled Records=28225792
		Shuffled Maps =15
		Failed Shuffles=0
		Merged Map outputs=15
		GC time elapsed (ms)=13168
		CPU time spent (ms)=403830
		Physical memory (bytes) snapshot=6978674688
		Virtual memory (bytes) snapshot=29039456256
		Total committed heap usage (bytes)=4911529984
	HBase Counters
		BYTES_IN_REMOTE_RESULTS=194636060
		BYTES_IN_RESULTS=5490524772
		MILLIS_BETWEEN_NEXTS=1834495
		NOT_SERVING_REGION_EXCEPTION=0
		NUM_SCANNER_RESTARTS=0
		REGIONS_SCANNED=15
		REMOTE_RPC_CALLS=5050
		REMOTE_RPC_RETRIES=0
		RPC_CALLS=141167
		RPC_RETRIES=0
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters
		Bytes Read=0
	File Output Format Counters
		Bytes Written=0
*** measure elapsed time,stoped at 2014-08-21 14:54:12:089
time elapsed :0days 0hours 5minutes 36seconds 209milliseconds[336.209seconds]

基于HBase Hadoop 分布式集群环境下的MapReduce程序开发