集成hive 的前提是apache hadoop 集群能够正常启动。
hadoop 版本 apach2.6.0 hive 版本:1.2.1
1、安装mysql 并赋予权限:
1.1:创建hive 用户和密码:
Create user ‘hive’ identified by ‘123456’
1.2:创建数据库:create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
1.3:授权允许远程登陆:grant all privileges on *.* to ‘user‘@‘hostname‘ identified by ‘passwd‘ with grant option;
flush privileges;
1.4 将mysql 驱动复制到 hive/lib 目录下
mysql-connector-java-5.1.38-bin.jar
2、hive 配置文件在conf 目录下:hive-env.sh
cp hive-env.sh.template hive-env.sh
最后添加如下配置:
export JAVA_HOME=/usr/java/jdk1.7.0_79 export HIVE_HOME=/home/hadoop/hive-1.2.1-bin export HADOOP_HOME=/home/hadoop/hadoop-2.6.0
3、hive-site.xml
3.1hdfs 上的文件需要事先创建,如果出现权限问题修改对应的权限即可:hadoop dfs -chmod -R 777 hdfs://node17:9000/hive/scratchdir
3.2需要将下述jar文件复制到hive/lib目录下
home/hadoop/hive-1.2.1-bin/lib/hive-hbase-handler-1.2.1.jar
file:///home/hadoop/hive-1.2.1-bin/lib/protobuf-java-2.5.0.jar,
file:///home/hadoop/hive-1.2.1-bin/lib/hbase-client-1.0.0.jar,
file:///home/hadoop/hive-1.2.1-bin/lib/hbase-common-1.0.0.jar,
file:///home/hadoop/hive-1.2.1-bin/lib/zookeeper-3.4.6.jar,
file:///home/hadoop/hive-1.2.1-bin/lib/guava-12.0.1.jar
<configuration><property><name>hive.metastore.warehouse.dir</name><value>hdfs://node17:9000/hive/warehouse</value></property><property><name>hive.exec.scratchdir</name><value>hdfs://node17:9000/hive/scratchdir</value></property><property><name>hive.querylog.location</name><value>/home/hadoop/hive-1.2.1-bin/logs</value></property><property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://node15:3306/hive?createDatabaseIfNotExist=true</value></property> <property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value></property> <property><name>javax.jdo.option.ConnectionUserName</name><value>root</value></property><property> <name>javax.jdo.option.ConnectionPassword</name><value>123456</value></property> <property><name>hive.aux.jars.path</name><value>file:///home/hadoop/hive-1.2.1-bin/lib/hive-hbase-handler-1.2.1.jar ,file:///home/hadoop/hive-1.2.1-bin/lib/protobuf-java-2.5.0.jar,file:///home/hadoop/hive-1.2.1-bin/lib/hbase-client-1.0.0.jar,file:///home/hadoop/hive-1.2.1-bin/lib/hbase-common-1.0.0.jar,file:///home/hadoop/hive-1.2.1-bin/lib/zookeeper-3.4.6.jar,file:///home/hadoop/hive-1.2.1-bin/lib/guava-12.0.1.jar</value></property> <property><name>hive.metastore.uris</name><value>thrift://node17:9083</value> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/home/hadoop/hive-1.2.1-bin/tmp</value> </property> </configuration>
出现问题:
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
解决办法:
启动metastore service服务即可。
./hive --service metastore
解决办法:
删除 yarn 下的 jline-0.9.94.jar jar 包即可。
./hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar