一、搭建hadoop环境

《OD大数据实战》hadoop伪分布式环境搭建

二、hive环境搭建

1. 准备安装文件

下载地址：

http://archive.cloudera.com/cdh5/cdh/5/

hive-0.13.1-cdh5.3.6.tar.gz

2. 解压

tar -zxvf hive-0.13.1-cdh5.3.6.tar.gz -C /opt/modules/cdh/

3. 修改配置

cd /opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf
mv hive-env.sh.template hive-env.sh
mv hive-default.xml.template hive-site.xml
mv hive-exec-log4j.properties.template hive-exec-log4j.properties
mv hive-log4j.properties.template hive-log4j.properties

1）修改hive-env.sh

#增加一行
export JAVA_HOME=/opt/modules/jdk1.7.0_67

HADOOP_HOME=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6

export HIVE_CONF_DIR=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/conf

2）修改hive-log4j.properties

hive.log.dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/logs

3）修改hive-exec-log4j.properties

hive.log.dir=/opt/modules/cdh/hive-0.13.1-cdh5.3.6/logs

4）修改hive-site.xml

<property>
  <name>hive.lazysimple.extended_boolean_literal</name>
  <value>false</value>
  <description>
    LazySiimpleSerde uses this properties to determine if it treats ‘T‘, ‘t‘, ‘F‘, ‘f‘,
    ‘1‘, and ‘0‘ as extened, legal boolean literal, in addition to ‘TRUE‘ and ‘FALSE‘.
    The default is false, which means only ‘TRUE‘ and ‘FALSE‘ are treated as legal
    boolean literal.
  </description>
</property>

<property>
  <name>hive.mapjoin.optimized.hashtable</name>
  <value>true</value>
  <description>Whether Hive should use memory-optimized hash table for MapJoin. Only works on Tez, because memory-optimized hashtable cannot be serialized.</description>
</property>

4. 验证hive环境结果

bin/hive
dfs -ls /;

三、mysql环境搭建

1. 目标是安装mysql 5.1.17

2. 在官网下载yum源

http://dev.mysql.com/downloads/repo/yum/

http://repo.mysql.com//mysql57-community-release-el6-8.noarch.rpm

3. 安装yum源到/etc/yum.repos.d/目录

sudo rpm -Uvh mysql57-community-release-el6-8.noarch.rpmcd /etc/yum.repos.d/

4. 修改yum源配置

修改文件：mysql-community.repo 和mysql-community-resource.repo

5.6 enable = 1

5.7 enable = 0

5. 安装mysql

sudo yum -y install mysql-community-server

6. mysql安全性设置

sudo mysql_secure_installation

grant all privileges on *.* to ‘root‘@‘%‘ identified by ‘beifeng‘ with grant option

7. 验证mysql安装结果

进入命令行： mysql -uroot -p

四、本地mysql作为metastore模式

五、远程mysql作为metastore模式