1.准备环境
虚拟机（redhat enterprise linux 6.5）
jdk-8u92-linux-x64.tar.gz
hadoop-0.20.2.tar.gz
2.关闭虚拟机的防火墙，selinux，配置SSH免密码登录
[[email protected] ~]# vim /etc/sysconfig/selinux
[[email protected] ~]# iptables -F
[[email protected] ~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]
[[email protected] ~]# service iptables stop
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
[[email protected] ~]# chkconfig iptables off
[[email protected] ~]# chkconfig iptables --list
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[[email protected] ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): #这里直接回车
Enter same passphrase again: #这里直接回车
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
64:d2:fa:ee:61:ef:29:b0:c8:55:1e:6b:a3:6d:1b:d4 [email protected]
The key‘s randomart image is:
+--[ RSA 2048]----+
| |
| . |
| . + |
| =o. |
| .oSoE |
| oo= |
| . o *=. |
| o oo++ . |
| o+++ |
+-----------------+
[[email protected] ~]# ssh-copy-id localhost #这里写localhost或者主机名均可
The authenticity of host ‘localhost (::1)‘ can‘t be established.
RSA key fingerprint is a5:c4:4e:54:ea:2d:72:3f:9e:65:a2:ac:cd:41:ce:ca.
Are you sure you want to continue connecting (yes/no)? yes #这里输入yes
Warning: Permanently added ‘localhost‘ (RSA) to the list of known hosts.
[email protected]‘s password: #这里输入密码
Now try logging into the machine, with "ssh ‘localhost‘", and check in:
.ssh/authorized_keys
to make sure we haven‘t added extra keys that you weren‘t expecting.
测试登陆下
[[email protected] ~]# ssh localhost
Last login: Sat Oct 8 17:16:27 2016 from sishen.161.cn
[[email protected] ~]# exit
logout
Connection to localhost closed.
成功！！！
3.配置环境
首先解压
[[email protected] ~]# tar -xf jdk-8u92-linux-x64.tar.gz -C /usr/src/hadoop/
然后编辑/etc/profile文件
[[email protected] ~]# vim /etc/profile
末尾添加以下内容
79 export JAVA_HOME=/usr/src/hadoop/jdk1.8.0_92
80 export HADOOP_HOME=/usr/src/hadoop/hadoop-0.20.2
81 export PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin
保存退出
测试
[[email protected] ~]# java -version
java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)
成功！！！
然后开始配置hadoop的配置文件
[[email protected] hadoop-0.20.2]# cd /usr/src/hadoop/hadoop-0.20.2/conf/
[[email protected] conf]# ls
capacity-scheduler.xml hadoop-policy.xml slaves
configuration.xsl hdfs-site.xml ssl-client.xml.example
core-site.xml log4j.properties ssl-server.xml.example
hadoop-env.sh mapred-site.xml
hadoop-metrics.properties masters
标红的文件是我们要编辑的
首先编辑hadoop-env.sh ，使用vim打开文件后，找到 # export JAVA_HOME=/usr/lib/j2sdk1.5-sun（大约在第9行左右），然后在此行下面添加如下内容
export JAVA_HOME=/usr/src/hadoop/jdk1.8.0_92，保存并退出
然后编辑core-site.xml文件，使用vim打开之后找到<configuration>,修改为以下内容
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
编辑hdfs-site.xml文件，在<configuration>标签内添加如下内容（数字是行号！不用写数字）
7 <property>
8 <name>dfs.data.dir</name>
9 <value>/usr/src/hadoop/hadoop-0.20.2/data</value>
10 </property>
11 <property>
12 <name>dfs.replication</name>
13 <value>1</value>
14 </property>
编辑mapred-site.xml文件，找到<configuration>标签后，在添加如下内容（数字为行号！不用写数字）
7 <property>
8 <name>mapred.job.tracker</name>
9 <value>localhost:9001</value>
10 </property>
保存退出以后，开始格式化
[[email protected] ~]# hadoop namenode –format
16/10/09 11:42:11 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = sishen.161.cn/192.168.186.161
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by ‘chrisdo‘ on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
16/10/09 11:42:12 INFO namenode.FSNamesystem: fsOwner=root,root
16/10/09 11:42:12 INFO namenode.FSNamesystem: supergroup=supergroup
16/10/09 11:42:12 INFO namenode.FSNamesystem: isPermissionEnabled=true
16/10/09 11:42:12 INFO common.Storage: Image file of size 94 saved in 0 seconds.
16/10/09 11:42:12 INFO common.Storage: Storage directory /tmp/hadoop-root/dfs/name has been successfully formatted. //格式化成功的标志
16/10/09 11:42:12 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at sishen.161.cn/192.168.186.161
************************************************************/
启动hadoop集群
[[email protected] ~]# start-all.sh
检查
[[email protected] ~]# jps
3058 TaskTracker
2898 SecondaryNameNode
2694 NameNode
2966 JobTracker
2790 DataNode
3111 Jps
停止hadoop集群
[[email protected] ~]# stop-all.sh
stopping jobtracker
localhost: stopping tasktracker
stopping namenode
localhost: stopping datanode
localhost: stopping secondarynamenode
[[email protected] ~]# jps
3426 Jps
至此hadoop-0.20.2伪分布式安全完成！