数学之路-分布式计算-hadoop、python与R(3)

二、ubuntu 12.04hadoop2.6配置与安装

1、java7配置

#su

#addusermyhaspl

下面增加su权限

使得myhaspl用户属于sudo组

#vi/etc/group

sudo:x:27:,myhaspl

然后重新启动

#reboot

转到myhaspl用户，开始下载解压java。

本博客所有内容是原创，如果转载请注明来源

http://blog.csdn.net/myhaspl/

#sumyhaspl

$wgethttp://download.oracle.com/otn-pub/java/jdk/7u71-b14/jdk-7u71-linux-x64.tar.gz

[email protected]:~$tar -xzvf jdk-7u71-linux-x64.tar.gz

[email protected]:~$sudo mv jdk1.7.0_71 /usr/lib/java

[email protected]:/home/myhaspl#vi  /etc/profile

exportJAVA_HOME=/usr/lib/java

exportJRE_HOME=${JAVA_HOME}/jre

exportCLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

exportPATH=${JAVA_HOME}/bin:$PATH

#reboot

[email protected]:~$sudo update-alternatives --install /usr/bin/java java /usr/lib/java/bin/java300

update-alternatives:using /usr/lib/java/bin/java to provide /usr/bin/java (java) in auto mode

[email protected]:~$sudo update-alternatives --install /usr/bin/javac javac /usr/lib/java/bin/javac300

update-alternatives:using /usr/lib/java/bin/javac to provide /usr/bin/javac (javac) in auto mode

[email protected]:~$  java -version

javaversion "1.7.0_71"

Java(TM)SE Runtime Environment (build 1.7.0_71-b14)

JavaHotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

[email protected]:~$

[email protected]:~$vi HelloWorldjava

publicclass HelloWorld{

publicstatic void main (String[] args){

System.out.println("HelloWorld");

}

}

[email protected]:~$javac HelloWorld.java

[email protected]:~$java HelloWorld

HelloWorld

[email protected]:~$sudo apt-get install ssh

[email protected]:~$sudo apt-get install rsync

2、hadoop2.6

下载解压hadoop

[email protected]:~$wget http://mirrors.cnnic.cn/apache/hadoop/common/stable/hadoop-2.6.0.tar.gz

[email protected]:~$tar -xzvf hadoop-2.6.0.tar.gz

将hadoop移动到系统文件夹，并授予相关权限。

[email protected]:~$sudo mv hadoop-2.6.0 /usr/local/

[email protected]:~$cd /usr/local/

[email protected]:~$sudo chown -R root:root hadoop-2.6.0/

[email protected]:/home/myhaspl#vi  /etc/profile

exportHADOOP_PREFIX=/usr/local/hadoop-2.6.0

[email protected]:/usr/local/hadoop-2.6.0$  export HADOOP_PREFIX=/usr/local/hadoop

[email protected]:/usr/local/hadoop-2.6.0$cd ~

显示hadoop文档

[email protected]:~$cd/usr/local/hadoop-2.6.0

[email protected]:~$bin/hadoop

Usage:hadoop [--config confdir] COMMAND

where COMMAND is one of:

fs                   run a generic filesystemuser client

version              print the version

jar <jar>            run a jar file

checknative [-a|-h]  check native hadoop and compression librariesavailability

distcp <srcurl> <desturl> copyfile or directories recursively

archive -archiveName NAME -p <parentpath> <src>* <dest> create a hadoop archive

classpath            prints the class path needed to get the

credential           interact with credential providers

Hadoop jar and therequired libraries

daemonlog            get/set the log level for eachdaemon

trace                view and modify Hadoop tracingsettings

or

CLASSNAME            run the class named CLASSNAME

Mostcommands print help when invoked w/o parameters.

伪分布式

hdfs的地址及端口号

[email protected]:~$sudo vi /usr/local/hadoop-2.6.0/etc/hadoop/core-site.xml <configuration>

<property>

<name>fs.defaultFS</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

~

hdfs备份数量,改为1

etc/hadoop/hdfs-site.xml:

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

</configuration>