linux系统下单节点hadoop2的配置

Jdk安装:

jdk-7u45-linux-x64.gz

cp jdk-7u45-linux-x64.gz /usr/java/
cd /usr/java/
tar -zxvf jdk-7u45-linux-x64.gz

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.7
export HADOOP_INSTALL=/data/hadoop-2.2.0
export MAVEN_HOME=/usr/local/apache-maven-3.2.3
export PATH=/usr/local/mysql-5.5/bin:$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$MAVEN_HOME/bin

hadoop安装

hadoop-2.2.0-64bits.tar.gz

在data 文件夹下解压 hadoop 压缩包

cp hadoop-2.2.0-64bits.tar.gz /data/

tar -zxvf hadoop-2.2.0-64bits.tar.gz

配置 无验证ssh登陆

配置ssh
ssh-keygen -t rsa -P ""
cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Hadoop环境配置

不用单独建hadoop账户

我是在root权限下配置的

修改core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/hadoop/hadooptmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://10.232.82.221:9000</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri‘s scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri‘s authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>

修改hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>

</configuration>

修改mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

<property>
<name>mapred.job.tracker</name>
<value>10.232.82.221:9001</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>

</configuration>

1.在没有数据的情况下,先格式化再启动

格式化命令:hadoop namenode -format

2.我的文件是在sbin下:

启动命令:sbin/./start-all.Sh

3.jps命令查看进程

4.bin下执行hadoop fs -ls / 

5.ip:50070 和 ip:50030 在浏览器中监测进程

有问题可以一起交流留言

时间: 2024-10-27 06:56:02

linux系统下单节点hadoop2的配置的相关文章

64位ubuntu下单节点安裝配置hadoop2.5.0

1.在Ubuntu下创建hadoop用户组和用户 2.在Ubuntu下安装JDK(可以在官网下载linux包然后解压安装) 3.hadoop下载:http://www.apache.org/dyn/closer.cgi/hadoop/common/ 因为64位ubuntu需要编译源文件:故下载hadoop-2.5.0-src.tar.gz (1)编译时需要安装protobuf-2.5.0,解压下载的protobuf-2.5.0.tar.gz,进入protobuf-2.5.0目录,执行下面命令 .

linux系统vps服务器简单安全配置

虽然说linux 系统比windows安全性要高一些,不过一些简单的安全配置也是必要的.因为毕竟现在市面的破译工具太多了,如可以社会工程学你密码.那么就一起看学习下入门级安全配置吧. 工具/原料 linux 系统 vps或者服务器 DDos deflate 方法/步骤 第一.远程管理端口.市面上有很多工具采用字典方式套取你的管理员密码,我们至少可以创造一些麻烦出来.修改ssh 的登陆端口,默认端口是22.扫描字典穷举密码他们都是从默认的开始.如果你把端口改成4位数的也大大提高了他们的难度及时间.

Linux系统FTP服务器搭建与配置

一.简介 1.简介 File Transfer Protocol(文件传输协议)的英文简称,而中文简称为"文件传输协议".用于Internet上的控制文件的双向传输.同时,它也是一个应用程序(Application).基于不同的操作系统有不同的FTP应用程序,而所有这些应用程序都遵守同一种哦协议以传输文件.在FTP的使用当中,用户经常遇到两个概念:"下载(Download)"和"上传(Upload)". 下载文件就是将文件从远程主机拷贝到自己的计

linux系统中apache虚拟目录配置

在搭建网站的时候,我们会经常在同一台服务器上面搭建多个站点,这时候就需要用到apache的虚拟机知识.下面就linux系统下apache虚拟目录的配置说简要说明: 1.源代码安装apache + PHP + MySQL(请查考我的博文lamp环境搭建): 2.在本地计算机的hosts文件加入如下行: 192.168.137.10 erp.100msh.com 192.168.137.10 mopadmin.100msh.com 然后cmd中 ping erp.100msh.com 检查是否网络通

Linux系统 /etc/security/limits.conf 配置

1. limits.conf 配置文件描述 limits.conf文件实际是Linux PAM(插入式认证模块,Pluggable Authentication Modules)中 pam_limits.so 的配置文件,突破系统的默认限制,对系统访问资源有一定保护作用. limits.conf 和sysctl.conf区别在于limits.conf是针对用户,而sysctl.conf是针对整个系统参数配置. 2. limits.conf工作原理 limits.conf是 pam_limits.

让虚拟机中的Linux系统上Internet网的配置方法

 1.背景情况 在宿主主机可以上网的情况下,虚拟机中的操作系统一般由NAT模式与宿主主机通信, 默认情况下虚拟机是无法上internet网的.但是经过正确的配置,完全可以让Vmware中的虚拟Linux系统一样可以上网 2.配置方法 2.1.查看vmware上的NAT网络地址与网关地址 (1)网络地址查看与验证 点击"编辑"à"虚拟网络编辑器" 从上面看到NAT的网段是192.168.2.0 (2)查看NAT设置,获取NAT的网关地址 2.2虚拟机内系统配置 2

保证Linux系统安全之firewalld防火墙配置地址伪装和端口转发详解

通过保证Linux系统安全之firewalld防火墙入门详解认识Linux系统firewalld防火墙,并可以编写一些相对简单一些的防火墙规则.Linux防火墙可以充当路由器(网关).路由器上的NAT技术,同样可以通过Linux防火墙来实现.地址伪装和端口转发说白了就是路由器中的NAT技术. 一.地址伪装和端口转发简介 firewalld防火墙支持两种类型的NAT: (1)地址伪装 地址伪装:基于源地址进行转换,通过地址伪装,NAT设备将经过设备的数据包转发到指定接收方,同时将通过的数据包的源地

虚拟机下Linux系统Hadoop单机/伪分布式配置:Hadoop2.5.2+Ubuntu14.04(半原创)

系统: Ubuntu 14.04 64bit Hadoop版本: Hadoop 2.5.2 (stable) JDK版本: JDK 1.6 虚拟机及Ubuntu安装 1. 下载并安装 VMware workstation 11 下载地址:https://my.vmware.com/web/vmware/info/slug/desktop_end_user_computing/vmware_workstation/11_0?wd=%20VMware%20workstation%2011%20&is

linux系统的初化始配置(包括网络,主机名,关闭firewalld与selinux)

每次我们使用Linux都会对系统进行初始化的配置,下面我们一一列出来. 1.服务的开启 systemctl enable firewalld.service //将指定的服务设置为开机启动 systemctl disable firewalld.service //在每次重启开机时间,不自动启动这个服务 systemctl is-enabled firewalld.service //查询是否设置为开机自动启动 2.服务的临时开启.关闭.重启. systemctl start filewalld