HDP-2.6.5 & ambari-2.6.2 安装配置

安装文件列表

* ambari.repo
* ambari-2.6.2.2-centos7.tar.gz
* hdp.repo
* HDP-2.6.5.0-centos7-rpm.tar.gz
* HDP-GPL-2.6.5.0-centos7-gpl.tar.gz
* HDP-UTILS-1.1.0.22-centos7.tar.gz
* apache-tomcat-8.5.27.tar.gz
* mysql-connector-java-5.1.46.tar.gz

自行至 https://hortonworks.com 下载

部署环境

System:   CentOS 7.x
JDK:           JDK-1.8.xxx
Tomcat:      Tomcat-8.5.27 (或者使用 Nginx、Httpd)
MariaDB:    MariaDB-5.5.60

集群角色

Ambari-Server:    10.0.2.20
Ambari-Agent01: 10.0.2.21
Ambari-Agent02: 10.0.2.22
cat >>/etc/hosts<<EOF
10.0.2.20  Ambari-Server         server.ambari.com
10.0.2.21  Ambari-Agent01      agent01.ambari.com
10.0.2.22  Ambari-Agent02      agent02.ambari.com
EOF

解决依赖

/usr/bin/yum install zlib zlib-devel libjpeg libjpeg-devel libpng libpng-devel gd gd-devel curl curl-devel freetype freetype-devel libxslt libxslt-devel libxml2 libxml2-devel ncurses ncurses-devel gcc gcc-c++ flex flex-devel libtiff-devel pam-devel libffi-devel e2fsprogs-devel openssl* ntpdate tree autoconf lrzsz uuid dstat bind-utils vim rsync net-tools wget -y
依赖问题根据自己服务器情况自行安装,上述依赖是我的服务器基础优化安装的依赖包

配置JAVA环境

tar xf jdk-8u162-linux-x64.tar.gz -C /data/
cd /data;ln -s jdk1.8.0_162 jdk;cd
cat >>/etc/profile<<EOF
export JAVA_HOME=/data/jdk
export JRE_HOME=\$\{JAVA_HOME\}/jre
export CLASS_PATH=.:\$\{JAVA_HOME\}/lib:\$\{JRE_HOME\}/lib
export PATH=\$\{JAVA_HOME\}/bin:\$\{JRE_HOME\}/bin:\$\{PATH\}
EOF
. /etc/profile

配置免密登陆

[[email protected] ~]$ ssh-keygen      # 一路回车至结束
[[email protected] .ssh]$ ssh-copy-id 10.0.2.20
[[email protected] .ssh]$ for n in 21 22;do scp authorized_keys id_rsa id_rsa.pub 10.0.2.$n:/root/.ssh/;done
# 为去除‘yes‘输入,执行下面的命令
[[email protected] ~]$ for n in Ambari-Server Ambari-Agent01 Ambari-Agent02 server.ambari.com agent01.ambari.com agent02.ambari.com;do ssh $n "uptime";done

同步hosts文件

[[email protected] ~]$ for n in 21 22;do scp /etc/hosts 10.0.2.$n:/etc/;done

安装MariaDB

[[email protected] ~]$ yum install mariadb-server mariadb -y
添加配置 skip-name-resolve=on 到 /etc/my.cnf
[[email protected] ~]$ systemctl start mariadb.service
[[email protected] ~]$ mysqladmin -uroot password 123456
[[email protected] yum.repos.d]$ mysql -uroot -p123456 -e "create database ambari;"
[[email protected] yum.repos.d]$ mysql -uroot -p123456 -e "grant all privileges on ambari.* to [email protected]‘localhost‘ identified by ‘123456‘;"
[[email protected] yum.repos.d]$ mysql -uroot -p123456 -e "grant all privileges on ambari.* to [email protected]‘10.0.2.20‘ identified by ‘123456‘;"
[[email protected] yum.repos.d]$ mysql -uroot -p123456 -e "flush privileges;"

安装Tomcat

[[email protected] ~]$ tar xf apache-tomcat-8.5.27.tar.gz -C /data/
[[email protected] data]$ cd /data/;ln -s apache-tomcat-8.5.27 tomcat;cd
[[email protected] ~]$ tar xf mysql-connector-java-5.1.46.tar.gz
[[email protected] ~]$ mv mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /data/tomcat/lib/
[[email protected] ~]$ mkdir /usr/share/java
[[email protected] ~]$ mv mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar /usr/share/java/

配置Ambari、HDP本地yum库

[[email protected] hdp]$ rm -fr /data/tomcat/webapps/*
[[email protected] hdp]$ mkdir /data/tomcat/webapps/hdp/HDP-UTILS -p
[[email protected] conf]$ tar xf ambari-2.6.2.2-centos7.tar.gz -C /data/tomcat/webapps/hdp
[[email protected] conf]$ tar xf ambari-2.6.2.2-centos7.tar.gz -C /data/tomcat/webapps/hdp
[[email protected] conf]$ tar xf HDP-GPL-2.6.5.0-centos7-gpl.tar.gz -C /data/tomcat/webapps/hdp
[[email protected] conf]$ tar xf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /data/tomcat/webapps/hdp
[[email protected] conf]$ tar xf HDP-2.6.5.0-centos7-rpm.tar.gz -C /data/tomcat/webapps/hdp
[[email protected] conf]$ sed -i ‘112c<param-value>true</param-value>‘ web.xml
[[email protected] bin]$ /data/tomcat/bin/startup.sh
# 官方repo文件位置
[[email protected] 2.6.2.2-1]$ ll /data/tomcat/webapps/hdp/ambari/centos7/2.6.2.2-1/ambari.repo
-rw-r--r-- 1 root root 304 6月  12 2018 /data/tomcat/webapps/hdp/ambari/centos7/2.6.2.2-1/ambari.repo
[[email protected] 2.6.5.0-292]$ ll /data/tomcat/webapps/hdp/HDP/centos7/2.6.5.0-292/hdp.repo
-rw-r--r-- 1 root root 575 5月  15 2018 /data/tomcat/webapps/hdp/HDP/centos7/2.6.5.0-292/hdp.repo
[[email protected] 2.6.5.0-292]$ ll /data/tomcat/webapps/hdp/HDP-GPL/centos7/2.6.5.0-292/hdp-gpl.gpl.repo
-rw-r--r-- 1 root root 310 5月  15 2018 /data/tomcat/webapps/hdp/HDP-GPL/centos7/2.6.5.0-292/hdp-gpl.gpl.repo
[[email protected]i-Server 1.1.0.22]$ ll /data/tomcat/webapps/hdp/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/hdp-utils.repo
-rw-r--r-- 1 root root 197 5月  29 2018 /data/tomcat/webapps/hdp/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/hdp-utils.repo
# 参照官方repo文件,创建本地ambari.repo文件
[[email protected] yum.repos.d]$ cat >/etc/yum.repos.d/ambari.repo<<EOF
#VERSION_NUMBER=2.6.2.2-1
[ambari-2.6.2.2]
name=ambari Version - ambari-2.6.2.2
baseurl=http://Ambari-Server:8080/hdp/ambari/centos7/2.6.2.2-1
gpgcheck=0
gpgkey=http://Ambari-Server:8080/hdp/ambari/centos7/2.6.2.2-1/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

#VERSION_NUMBER=2.6.5.0-292
[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0
baseurl=http://Ambari-Server:8080/hdp/HDP/centos7/2.6.5.0-292
gpgcheck=0
gpgkey=http://Ambari-Server:8080/hdp/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://Ambari-Server:8080/hdp/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22
gpgcheck=0
gpgkey=http://Ambari-Server:8080/hdp/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

#VERSION_NUMBER=2.6.5.0-292
[HDP-GPL-2.6.5.0]
name=HDP-GPL Version - HDP-GPL-2.6.5.0
baseurl=http://Ambari-Server:8080/hdp/HDP-GPL/centos7/2.6.5.0-292
gpgcheck=0
gpgkey=http://Ambari-Server:8080/hdp/HDP-GPL/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
EOF
# 同步ambari.repo至所有点节
[[email protected] yum.repos.d]$ for n in 21 22;do scp /etc/yum.repos.d/ambari.repo 10.0.2.$n:/etc/yum.repos.d/;done
# 更新yum cache
[[email protected] yum.repos.d]$ for n in 20 21 22;do ssh 10.0.2.$n "yum clean all";done
[[email protected] yum.repos.d]$ for n in 20 21 22;do ssh 10.0.2.$n "yum repolist";done

安装配置Ambari-Server

[[email protected] ~]$ yum install  ambari-server -y
[[email protected] ambari-server]$ echo "client.api.port=9000" >>/etc/ambari-server/conf/ambari.properties
[[email protected] ~]$ echo "server.jdbc.driver.path=/data/tomcat/lib/mysql-connector-java-5.1.46-bin.jar" >>/etc/ambari-server/conf/ambari.properties
# 设置Ambari
[[email protected] ~]$ ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is ‘disabled‘
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /data/jdk
Validating JDK on Ambari Server...done.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (123456):
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
Extracting system views...
ambari-admin-2.6.2.2.1.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server ‘setup‘ completed successfully.
# 根据设置过程中的提示执行SQL
[[email protected] ~]$ mysql -uroot -p123456 -e "use ambari;source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;"
[[email protected] ~]$ mysql -uroot -p123456 -e "show tables from ambari;"
# 启动Ambari-server,因为systemctl没有启动信息输出,不使用systemctl启动
[[email protected] ambari-server]$ /etc/init.d/ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start..............................................
Server started listening on 9000

DB configs consistency check: no errors and warnings were found.
Ambari Server ‘start‘ completed successfully.

安装配置Ambari-agent

[[email protected] ~]$ for n in 20 21 22;do ssh 10.0.2.$n "yum install ambari-agent -y";done
[[email protected] conf]$ sed -i "s#hostname=localhost#hostname=Ambari-Server#" /etc/ambari-agent/conf/ambari-agent.ini
# 强制指定安全协议
[[email protected] conf]$ sed -i ‘/\[security\]/a\force_https_protocol=PROTOCOL_TLSv1_2‘ /etc/ambari-agent/conf/ambari-agent.ini
# 同步配置文件至所有节点
[[email protected] conf]$ for n in 21 22;do scp /etc/ambari-agent/conf/ambari-agent.ini 10.0.2.$n:/etc/ambari-agent/conf/;done
# 启动所有节点Agent
[[email protected] conf]$ for n in 20 21 22;do ssh 10.0.2.$n "/etc/init.d/ambari-agent start";done

页面配置Ambari****

默认登陆权限:admin   admin

以下省略......

原文地址:http://blog.51cto.com/784687488/2329891

时间: 2024-10-26 00:30:46

HDP-2.6.5 & ambari-2.6.2 安装配置的相关文章

ambari之hadoop的安装

前期环境准备: 三台机器:192.168.1.25(1-3):251-ambari_master:252-ambari_slare1:253-ambari_slare2 机器配置,系统环境 使用虚拟机,系统rhel6.6 ; centos6.5 所有机器都需要执行一下设置(我已经设置好了,可以看一下): 1.打开最大文件数设置 [[email protected]_slare2 ~]# ulimit -Hn10000 [[email protected]_slare2 ~]# ulimit -S

ambari 2.6.2 安装 hdp 2.6.5.0 遇到的问题

1.hive-client 无法安装 一直报错(symlink target  already exists and it is not a symlink.),hive-client 已经存在且不是符号连接,查看确实如此 试了很多种办法,比如重新安装,比如手动删除,手动连接,都没有奏效 DO: 最后通过查看错误日志,拿到执行报错的脚本(ambari-python-wrap /usr/bin/hdp-select set hive-client 2.6.5.0-292),删除存在的目录后,再执行

基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库之一 —— MariaDB 安装配置

一.安装并使用MariaDB作为Ambari.Hive.Hue的存储数据库. yum install mariadb-server mariadb 启动.查看状态,检查mariadb是否成功安装 systemctl start mariadb systemctl status mariadb 二.配置mariadb 1,首先停止MariaDb 服务的运行 systemctl stop mariadb 2,编辑/etc/my.cnf 在[mysqld]段下,添加如下几行行配置: transacti

ambari本地源自动化安装hortonworks hadoop(转)

原文地址:http://blog.chinaunix.net/xmlrpc.php?r=blog/article&uid=26230811&id=4023821 环境配置 Hostname IP OS Ambari 172.16.235.164 Centos X64 base install Maste.hadoop 172.16.235.165 Centos X64 base install Datanode1.hadoop 172.16.235.166 Centos X64 base

[原创]CentOS 6 Ambari Hadoop平台的安装(本地库)

1. 安装前的准备 1.1 硬件要求 无特殊需求(根据Hadoop各个组件需求及实际情况,一般建议单个集群的机器数量在3台及以上:1台Master结点,2台机器作为Slave结点). 1.2 软件要求 1.2.1 操作系统 仅支持如下操作系统: Red Hat Enterprise Linux (RHEL) v5.x or 6.x (64-bit) CentOS v5.x or 6.x (64-bit) SUSE Linux Enterprise Server (SLES) 11, SP1 (6

Ambari 安装配置 MySql

在安装Ambari时,Ambari默认的数据库是ProstgreSQL,对ProstgreSQL不太熟悉,选择使用MySql. 但Centos 7 默认支持的是MariaDB数据库. MariaDB是MySQL的一个分支,主要由开源社区在维护.安装过程中先将Centos7默认安装的MariaDB数据库删除,再重新安装MySql. 安装步骤: 软件下载  (mysql的rpm包有很多文件,只需要安装Server 和 Client) 数据库安装 用户权限配置 安装目录及功能 常用创建用户方法 安装软

ambari 2.7 下安装httpfs

主要参考https://henning.kropponline.de/2014/11/21/install-httpfs-gateway-kerberized-cluster/ambari2.7安装的hadoop 集群,没有自带httpfs,需要在hadoop的配置目录下手动创建httpfs-site.xml,内容参考上面的链接,主要注意kerberos认证的步骤 然后是在hadoop的配置目录下新建httpfs-signature.secret文件内容如下:hadoop httpfs secr

在先电IAAS平台中,搭建先电bigdata平台

以两台节点为例来组件Hadoop分布式集群,这里采用的系统版本为Centos7,如下表所示: 主机名 内存 硬盘 IP地址 角色 master 8192MB 100G 192.168.200.131 Ambari-Server Ambari-Agent slave1 4096MB 100G 192.168.200.133 Ambari-Agent 1.1 配置主机名 # master # hostnamectl set-hostname master # hostname master # sl

ambari+hdp安装

Yun2集群上Ambari+HDP安装 1. 安装环境说明: 系统版本:CentOS7 安装版本: Ambari版本:2.2.2.0 HDP版本:2.4.2.0 集群机器: Ambari Server :yun2 Ambari Agent:yun3 ,yun4,yun6,yun18,yun19,yun20,yun21 2. 操作系统环境准备 2.1 配置SSH免密码登录 主节点里root用户登录执行如下步骤: ssh-keygen –t rsa cd /root/.ssh cat id_rsa.

Ambari HDP集群搭建文档_zhoujp

一.配置主机和节点机器之间SSH无密登录 多台外网服务器配置时,需要在/etc/hosts中把本机的IP地址设置为内网IP地址 http://2d67df38.wiz02.com/share/s/0JpZYU2X7kl626aE730Q_Flq3daBjm08rAnl2oLpli2uMOfo 二.为每台主机关闭iptables,SELinux,启动ntpd服务 三.安装Mysql,JDK 查看linux发行版本 cat /etc/redhat-release 从网站http://dev.mysq