本篇介绍cdh安装之前需要的一些必要配置，当然这些配置也可以用shell脚本来配置。在安装之前请先配置好yum源，在文中用的统一都是阿里源。
系统：centos7
主机3台：
master.com.cn 192.168.1.110
salver1.com.cn 192.168.1.111
salver2.com.cn 192.168.1.112
yum服务器：yum.com.cn 192.168.1.100。这里如果不知道怎么配置的朋友，可以参考第一篇：https://www.cnblogs.com/yaowentao/p/12296329.html
基本配置
1. 主机名在系统安装的配置完毕，如果没有配置，可以进系统在修改
1 $ vi /etc/hostname
2. 配置host文件
1 $ vi /etc/hosts 2 192.168.1.100 yum.com.cn 3 192.168.1.110 master.com.cn 4 192.168.1.111 salver1.com.cn 5 192.168.1.112 salver2.com.cn
3. 关闭防火墙，selinux。配置完后重启
1 $ sudo systemctl disable firewalld #停止防火墙 2 $ sudo systemctl stop firewalld #禁止开机启动防火墙 3 4 $ vi /etc/selinux/config 5 SELINUX=disable #关闭selinux
4. 配置ntp服务与ssh免密码登陆配置，请参考我的另外2篇文档
5. 如果有本地的yum源，就配置。如果没有就使用公网的yum源。
5.1 配置yum 本地库,也可以下载官网的repo文件，一个是从网上下载，一个是从本地yum库下载
1 [cloudera-repo] 2 name=cloudera-repo 3 baseurl=http://192.168.10.225/cloudera-repos/cm6/6.2.1/redhat7/yum/ 4 enabled=1 5 gpgcheck=0
5.2 网上yum源，直接下载即可
1 $ sudo wget https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/
6. 安装cdh专用jdk（需要在所有主机上都要安装）
1 $ sudo yum -y install oracle-j2sdk1.8
7. 配置本地的mysql数据库，便于存放cdh服务器的一些配置，以及hadoop组件中需要的一些配置，如hive的源数据库。这里一般推荐用mysql，当然oracle等其他的数据库也可以。
- 直接用yum源安装
-
1 $ wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm 2 $ sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm 3 $ sudo yum update 4 $ sudo yum install mysql-server 5 $ sudo systemctl start mysqld
- 配置mysql数据，并进行初始化，设置root密码
-
$vi /etc/my.cnf [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock transaction-isolation = READ-COMMITTED # Disabling symbolic-links is recommended to prevent assorted security risks; # to do so, uncomment this line: symbolic-links = 0 key_buffer_size = 32M max_allowed_packet = 32M thread_stack = 256K thread_cache_size = 64 query_cache_limit = 8M query_cache_size = 64M query_cache_type = 1 max_connections = 550 #expire_logs_days = 10 #max_binlog_size = 100M #log_bin should be on a disk with enough free space. #Replace ‘/var/lib/mysql/mysql_binary_log‘ with an appropriate path for your #system and chown the specified folder to the mysql user. log_bin=/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set #a server_id, MySQL will not start. The server_id must be unique within #the replicating group. server_id=1 binlog_format = mixed read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M # InnoDB settings innodb_file_per_table = 1 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 64M innodb_buffer_pool_size = 4G innodb_thread_concurrency = 8 innodb_flush_method = O_DIRECT innodb_log_file_size = 512M [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid sql_mode=STRICT_ALL_TABLES
-
配置开机自启动
-
1 $ sudo systemctl enable mysqld 2 $ sudo systemctl start mysqld
- 初始化mysql，设置root密码
-
1 $ sudo /usr/bin/mysql_secure_installation 2 [...] 3 Enter current password for root (enter for none): 4 OK, successfully used password, moving on... 5 [...] 6 Set root password? [Y/n] Y 7 New password: 8 Re-enter new password: 9 Remove anonymous users? [Y/n] Y 10 [...] 11 Disallow root login remotely? [Y/n] N 12 [...] 13 Remove test database and access to it [Y/n] Y 14 [...] 15 Reload privilege tables now? [Y/n] Y 16 All done!
- 安装mysql的jdbc
-
1 $ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz 2 $ tar zxvf mysql-connector-java-5.1.46.tar.gz 3 4 $ sudo mkdir -p /usr/share/java/ 5 $ cd mysql-connector-java-5.1.46 6 $ sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
- 给root授远程登陆权限
-
1 mysql> GRANT ALL PRIVILEGES ON *.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘ WITH GRANT OPTION; 2 mysql> FLUSH PRIVILEGES;
- 创建cm服务，hadoop组件需要的库。用户统一用root。
-
1 CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 2 GRANT ALL ON scm.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 3 CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 4 GRANT ALL ON amon.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 5 CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 6 GRANT ALL ON rman.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 7 CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 8 GRANT ALL ON hue.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 9 CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 10 GRANT ALL ON metastore.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 11 CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 12 GRANT ALL ON sentry.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 13 CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 14 GRANT ALL ON nav.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 15 CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 16 GRANT ALL ON navms.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘; 17 CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; 18 GRANT ALL ON oozie.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘123456‘;
- 安装cm服务器文件
-
1 master: 2 $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server 3
4 salver: 5 $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent
- scm初始化
-
1 $ sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root
-
- 完成以上配置，就能正常登陆cm服务器的前台了。下面进入服务器的配置。请看第三篇。
- 启动cm服务
1 $ sudo systemctl start cloudera-scm-server 2 $ sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
- 启动cm服务
启动成功：
各位小伙伴，对以上如有问题，请在下方留言沟通。以上都是我亲自测试通过安装成功的。
转发，请注明出处！
原文地址：https://www.cnblogs.com/yaowentao/p/12297725.html
时间： 2024-11-10 10:31:59