介绍
如何用ansible的playbook实现自动decommission hadoop 的node ?
正文
- 对于如何部署安装ansible ，请看http://haohaozhang.blog.51cto.com/9176600/1610499
- hadoop部署，请看http://haohaozhang.blog.51cto.com/9176600/1607089
- 使用hadoop用户创建目录ansible_decom,里面的目录结构如下：
ansible_decom/ ansible_decom/roles ansible_decom/roles/nn ansible_decom/roles/nn/tasks ansible_decom/roles/nn/tasks/main.yml ansible_decom/roles/decom ansible_decom/roles/decom/tasks ansible_decom/roles/decom/tasks/main.yml ansible_decom/roles/rm ansible_decom/roles/rm/tasks ansible_decom/roles/rm/tasks/main.yml ansible_decom/roles/get-status ansible_decom/roles/get-status/tasks ansible_decom/roles/get-status/tasks/main.yml ansible_decom/group_vars ansible_decom/group_vars/all ansible_decom/exclude-iplist ansible_decom/site.yml ansible_decom/hosts
以yml结尾的是playbook文件，其中site.yml是主文件，他会调用roles下面的子yml文件。
hosts文件定义的是主机列表。
[rmservers] 10.9.214.151 10.9.214.15 10.9.214.105 10.9.214.113 [nnservers] 10.9.214.151 10.9.214.15 10.9.214.105 10.9.214.113 [client] 10.9.214.149 [decomnodes] 10.9.214.149 10.9.214.160
exclude-iplist定义的是要decommission的主机列表。
10.9.214.160 10.9.214.149
site.yml
--- # this playbook displays decommission nodes - name: add node to mapred-exclude and refreshnode hosts: rmservers remote_user: hadoop roles: - rm - name: add node to hdfs-exclude and refreshnodes hosts: nnservers remote_user: hadoop roles: - nn - name: stop nodemanager and datanode service hosts: decomnodes remote_user: hadoop roles: - decom - name: get status for hosts hosts: client roles: - get-status
roles/decom/tasks/main.yml
--- # stop services - name: stop nodemanager command: /opt/hadoop-2.6.0/sbin/yarn-daemon.sh stop nodemanager - name: stop datanode command: /opt/hadoop-2.6.0/sbin/hadoop-daemon.sh stop datanode
roles/nn/tasks/main.yml
--- # step1. add decom nodes to hdfs-excluse file # step2. refresh nodes - name: add decom nodes to hdfs-exclude file copy: src=exclude-iplist dest=/opt/hadoop-2.6.0/etc/hadoop/ - shell: cd /opt/hadoop-2.6.0/etc/hadoop/ ; cat exclude-iplist >> hdfs-exclude ; rm exclude-iplist - name: refresh nodes shell: /opt/hadoop-2.6.0/bin/hdfs dfsadmin -refreshNodes
roles/rm/tasks/main.yml
--- # step1. add decom nodes to mapred-excluse file # step2. refresh nodes - name: add decom nodes to mapred-exclude file copy: src=exclude-iplist dest=/opt/hadoop-2.6.0/etc/hadoop/ - shell: cd /opt/hadoop-2.6.0/etc/hadoop/ ; cat exclude-iplist >> mapred-exclude ; rm exclude-iplist - name: refresh nodes shell: /opt/hadoop-2.6.0/bin/yarn rmadmin -refreshNodes
4. 执行ansible的playbook
asible-playbook -i hosts site.yml
执行若有失败会报错的。
5. 完成之后查看http://10.9.214.151:50070看Live Node和Dead Node的数目和个数是不是变了。
时间： 2024-11-25 05:40:16