大家需要在每个节点上提前装好"expect"工具
expect的使用请看我的另一篇文章：
http://tianxingzhe.blog.51cto.com/3390077/1687661
spawn命令激活一个Unix程序来进行交互式的运行。
send命令向进程发送字符串。
expect命令等待进程的某些字符串
set timeout 1 设置超时时间 timeout -1 为永不超时
expect eof
只有spawn执行的命令结果才会被expect捕捉到，因为spawn会启动一个进程，只有这个进程的相关信息才会被捕捉到，主要包括：标准输入的提示信息，eof和timeout。
这里，eof是必须去匹配的，在spawn进程结束后会向expect发送eof；如果不去匹配，有时也能运行，比如sleep多少秒后再去spawn下一个命令，但是不要依赖这种行为，很有可能今天还可以，明天就不能用了。
expect \"#\" 期待返回shell提示符(是#或者$)
interact 命令
执行完成后保持交互状态，把控制权交给控制台，这个时候就可以手工操作了。如果没有这一句登录完成后会退出，而不是留在远程终端上。如果你只是登录过去执行一段命令就退出，可改为 expect eof
id_dsa/ id_dsa.pub：你用openssh工具生成的私钥公钥对
authorized_keys ：你使用ssh连接的linux服务器需要认证你的身份，所以你需要在连接的linux服务器上安装自已的公钥，authorized_keys这里面就是存放你自己的id_dsa.pub的内容
scp是有Security的文件copy，基于ssh登录。操作起来比较方便，比如要把当前一个文件copy到远程另外一台主机上，可以如下命令。
scp /home/daisy/full.tar.gz
大体思路
1、首先在一个文本文件中保存1000台机器的hadoop用户名和密码
2、用shell遍历这个文件 写一个循环用namenode的去循环登陆其他的999个节点，执行生成密钥的工作，然后把生成的公钥写回namenode
3、在namenode上生成密钥 写入这个文件
4、把第三部生成的文件拷贝到剩下的机器上
5、用循环遍历验证免密的效果
本解决方法主要包括两个脚本： sshpass.sh和ssh4slaves
1. sshpass.sh
#!/bin/bash
# Name : sshpass.sh
# Time : 17/09/2012
# Author : [email protected]
# Purpose : For fast and easy setup of the SSH Passwordless access among all the nodes
# in a cluster.
# User : Any user you are performing the test! Better to settup a separate user from your
# working env to avoid troubles!!! "root" is used in this example, and you can change it
# via the export virable "USER=root"
# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!
# And this likely makes sense as no body want to put more trouble on this.
# Usage : 1st, make sure the script has the execute permisison "chmod +x ssh_pass.sh"
# ./ssh_pass.sh password
# : 2nd, ensure the "ssh4slaves.sh" script is with ssh_pass.sh for all nodes setup!!!
# : 3rd, "expect" has to be installed on all the nodes for the SSH config
export FILELOC="/root"
export SLAVESFILE="$FILELOC/sshslaves"
export HOSTS=`cat $FILELOC/sshhosts`
export SLAVES=`cat $FILELOC/sshslaves`
export SSH4SLAVESCRIPT="$FILELOC/ssh4slaves.sh"
export MASTER=hdp01
export USER=root
export PASSWD=$1
export SSHLOC="$FILELOC/.ssh/"
export RSAFILE="$FILELOC/.ssh/id_rsa"
export RSAPUBFILE="$FILELOC/.ssh/id_rsa.pub"
export AUTHFILE="$FILELOC/.ssh/authorized_keys"
export EXPECTCHK=`rpm -qa expect | wc -l`
#
if [ $EXPECTCHK != 1 ]
then
echo ‘‘
echo "########################################################################################"
echo "Please install the \"expect\" package first on all nodes to allow the script to run!!!"
echo "yum -y install expect"
echo "########################################################################################"
else
if [ -e $RSAFILE ]
then
echo "########################################################################################"
echo "Attention: This is for TEST ONLY, please fully test it before applying it to PROD"
echo "environment!!! OR you might get in trouble!!!"
echo ‘‘
echo "BETTER TO HAVE A NEW USER FOR THE TEST TO AVOID DESTROYING YOUR ENVIRONMENT!"
echo ‘‘
echo "Please manually delete the ssh related file on each host before executing the script!!!"
echo ‘‘
for host in $HOSTS
do
echo "Please run command on $host: rm -rf $SSHLOC"
done
echo "########################################################################################"
else
# Just generate
for host in $HOSTS
do
if [ $host = "$MASTER" ]
then
echo ‘‘
echo "###########################################################"
echo "Generating RSA keys for MASTER host $MASTER"
echo "###########################################################"
echo ‘‘
expect -c "
set timeout 1
spawn ssh [email protected]$host
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect \"#\"
send \"ssh-keygen -t rsa -P ‘‘ -f $RSAFILE\r\"
expect \"#\"
send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect eof
"
else
echo ‘‘
echo "###########################################################"
echo "Generating RSA keys for all OTHER hosts..."
echo "hostname is $host"
echo "###########################################################"
echo ‘‘
expect -c "
set timeout 1
spawn ssh [email protected]$host
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect \"#\"
send \"ssh-keygen -t rsa -P ‘‘ -f $RSAFILE\r\"
expect \"#\"
send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"
expect \"yes/no\"
send -- \"yes\r\"
expect \"password:\"
send -- \"$PASSWD\r\"
expect eof
"
fi
done
###
for host in $SLAVES
do
echo ‘‘
echo "############################################################################"
echo "Copying authorized_keys to host $host from the MASTER host $MASTER..."
echo "############################################################################"
echo ‘‘
expect -c "
set timeout 1
spawn scp $AUTHFILE "[email protected]$host:$SSHLOC"
expect \"password:\"
send -- $PASSWD\r
expect eof
"
done
#
for host in $SLAVES
do
echo ‘‘
echo "############################################################################"
echo "Distributing the $SLAVESFILE file to slave host $host..."
echo "############################################################################"
echo ‘‘
scp $SLAVESFILE "$host:$FILELOC"
echo ‘‘
echo "############################################################################"
echo "Distributing the $SSH4SLAVESCRIPT script to slave host $host..."
echo "############################################################################"
echo ‘‘
scp $SSH4SLAVESCRIPT "$host:$FILELOC"
done
for host in $SLAVES
do
echo ‘‘
echo "############################################################################"
echo "Working on the slaves node $host to ensure no prompt for the "yes/no" question..."
echo "############################################################################"
echo ‘‘
ssh -q [email protected]$host $SSH4SLAVESCRIPT
done
### Check whether the Passwordless ssh works ###
for host in $HOSTS
do
echo ‘‘
echo "############################################################################"
echo "Check whether the Passwordless SSH works for $host..."
echo "############################################################################"
echo ‘‘
ssh $host uname -a && date
done
fi
fi
###
# rm -rf /root/.ssh
# mv /root/.ssh /root/sshlogin
#{ eval "$GET_ID" ; } | ssh $1 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#cat /root/.ssh/id_rsa.pub | ssh hdp01 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#/root/.ssh/id_rsa.pub
./ssh_pass.sh password （password替换程序里的$1参数），本例中密码为stonetest
2. ssh4slaves
#!/bin/bash # Name : ssh4slaves.sh # Time : 17/09/2012 # Author : [email protected] # Purpose : For fast and easy setup of the SSH Passwordless access among all the slave nodes # in a cluster. Mainly to ensure no prompt for "yes/no" again!!! # User : Any user you are performing the test! Better to settup a separate user from your # working env to avoid troubles!!! "root" is used in this example, and you can change it # via the export virable "USER=root" # Attention: The test env is assuming that each $USER on each $HOST is usring the same password! # And this likely makes sense as no body want to put more trouble on this. # Usage : This script is called by the main script "ssh_pass.sh" # 1st, make sure the script has the execute permisison "chmod +x ssh4slaves.sh" before # distributing it to other slaves node. # 2nd, Remember to change variable "PASSWORD" before start the main script "sshpass.sh" export FILELOC="/root" export SLAVES=`cat $FILELOC/sshslaves` export USER=root export PASSWD=stonetest for host in $SLAVES do echo ‘‘ echo "Ensure ssh passwordless works among all slave nodes..." echo ‘‘ expect -c " set timeout 1 spawn ssh [email protected]$host expect \"yes/no\" send -- \"yes\r\" expect eof " done
3. 其他配置
[[email protected] ~]# pwd /root [[email protected] ~]# cat sshhosts hdp01 hdp02 hdp03 [[email protected] ~]# cat sshslaves hdp02 hdp03 [[email protected] ~]# ls -lrth | tail -2 -rwxr-xr-x 1 root root 1.3K Sep 18 02:08 ssh4slaves.sh -rwxr-xr-x 1 root root 6.5K Sep 18 02:11 ssh_pass.sh
4. 测试输出
[[email protected] ~]# ./ssh_pass.sh stonetest ########################################################### Generating RSA keys for MASTER host hdp01 ########################################################### spawn ssh [email protected] The authenticity of host ‘hdp01 (192.168.1.121)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp01,192.168.1.121‘ (RSA) to the list of known hosts. [email protected]‘s password: Last login: Tue Sep 18 02:09:29 2012 from hdp02.dbinterest.local [[email protected] ~]# ssh-keygen -t rsa -P ‘‘ -f /root/.ssh/id_rsa Generating public/private rsa key pair. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 3a:c3:98:b3:e4:39:fa:fe:87:c6:22:90:16:57:4e:47 [email protected] The key‘s randomart image is: +--[ RSA 2048]----+ | .E | | o . | | + . | | . . . | | .o S | |o. + . | |.. =.=. | | .oo++o. | | .=*=.. | +-----------------+ [[email protected] ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 [email protected]‘s password: Now try logging into the machine, with "ssh ‘hdp01‘", and check in: .ssh/authorized_keys to make sure we haven‘t added extra keys that you weren‘t expecting. [[email protected] ~]# ########################################################### Generating RSA keys for all OTHER hosts... hostname is hdp02 ########################################################### spawn ssh [email protected] The authenticity of host ‘hdp02 (192.168.1.122)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp02,192.168.1.122‘ (RSA) to the list of known hosts. [email protected]‘s password: Last login: Tue Sep 18 02:09:23 2012 from hdp02.dbinterest.local [[email protected] ~]# ssh-keygen -t rsa -P ‘‘ -f /root/.ssh/id_rsa Generating public/private rsa key pair. Created directory ‘/root/.ssh‘. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: a9:89:fe:40:8a:8e:21:55:da:3b:6b:68:4f:3e:8f:fc [email protected] The key‘s randomart image is: +--[ RSA 2048]----+ | | | | | . | | + . | | o o S | | o o o o | |+ ..* o | |+.o=o= | |.o oB=E | +-----------------+ [[email protected] ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 The authenticity of host ‘hdp01 (192.168.1.121)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp01,192.168.1.121‘ (RSA) to the list of known hosts. [email protected]‘s password: Now try logging into the machine, with "ssh ‘hdp01‘", and check in: .ssh/authorized_keys to make sure we haven‘t added extra keys that you weren‘t expecting. ########################################################### Generating RSA keys for all OTHER hosts... hostname is hdp03 ########################################################### spawn ssh [email protected] The authenticity of host ‘hdp03 (192.168.1.123)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp03,192.168.1.123‘ (RSA) to the list of known hosts. [email protected]‘s password: Last login: Tue Sep 18 02:09:19 2012 from hdp02.dbinterest.local [[email protected] ~]# ssh-keygen -t rsa -P ‘‘ -f /root/.ssh/id_rsa Generating public/private rsa key pair. Created directory ‘/root/.ssh‘. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: a4:3d:dd:54:42:c0:45:ec:ed:ae:d6:bd:14:a0:9b:16 [email protected] The key‘s randomart image is: +--[ RSA 2048]----+ | ..*= . | | . .o | | . ..o | | + . oo o | | . S .E.. . | | . + . .| | + o o | | . . + .| | ... ..| +-----------------+ [[email protected] ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01 The authenticity of host ‘hdp01 (192.168.1.121)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp01,192.168.1.121‘ (RSA) to the list of known hosts. [email protected]‘s password: Now try logging into the machine, with "ssh ‘hdp01‘", and check in: .ssh/authorized_keys to make sure we haven‘t added extra keys that you weren‘t expecting. [[email protected] ~]# ############################################################################ Copying authorized_keys to host hdp02 from the MASTER host hdp01... ############################################################################ spawn scp /root/.ssh/authorized_keys [email protected]:/root/.ssh/ [email protected]‘s password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################ Copying authorized_keys to host hdp03 from the MASTER host hdp01... ############################################################################ spawn scp /root/.ssh/authorized_keys [email protected]:/root/.ssh/ [email protected]‘s password: authorized_keys 100% 1227 1.2KB/s 00:00 ############################################################################ Distributing the /root/sshslaves file to slave host hdp02... ############################################################################ sshslaves 100% 12 0.0KB/s 00:00 ############################################################################ Distributing the /root/ssh4slaves.sh script to slave host hdp02... ############################################################################ ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################ Distributing the /root/sshslaves file to slave host hdp03... ############################################################################ sshslaves 100% 12 0.0KB/s 00:00 ############################################################################ Distributing the /root/ssh4slaves.sh script to slave host hdp03... ############################################################################ ssh4slaves.sh 100% 1277 1.3KB/s 00:00 ############################################################################ Working on the slaves node hdp02 to ensure no prompt for the yes/no question... ############################################################################ Ensure ssh passwordless works among all slave nodes... spawn ssh [email protected] The authenticity of host ‘hdp02 (192.168.1.122)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp02,192.168.1.122‘ (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:54 2012 from hdp01.dbinterest.local [[email protected] ~]# Ensure ssh passwordless works among all slave nodes... spawn ssh [email protected] The authenticity of host ‘hdp03 (192.168.1.123)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp03,192.168.1.123‘ (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:55 2012 from hdp01.dbinterest.local [[email protected] ~]# ############################################################################ Working on the slaves node hdp03 to ensure no prompt for the yes/no question... ############################################################################ Ensure ssh passwordless works among all slave nodes... spawn ssh [email protected] The authenticity of host ‘hdp02 (192.168.1.122)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp02,192.168.1.122‘ (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:58 2012 from hdp02.dbinterest.local [[email protected] ~]# Ensure ssh passwordless works among all slave nodes... spawn ssh [email protected] The authenticity of host ‘hdp03 (192.168.1.123)‘ can‘t be established. RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ‘hdp03,192.168.1.123‘ (RSA) to the list of known hosts. Last login: Tue Sep 18 02:11:59 2012 from hdp02.dbinterest.local ############################################################################ Check whether the Passwordless SSH works for hdp01... ############################################################################ Linux hdp01.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:05 PDT 2012 ############################################################################ Check whether the Passwordless SSH works for hdp02... ############################################################################ Linux hdp02.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:05 PDT 2012 ############################################################################ Check whether the Passwordless SSH works for hdp03... ############################################################################ Linux hdp03.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Tue Sep 18 02:12:06 PDT 2012
5. 其他节点测试
[[email protected] ~]# [[email protected] ~]# ssh hdp02 Last login: Tue Sep 18 02:12:00 2012 from hdp03.dbinterest.local [[email protected] ~]# exit logout Connection to hdp02 closed. [[email protected] ~]# ssh hdp03 Last login: Tue Sep 18 02:12:02 2012 from hdp03.dbinterest.local [[email protected] ~]# exit logout Connection to hdp03 closed. [[email protected] ~]# ---------- [[email protected] ~]# [[email protected] ~]# ssh hdp01 Last login: Tue Sep 18 02:12:22 2012 from hdp02.dbinterest.local [[email protected] ~]# exit logout Connection to hdp01 closed. [[email protected] ~]# ssh hdp02 Last login: Tue Sep 18 02:12:25 2012 from hdp02.dbinterest.local [[email protected] ~]# exit logout Connection to hdp02 closed. [[email protected]03 ~]# ssh hdp03 Last login: Tue Sep 18 02:12:30 2012 from hdp02.dbinterest.local [[email protected] ~]# exit logout Connection to hdp03 closed. [[email protected] ~]#
代码下载见附件
参考文章：
http://www.cnblogs.com/iloveyoucc/archive/2012/05/11/2496433.html
http://f.dataguru.cn/thread-19920-1-1.html
时间： 2024-10-12 23:28:43