通过以上的学习,我们已经对git非常熟悉了,可以设计一个自动化部署脚本:
q??约定:
1.已经有一个可以上线的代码在git仓库。
2.我们现在要做10个集群节点的一键部署,秒级回滚。
3.所有的web服务,都应该使用普通用户。(强烈建议)
4.所有的web服务都不应该监听80端口,除了负载均衡。
q??自动化部署思路大纲:
1.获取最新代码
2.编译(可选)
3.配置文件(软连接或者拷贝)。
4.打包(tar,加速传输)
5.文件分发(Scp Rsync Salt)(不需要密码验证)
6.将目标服务器移除集群(注释配置文件)
7.解压
8.防止webroot站点目录
9.scp差异文件(可能有一个节点配置文件不一样)
10.重启Web服务
11.测试
12.正常回退实践
13.紧急回退实践
q??1.环境说明
我这里使用1台负载均衡两台web来演示(salstack批量管理会有相关章节):
主机名 |
IP地址 |
描述 |
部署服务 |
git-node |
192.168.56.11 |
部署机兼git仓库 |
Gitlab |
lb-node1 |
192.168.56.100 |
负载均衡服务 |
Nginx |
pre-node1 |
192.168.56.12 |
Web测试节点 |
Nginx+PHP |
Web-node1 |
192.168.56.13 |
Web节点node1 |
Nginx+PHP |
Web-node2 |
192.168.56.14 |
Web节点node2 |
Nginx+PHP |
q??2.创建相关用户(部署机以及web节点都需要操作)
useradd www &&??echo "123"|passwd --stdin www
q??3.部署机能登陆到web节点任意一台机器(部署机操作)
[[email protected] ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/www/.ssh/id_rsa):
Created directory ‘/home/www/.ssh‘.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/www/.ssh/id_rsa.
Your public key has been saved in /home/www/.ssh/id_rsa.pub.
The key fingerprint is:
fc:75:01:96:47:27:d5:d4:e3:16:e7:15:3a:b1:a4:87 [email protected]
The key‘s randomart image is:
+--[ RSA 2048]----+
|????????????o=o+B|
|???????????.=.===|
|???????????E *o.=|
|???????.. .+.|
|????????S. o??|
|?????????. . .|
|??????????.|
|?????????????????|
|?????????????????|
+-----------------+
//分发各个web节点,会提示输入密码
[[email protected] ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
[[email protected] ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
[[email protected] ~]# ssh 192.168.56.13??//测试一台
Last login: Mon Nov??7 03:45:08 2016 from 192.168.56.11
[[email protected] ~]$
q??4.gitlab服务部署(部署机操作)
安装配置依赖项
?[[email protected] ~]# yum install curl openssh-server postfix
[[email protected] ~]# systemctl enable sshd postfix
[[email protected] ~]# systemctl start sshd postfix
[[email protected] ~]# firewall-cmd --permanent --add-service=http
[[email protected] ~]# systemctl reload firewalld
添加GitLab仓库,并安装到服务器上
[[email protected] ~]# curl -sS http://packages.gitlab.cc/install/gitlab-ce/script.rpm.sh | sudo bash
[[email protected] ~]# yum install gitlab-ce
启动GitLab
gitlab-ctl reconfigure
浏览到主机名和登录Browse to the hostname and login
首次访问GitLab,系统会让你重新设置管理员的密码,设置成功后会返回登录界面.
默认的管理员账号是root,如果你想更改默认管理员账号,请输入上面设置的新密码登录系统后修改帐号名.
q??5.创建项目相关目录(部署机操作)
项目代码仓库
mkdir -p /deploy/code/rainbow_pro
项目源代码仓库
mkdir -p /deploy/source/rainbow_pro
存放配置文件
mkdir -p /deploy/config/rainbow_pro/config
存放管理配置文件
mkdir -p /deploy/config/rainbow_pro/admin_config
mkdir -p /deploy/tmp
mkdir -p /deploy/logs
授权www用户
chown -R www:www /deploy
chown -R www:www /home/www/
查看目录结构
tree /deploy/
/deploy/
├──?code
│???└──?rainbow_pro
├──?config
│???└──?rainbow_pro
│???????├──?admin_config
│???????└──?config
├──?logs
├──?source
│???└──?rainbow_pro
└──?tmp
q??6.web节点需要创建的项目目录(rainbow_pro是项目名称,如果修改了就需要修改脚本)
mkdir –p /deploy/tmp
mkdir –p /deploy/code/rainbow_pro
schown -R www.www /deploy/
q??7.负载均衡服务器配置
安装nginx
[[email protected] ~]# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
[[email protected] ~]# yum install -y nginx
配置nginx
[[email protected] ~]# egrep -v ‘#|^$‘ /etc/nginx/nginx.conf.default??> /etc/nginx/nginx.conf
[[email protected] ~]# cat /etc/nginx/nginx.conf
worker_processes??1;
events {
????worker_connections??1024;
}
http {
????include???????mime.types;
????default_type??application/octet-stream;
????sendfile????????on;
????keepalive_timeout??65;
????upstream web_server {
????server 192.168.56.13:8888;
????server 192.168.56.14:8888;
????}
????server {
????????listen???????80;
????????server_name??linux.git.com;
????????location / {
????????????root???html;
????????????index??index.html index.htm;
????proxy_pass http://web_server;
????????}
????????error_page???500 502 503 504??/50x.html;
????????location = /50x.html {
????????????root???html;
????????}
????}
}
启动负载均衡Nginx服务
[[email protected] ~]# systemctl start nginx
q??8.sudo授权(所有web操作)
sed -i ‘98a www?????ALL=(ALL)???????NOPASSWD:ALL‘ /etc/sudoers
sed -i ‘[email protected]@\#[email protected]‘ /etc/sudoers
q??9.配置nginx(所有web操作)
安装nginx、php、php-fpm
[[email protected] ~]#wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
[[email protected] ~]# yum install -y nginx php php-fpm
配置nginx
[[email protected] ~]# egrep -v ‘#|^$‘ /etc/nginx/nginx.conf.default??> /etc/nginx/nginx.conf
[[email protected] ~]# cat /etc/nginx/nginx.conf
user www www;
worker_processes??1;
events {
????worker_connections??1024;
}
http {
????include???????mime.types;
????default_type??application/octet-stream;
????sendfile????????on;
????keepalive_timeout??65;
????server {
????????listen???????8888;
????????server_name??192.168.56.13;??#不同节点更改为不同节点IP
????????location / {
????????????root???/home/www/rainbow_pro;
????????????index??index.html index.htm;
????????}
????????error_page???500 502 503 504??/50x.html;
?location = /50x.html {
????????????root???html;
????????}
????}
}
启动所有web节点Nginx+PHP
[[email protected] ~]# systemctl start nginx
[[email protected] ~]# systemctl start php-fpm
q??10.部署机切换www用户,添加远程项目,执行如下脚本(注意:如无法pull,请添加www用户key至gitlab)
[[email protected] ~]# echo "192.168.56.11 git-node1" >> /etc/hosts
[[email protected] ~]# su – www
?
//首次使用需要执行如下步骤
[[email protected] ~]$ cd /deploy/source/rainbow_pro/
[[email protected] rainbow_pro]$ git init
初始化空的?Git?版本库于?/deploy/source/rainbow_pro/.git/
[[email protected] rainbow_pro]$ git remote add origin [email protected]:root/git_demo.git
[[email protected] rainbow_pro]$ git pull origin master
?
?
[[email protected] ~]$ cat deploy.sh
#!/bin/bash
?
#!/bin/bash
?
#?添加www用户,并且做好相关机器的sshkey认证(部署机能登陆到web节点任意一台机器)
# Nginx?权限必须让www用户可访问
?
#项目代码仓库
# mkdir -p /deploy/code/rainbow_pro
?
#项目源代码仓库
# mkdir -p /deploy/source/rainbow_pro
?
#存放配置文件
# mkdir -p /deploy/config/rainbow_pro/config
?
#存放管理配置文件
# mkdir -p /deploy/config/rainbow_pro/admin_config?
?
# mkdir -p /deploy/tmp
?
# mkdir -p /deploy/logs
?
# web站点存放目录
# mkdir -p /home/www/
?
#授权www用户
# chown -R www:www /deploy
# chown -R www:www /home/www/
?
?
MSG(){
??if [ $? -eq 0 ];then
????echo "$1 OK"
????else
????echo "$1 FAIL"
????shell_unlock;
????exit 1
??fi
}
?
?
#?定义代码变量
PRO_NAME="rainbow_pro"
CODE_DIR="/deploy/code/$PRO_NAME"
SOURCE_DIR="/deploy/source/$PRO_NAME"
CONFIG_DIR="/deploy/config/$PRO_NAME"
WEB_DIR="/home/www"
TMP_DIR="/deploy/tmp"
?
?
# web节点列表信息
PRE_LIST="192.168.56.12"
GROUP1_LIST="192.168.56.13 192.168.56.14"
ROLLBACK_LIST="192.168.90.13 192.168.56.14"
?
?
# Date/Time Veriables
CTIME=$(date "+%F-%H-%M")
?
# Shell Env
SHELL_NAME="deploy.sh"
SHELL_DIR="/home/www"
?
?
#日志定义
LOG_FILE="${SHELL_DIR}/${SHELL_NAME}".log
LOCK_FILE="/tmp/deploy.lock"
?
#?脚本锁
shell_lock(){
????touch ${LOCK_FILE}
}
?
shell_unlock(){
????rm -f ${LOCK_FILE}
}
?
?
#?测试URL
url_test(){
????URL=$1
????curl -s --head $URL |egrep ‘200|301|302‘
}
?
#?日志记录
writelog(){
????LOGINFO=$1
????[ -f ${LOG_FILE} ] || touch ${LOG_FILE}
????echo "${CTIME}: ${SHELL_NAME} : ${LOGINFO}" >> ${LOG_FILE}
}
?
?
#?获取代码
code_get(){
????writelog "code_get"
????cd $SOURCE_DIR && git pull origin $DEPLOY_METHOD
????GIT_CID=$(git log|awk ‘NR==1{print $2}‘|cut -c 1-6)
????PKG_VER="${CTIME}_${GIT_CID}"
????PKG_NAME="${PRO_NAME}_${PKG_VER}"
????cp -r ${SOURCE_DIR} ${TMP_DIR}/${PKG_NAME}
}
?
#代码编译过程(php没有编译过程)
code_bulid(){
????echo code_bulid
}
?
#代码配置文件
code_config(){
????writelog "code_config"
????/bin/cp -r ${CONFIG_DIR}/config.php ${TMP_DIR}/${PKG_NAME}/config.php
}
?
#打包代码并去除.git目录
code_tar(){
????writelog "code_tar"
????cd ${TMP_DIR} && tar czf ${PKG_NAME}_tar.gz ${PKG_NAME} --exclude=.git --exclude=.gitignore
????writelog "${PKG_NAME}_tar.gz"
}
?
?
#代码发送至各个节点
code_scp(){
????writelog "code_scp"
????for node in $PRE_LIST;do
????????????scp ${TMP_DIR}/${PKG_NAME}_tar.gz $node:${TMP_DIR}/
????done
?
????for node in $GROUP1_LIST;do
????????????scp ${TMP_DIR}/${PKG_NAME}_tar.gz $node:${TMP_DIR}/
????done
?
}
?
#部署预生产环境代码
pre_deploy(){
????writelog "Pre "$node" deploy code"
????for node in ${PRE_LIST};do
????????????ssh ${node} "cd ${TMP_DIR} && tar xf ${PKG_NAME}_tar.gz -C ${CODE_DIR}/"
????????????ssh ${node} "rm -f $WEB_DIR/${PRO_NAME} && ln -s ${CODE_DIR}/${PKG_NAME} $WEB_DIR/${PRO_NAME}"
???????????????????MSG "Pre "$node" deploy code"
????done
}
?
#测试预生产环境代码
pre_test(){
????for node in ${PRE_LIST};do
?????????????url_test "http://${node}:8888/index.html" >/dev/null
???????????????????MSG "Pre "$node" Test_URL"
????done
}
?
?
#分组部署代码
group1_deploy(){
????????writelog "Pro "$node" deploy code"
????for node in $GROUP1_LIST;do
?????????????ssh $node "cd ${TMP_DIR} && tar xf ${PKG_NAME}_tar.gz -C ${CODE_DIR}/"
?????????????ssh $node "rm -f $WEB_DIR/${PRO_NAME} && ln -s ${CODE_DIR}/${PKG_NAME} $WEB_DIR/${PRO_NAME}"
???????????????????MSG "Pro "$node" deploy code"
????done
}
?
?
#分组测试代码
group1_test(){
????for node in ${GROUP1_LIST};do
????????url_test "http://${node}:8888/index.html" >/dev/null
????????????MSG "Pro "$node" Test_URL"
????done
}
?
?
#重启php清理opcode缓存
code_reload(){
????for node in $GROUP1_LIST;do
????????????ssh $node "sudo??systemctl restart php-fpm"
???????????????????MSG "Pro "$node" php_fpm reload"
????done
}
?
?
#列出web节点最近2天部署最新代码
rollback_list(){
??for node in $GROUP1_LIST;do
????ssh $node /usr/sbin/ifconfig eth0|awk ‘NR==2 {print $2}‘|sed -r ‘s#(.*)# echo "\=\=\=\1\=\=\="#g‘|bash
????????ssh $node ls -l "$WEB_DIR"|grep "$PRO_NAME" &&\
????????ssh $node find "$CODE_DIR/" -maxdepth 1 -mtime -5|sed 1d|awk -F ‘/‘ ‘{print $5}‘
????????ssh $node find "$CODE_DIR" -type d -name "$PRO_NAME*" -mtime +30|xargs rm -fr
??done
}
?
?
rollback_fun(){
??if [ -z $ROOLBACK ];then
????shell_unlock;
????echo "Please input rollback version" && exit;
else
????for node in $ROLLBACK_LIST;do
????ssh $node rm -f $WEB_DIR/${PRO_NAME} && \
????ssh $node ln -s ${CODE_DIR}/$ROOLBACK $WEB_DIR/${PRO_NAME}
????done
fi
}
?
?
main(){
??if [ -f "$LOCK_FILE" ];then
????echo "Deploy is Running" && exit;
??fi
?
??DEPLOY_METHOD="$1"
??ROOLBACK="$2"
??case $DEPLOY_METHOD in
????deploy|master|dev)???#可写多个分支
????????????shell_lock;
????????????code_get;
????????????code_bulid;
????????????code_config;
????????????code_tar;
????????????code_scp;
????????????pre_deploy;
????????????pre_test;
????????????group1_deploy;
????????????group1_test;
????????????shell_unlock;
????????????code_reload;
????????????;;
????list)
????rollback_list;
????????????;;
????rollback)
????????????shell_lock;
????????????rollback_fun $ROLLBACK;
????????????shell_unlock;
????????????code_reload;
????????????;;
????*)
????????????echo "$Usage:$0 [ branch | list | rollback ]"
????esac
}
?
main $1 $2
部署master分支代码
[[email protected] ~]$ sh deploy.sh master
来自?git-node1:root/git_demo
?* branch????????????master?????-> FETCH_HEAD
Already up-to-date.
code_bulid
rainbow_pro_2016-11-07-03-27_194199_tar.gz????????????100%??2700.3KB/s???00:00???
rainbow_pro_2016-11-07-03-27_194199_tar.gz????????????100%270?????0.3KB/s???00:00
rainbow_pro_2016-11-07-03-27_194199_tar.gz????????????100%??2700.3KB/s???00:00???
Pre 192.168.56.12 deploy code OK
Pre 192.168.56.12 Test_URL OK
Pro 192.168.56.13 deploy code OK
Pro 192.168.56.14 deploy code OK
Pro 192.168.56.13 Test_URL OK
Pro 192.168.56.14 Test_URL OK
Pro 192.168.56.13 php_fpm reload OK
Pro 192.168.56.14 php_fpm reload OK
查看当前版本
[[email protected] ~]$ sh deploy.sh list
===192.168.56.13===
lrwxrwxrwx 1 www www???60 11月??7 03:40 rainbow_pro -> /deploy/code/rainbow_pro/rainbow_pro_2016-11-07-03-40_194199
rainbow_pro_2016-11-07-04-10_194199
===192.168.56.14===
lrwxrwxrwx 1 www www 60 11月??7 05:09 rainbow_pro -> /deploy/code/rainbow_pro/rainbow_pro_2016-11-07-03-40_194199
rainbow_pro_2016-11-07-04-10_194199
测试访问服务器
[[email protected] ~]# #echo "192.168.56.100 linux.git.com" >> /etc/hosts
[[email protected] ~]# curl http://linux.git.com
hello boss
boos??doubi
更新代码重新部署
[[email protected] demo]# echo "git + nginx + php" > index.html
[[email protected] demo]# git add index.html
[[email protected] demo]# git commit -m "new file"
[master 75c7d1f] new file
[[email protected] demo]# git push origin master
Counting objects: 10, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), 575 bytes | 0 bytes/s, done.
Total 6 (delta 0), reused 0 (delta 0)
To [email protected]:root/git_demo.git
???1941990..75c7d1f??master -> master
?
//重新部署
[[email protected] ~]$ sh deploy.sh master?
?
//重新测试
[[email protected] ~]# curl http://linux.git.com
git + nginx + php
执行回滚操作,回退上一个版本
[[email protected] ~]$ sh deploy.sh rollback??//不允许直接执行
Please input rollback version
?
//首先使用list查看想要回退的版本
[[email protected] ~]$ sh deploy.sh list
===192.168.56.11===
lrwxrwxrwx 1 www www???60 11月??7 04:17 rainbow_pro -> /deploy/code/rainbow_pro/rainbow_pro_2016-11-07-04-17_75c7d1
rainbow_pro_2016-11-07-04-10_194199
rainbow_pro_2016-11-07-04-17_75c7d1
===192.168.56.12===
lrwxrwxrwx 1 www www 60 11月??7 05:46 rainbow_pro -> /deploy/code/rainbow_pro/rainbow_pro_2016-11-07-04-17_75c7d1
rainbow_pro_2016-11-07-04-10_194199
rainbow_pro_2016-11-07-04-17_75c7d1
?
?
//回退至上一个版本
[[email protected] ~]$ sh deploy.sh rollback rainbow_pro_2016-11-07-04-10_194199
Pro 192.168.56.11 php_fpm reload OK
Pro 192.168.56.12 php_fpm reload OK
?
//再次查看,已经回退上一个版本
[[email protected] ~]# curl http://linux.git.com
hello boss
boos??doubi
查看代码部署日志
[[email protected] ~]$ cat deploy.sh.log
2016-11-07-04-10: deploy.sh : code_get
2016-11-07-04-10: deploy.sh : code_config
2016-11-07-04-10: deploy.sh : code_tar
2016-11-07-04-10: deploy.sh : rainbow_pro_2016-11-07-04-10_194199_tar.gz
2016-11-07-04-10: deploy.sh : code_scp
2016-11-07-04-10: deploy.sh : Pre 192.168.56.12 deploy code
2016-11-07-04-17: deploy.sh : Pro 192.168.56.13 deploy code
2016-11-07-04-10: deploy.sh : Pro 192.168.56.14 deploy code
2016-11-07-04-17: deploy.sh : code_get
2016-11-07-04-17: deploy.sh : code_config
2016-11-07-04-17: deploy.sh : code_tar
2016-11-07-04-17: deploy.sh : rainbow_pro_2016-11-07-04-17_75c7d1_tar.gz
2016-11-07-04-17: deploy.sh : code_scp
2016-11-07-04-17: deploy.sh : Pre 192.168.56.12 deploy code
2016-11-07-04-17: deploy.sh : Pro 192.168.56.13 deploy code
2016-11-07-04-17: deploy.sh : Pro 192.168.56.14 deploy code
徐亮伟, 江湖人称标杆徐。多年互联网运维工作经验,曾负责过大规模集群架构自动化运维管理工作。擅长Web集群架构与自动化运维,曾负责国内某大型电商运维工作。
个人博客"[徐亮伟架构师之路](http://www.xuliangwei.com)"累计受益数万人。
笔者Q:552408925、572891887?
架构师群:471443208