Rsync + sersync 实时同步备份
一 Rsync + Sersync 实时同步介绍
1.Rsync 服务搭建介绍
云机上搭建Rsync server，在本地搭建Rsync Clinet.
2. Sersync 服务搭建介绍
在本地服务器上搭建 Sersync Server 检测本地的ftp目录变化，实时同步到云机上
二、Rsync编译安装
1.检查本机是否存在
[[email protected]云机 ~]# rpm -qa rsync
rsync-3.0.6-12.el6.x86_64
2.卸载
[[email protected]云机 ~]# yum remove rsync
说明：需要安装最新的版本就需要卸载老版本。
3.下载rsync软件
[[email protected]云机 ~]# cd /usr/local/src/
[[email protected]云机 src]# wget https://download.samba.org/pub/rsync/rsync-3.1.2.tar.gz
4.解压包
[[email protected]云机 src]# tar zxvf rsync-3.1.2.tar.gz
5.创建程序安装目录，并编译安装
[[email protected]云机 src]# cd rsync-3.1.2
[[email protected]云机 rsync-3.1.2]# mkdir /usr/local/rsync
[[email protected]云机 rsync-3.1.2]# ./configure --prefix=/usr/local/rsync
[[email protected]云机 rsync-3.1.2]# make && make install
6.配置环境变量
[[email protected]云机 ~]# vim /etc/profile.d/rsync.sh
|
#!/bin/sh export PATH=$PATH:/usr/local/srync/bin |
7.建立软连接
[[email protected]云机 ~]# ln -s /usr/local/rsync/bin/rsync
/usr/bin/rsync
8.查看当前版本
[[email protected]云机 ~]# rsync --version
rsync version 3.1.2 protocol version 31
三、
Rsync server配置
1.编辑配置文件
[[email protected]云机 ~]# vim /etc/
rsyncd.conf
|
uid = rsync gid = rsync port = 873 use chroot = on max connections = 200 timeout = 120 pid file = /var/run/rsyncd.pid lock file = /var/run/rsyncd.lock log file = /var/log/rsyncd.log [bakup] |
2.创建密码文件
[[email protected]云机 ~]# vim /etc/rsyncd.password
|
rsync_bakup:4297f44b13955235245b2497399d7a93 |
说明：定义格式 用户:密码
3.修改密码文件权限
[[email protected]云机 ~]# chmod 600 /etc/rsyncd.password
4.创建Rsync server服务启动的系统账户
[[email protected]云机 ~]# useradd rsync -M -s /bin/nologin
5.创建数据存放目录
[[email protected]云机 ~]# mkdir -p /data/bakup
6.修改目录权限
[[email protected]云机 ~]# chown rsync.rsync /data/bakup
7.启动服务
[[email protected]云机 ~]# rsync --daemon
说明：如果配置文件路径不是/etc/rsyncd.conf，启动是需要使用“–config” 参数指定配置文件。
8.查看服务是否启动
[[email protected]云机 ~]# ps axu |grep rsync
root
9198 0.0 0.0 107652 672
? Ss 19:41
0:00 rsync --daemon --config /etc/rsync/rsyncd.conf
root
9205 0.0 0.0 103308 848 pts/0
S+ 19:44 0:00 grep rsync
[[email protected]云机 ~]# netstat
-nlp |grep 873
tcp
0 0 0.0.0.0:873
0.0.0.0:*
LISTEN
9198/rsync
tcp
0 0
:::873
:::*
LISTEN 9198/rsync
9.配置rsync开机启动：
$ vi /etc/rc.local
# 在 rc.local 文件的底部，新加一行代码，代码如下
/usr/local/rsync/bin/rsync --daemon
# chmod +x /etc/rc.d/rc.local #否则重启不执行
四、客户端配置
1.创建密码文件
[[email protected]本地 ~]# vim /etc/rsyncd.password
|
4297f44b13955235245b2497399d7a93 |
2.修改文件权限
[[email protected]本地 ~]# chmod 600 /etc/rsyncd.password
3.安装客户端
[[email protected]本地 ~]# yum -y install rsync
说明：如果客户端存在，跳过该步骤。
五、测试
1.客户端上推送数据
[[email protected]本地 ~]# ll test.txt
-rw-r--r--. 1
kry kry 13 Mar 24 15:55 test.txt
[[email protected]本地 ~]# rsync -avzP test.txt [email protected]::bakup
--password-file=/etc/rsyncd.password
sending incremental file list
test.txt
11 100% 0.00kB/s 0:00:00 (xfer#1,
to-check=0/1)
sent 91 bytes received
27 bytes 236.00 bytes/sec
total size is 11 speedup
is 0.09
2.服务端查看文件是否推送成功
[[email protected]云机 bakup]# ll
total 4
-rw-r--r--. 1
rsync rsync 13 Mar 24 15:55 test.txt
3.从服务端拉取数据
[[email protected]本地 ~]# ll /tmp/
total 0
[[email protected]本地 ~]# rsync -avzP
[email protected]::bakup /tmp
--password-file=/etc/rsyncd.password
receiving incremental file
list
./
test.txt
13 100% 12.70kB/s 0:00:00 (xfer#1, to-check=7/17)
sent 64 bytes received
419 bytes 966.00 bytes/sec
total size is 13981
speedup is 28.95
4.查看拉取的文件
[ro[email protected]本地 ~]# ll /tmp/
total 4
-rw-r--r--. 1
kry kry 13 Mar 24 15:55 test.txt
说明：文件的属主组跟推送时保持一直。
六、安装配置sersync
1.下载地址
[[email protected]本地 ~]# cd /usr/local/src/
[[email protected]本地 src]# wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/sersync/sersync2.5.4_64bit_binary_stable_final.tar.gz
2.解压软件包
[[email protected]本地 src]# tar zxvf
sersync2.5.4_64bit_binary_stable_final.tar.gz
3.创建目录结构
[[email protected]本地 src]# mkdir /usr/local/sersync
[[email protected]本地 src]# mkdir /usr/local/sersync/conf
[[email protected]本地 src]# mkdir /usr/local/sersync/logs
[[email protected]本地 src]# mkdir /usr/local/sersync/bin
4.移动文件
[[email protected]本地 src]# mv GNU-Linux-x86/sersync2
/usr/local/sersync/bin/
[[email protected]本地 src]# mv GNU-Linux-x86/confxml.xml
/usr/local/sersync/conf
5.配置环境变量
[[email protected]本地 conf]# vim /etc/profile.d/sersync.sh
#!/bin/bash
export PATH=$PATH:/usr/local/sersync/bin
[[email protected]本地 conf]# source /etc/profile.d/sersync.sh
6.配置
[[email protected]本地 src]# cd /usr/local/sersync/conf
[[email protected]本地 conf]# cp confxml.xml confxml.xml.bak
[[email protected]本地 conf]# vim confxml.xml
|
<?xml version="1.0" encoding="ISO-8859-1"?> <head version="2.5"> # 设置本地IP和端口 <host hostip="localhost" port="8008"></host> # 开启DUBUG模式 <debug start="false"/> # 开启xfs文件系统 <fileSystem xfs="false"/> # 同步时忽略推送的文件(正则表达式),默认关闭 <filter start="false"> <exclude expression="(.*)\.svn"></exclude> <exclude expression="(.*)\.gz"></exclude> <exclude expression="^info/*"></exclude> <exclude expression="^static/*"></exclude> </filter> # 设置要监控的事件 <inotify> <delete start="false"/> <createFolder start="true"/> <createFile start="false"/> <closeWrite start="true"/> <moveFrom start="true"/> <moveTo start="true"/> <attrib start="false"/> <modify start="false"/> </inotify> <sersync> # 本地监视目录路径 <localpath watch=" /data/bakup/"> #定义同步Server ip和模块 <remote ip="47.93.252.110" name="bakup"/> <!--<remote ip="192.168.8.39" name="tongbu"/>--> <!--<remote ip="192.168.8.40" name="tongbu"/>--> </localpath> <rsync> # rsync指令参数 <commonParams params="-artuz"/> # rsync同步认证 <auth start="true" users="rsync_bakup" passwordfile="/etc/rsyncd.password"/> # 设置rsync远程服务端口，非默认端口需要打开自定义(若开启rsync+ssh, 则这里需定义SSH端口) <userDefinedPort start="false" port="874"/><!-- port=874 --> # 设置超时时间 <timeout start="ture" time="100"/><!-- timeout=100 --> # 设置rsync+ssh加密传输模式,默认关闭，开启需设置SSH加密证书 <ssh start="false"/> </rsync> # sersync传输失败日志脚本路径，每隔60会重新执行该脚本，执行完毕会自动清空。 <failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once--> # 设置rsync定时传输，默认关闭 <crontab start="false" schedule="600"><!--600mins--> <crontabfilter start="false"> <exclude expression="*.php"></exclude> <exclude expression="info/*"></exclude> </crontabfilter> </crontab> # 设置sersync传输后调用name指定的插件脚本，默认关闭 <plugin start="false" name="command"/> </sersync> # 插件脚本范例 <plugin name="command"> <param prefix="/bin/sh" suffix="" ignoreError="true"/> <!--prefix /opt/tongbu/mmm.sh suffix--> <filter start="false"> <include expression="(.*)\.php"/> <include expression="(.*)\.sh"/> </filter> </plugin> # 插件脚本范例 <plugin name="socket"> <localpath watch="/opt/tongbu"> <deshost ip="192.168.138.20" port="8009"/> </localpath> </plugin> <plugin name="refreshCDN"> <localpath watch="/data0/htdocs/cms.xoyo.com/site/"> <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/> <sendurl base="http://pic.xoyo.com/cms"/> <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/p_w_picpaths"/> </localpath> </plugin> </head> |
7.启动Sersync
[[email protected]本地 conf]# sersync2 -d -r -o
/usr/local/sersync/conf/confxml.xml
说明：还有些其他参数，使用“-h”选项查看。
8.设置sersync监控开机自动执行
#vi /etc/rc.d/rc.local
#编辑，在最后添加一行
/usr/local/sersync/bin/sersync2 -d -r -o /usr/local/sersync/conf/confxml.xml ＃设置开机自动运行脚本
# chmod +x /etc/rc.d/rc.local #否则重启不执行
七、测试
1.创建测试文件
[[email protected]本地 ~]# cd /data/bakup/
[[email protected]本地 bakup]# for f in `seq 10`;do touch $f;done
2.检查数据是否同步Rsync server
[[email protected]云机 ~]# cd /data/bakup/
[[email protected]云机 ~]# ll
八、Rsync服务常见问题汇总讲解：
1 客户端的错误现象：No route to host
rsync服务端开启的iptables防火墙
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
rsync: failed to connect to 172.16.1.41: No route to host (113)
rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
异常问题解决：
关闭rsync服务端的防火墙服务（iptables）
[[email protected] mnt]# /etc/init.d/iptables stop
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
[[email protected] mnt]# /etc/init.d/iptables status
iptables: Firewall is not running.
==============================================================================================
2 ERROR: The remote path must start with a module name not a /
rsync客户端执行rsync命令错误：
客户端的错误现象：
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::/backup
ERROR: The remote path must start with a module name not a /
rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
异常问题解决：
rsync命令语法理解错误，::/backup是错误的语法，应该为::backup(rsync模块)
==============================================================================================
3 @ERROR: auth failed on module backup
3. @ERROR: auth failed on module oldboy
客户端的错误现象：
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
Password:
@ERROR: auth failed on module backup
rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
异常问题解决：
1. 密码输入错误，用户名错误 #这里的密码是你设置的rsync的密码，千万别输错了！！
2. secrets file = /etc/rsync.password指定的密码文件和实际密码文件名称不一致
3. /etc/rsync.password文件权限不是600
4. rsync_backup:123456密码配置文件后面注意不要有空格
5. rsync客户端密码文件中只输入密码信息即可，不要输入虚拟认证用户名称
==============================================================================================
4 @ERROR: Unknown module ‘backup‘
4. Unknown module ‘backup‘
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
@ERROR: Unknown module ‘backup‘
rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
异常问题解决：
1、 /etc/rsyncd.conf配置文件模块名称书写错误
2、配置文件中网段限制不对
==============================================================================================
5 Permission denied
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
Password:
sending incremental file list
hosts
rsync: mkstemp ".hosts.5z3AOA" (in backup) failed: Permission denied (13)
sent 196 bytes received 27 bytes 63.71 bytes/sec
total size is 349 speedup is 1.57
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
异常问题解决：
1. 共享目录的属主和属组不正确，不是rsync
2. 共享目录的权限不正确，不是755
==============================================================================================
6 chdir failed
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
Password:
@ERROR: chdir failed
rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
异常问题解决：
1. 备份存储目录没有建立
2. 建立的备份存储目录和配置文件定义不一致
[[email protected] backup]# /etc/init.d/xinetd restart
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Stopping xinetd: [ OK ]
Starting xinetd: shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[ OK ]
说明：如果没有备份存储目录，xinetd服务都不能正确启动
==============================================================================================
7 invalid uid rsync
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup
Password:
@ERROR: invalid uid rsync
rsync error: error starting client-server protocol (code 5) at main.c(1503) [sender=3.0.6]
异常问题解决：
rsync服务对应rsync虚拟用户不存在了
==============================================================================================
8 客户端已经配置了密码文件，但免秘钥登录方式，依旧需要输入密码
password file must not be other-accessible
[[email protected] tmp]# rsync -avz /etc/hosts [email protected]::backup --password-file=/etc/rsync.password
password file must not be other-accessible
continuing without password file
Password:
sending incremental file list
sent 26 bytes received 8 bytes 5.23 bytes/sec
total size is 349 speedup is 10.26
异常问题解决：
rsync客户端的秘钥文件也必须是600权限
==============================================================================================
9 rsync客户端连接慢问题
错误日志输出
2017/03/08 20:14:43 [3422] params.c:Parameter() - Ignoring badly formed line in configuration file: ignore errors
2017/03/08 20:14:43 [3422] name lookup failed for 172.16.1.31: Name or service not known
2017/03/08 20:14:43 [3422] connect from UNKNOWN (172.16.1.31)
2017/03/08 20:14:43 [3422] rsync to backup/ from [email protected] (172.16.1.31)
2017/03/08 20:14:43 [3422] receiving file list
2017/03/08 20:14:43 [3422] sent 76 bytes received 83 bytes total size 349
正确日志输出
2017/03/08 20:16:45 [3443] params.c:Parameter() - Ignoring badly formed line in configuration file: ignore errors
2017/03/08 20:16:45 [3443] connect from nfs02 (172.16.1.31)
2017/03/08 20:16:45 [3443] rsync to backup/ from [email protected] (172.16.1.31)
2017/03/08 20:16:45 [3443] receiving file list
2017/03/08 20:16:45 [3443] sent 76 bytes received 83 bytes total size 349
异常问题解决：
查看日志进行分析
==============================================================================================
10 rsync服务没有正确启动Connection refused (111)
[[email protected] ~]# rsync -avz /etc/hosts [email protected]::backup
rsync: failed to connect to 172.16.1.41: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(124) [sender=3.0.6]
解决 rsync服务没开启
[[email protected] ~]# rsync --daemon
[[email protected] ~]# ss -lntup |grep rsync
tcp LISTEN 0 5 :::873 :::* users:(("rsync",1434,5))
tcp LISTEN 0 5 *:873 *:* users:(("rsync",1434,4))
[[email protected] ~]# rsync -avz /etc/hosts [email protected]::backup
Password:
sending incremental file list
hosts
sent 196 bytes received 27 bytes 49.56 bytes/sec
total size is 349 speedup is 1.57
==============================================================================================
11 port 22: Connection refused
环境：本地服务器集群内部传输利用远程ssh 报错
利用（telnet 172.16.1.31 22） 排查服务监听状态后采取的解决方法
[[email protected] ~]# rsync /etc/hosts 172.16.1.31:/tmp
ssh: connect to host 172.16.1.31 port 22: Connection refused
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
排错思路：
[[email protected] ~]# ping 172.16.1.31
PING 172.16.1.31 (172.16.1.31) 56(84) bytes of data.
64 bytes from 172.16.1.31: icmp_seq=1 ttl=64 time=0.628 ms
64 bytes from 172.16.1.31: icmp_seq=2 ttl=64 time=0.393 ms
64 bytes from 172.16.1.31: icmp_seq=3 ttl=64 time=1.06 ms
64 bytes from 172.16.1.31: icmp_seq=4 ttl=64 time=0.745 ms
[[email protected] ~]# traceroute 172.16.1.31
traceroute to 172.16.1.31 (172.16.1.31), 30 hops max, 60 byte packets
1 nfs01 (172.16.1.31) 0.597 ms 0.189 ms 0.965 ms
/etc/init.d/iptables status
iptables: Firewall is not running.
[[email protected] ~]#
[[email protected] ~]# netstat -lntup|grep 22
p 0 0 10.0.0.31:22 0.0.0.0:* LISTEN 1187/sshd
故障原因：无法连接
telnet 172.16.1.31 22
解决方法：
[[email protected]]# vim /etc/ssh/sshd_config
#Port 22
#AddressFamily any
#ListenAddress 10.0.0.31 改为 0.0.0.0
#ListenAddress ::
总结：内网传输通过SSH pro 22 表明22端口链接不上
==============================================================================================
Rsync服务端排错思路
m 查看rsync服务配置文件路径是否正确 /etc/rsyncd.conf
m 查看配置文件例的host allow，host deny，允许的ip网段是否是允许客户端访问的ip网段
m 查看配置文件中path参数里的路径是否存在，权限是否正确（正常应为配置文件中的UUID参数对应的属主和组）
m 查看rsync服务是否启动，端口是否存在 ps -ef netstat -lntup
m 查看iptables防火墙和SELinux是否开启允许rsync服务通过，也可以关闭
m 查看服务端rsync配置文件里的密码权限是否为600 密码文件格式是否正确，正确格式（用户名：密码）文件路径和配置文件里的secrect files 参数对应
m 如果是推送数据，要查看，配置rsyncd.conf 文件中用户是否对模块下目录有可读的权限
==============================================================================================
客户端排错思路
m 查看客户端rsync配置的密码文件是否为600的权限，密码文件格式是否正确，注意：仅需要有密码，并且和服务端的密码一致
m 用telnet链接rsync服务器ip地址873端口，查看服务是否启动（可测试服务端防火墙是否阻挡telnet10.0.0.100 873）
m 客户端执行命令是 rsync -avzP [email protected]::backup/test/test/ --password-file=/etc/rsync.password
m 此命令要记清楚尤其10.0.0.100::backup/test/处的双引号及随后的backup为模块名称
原文地址：https://www.cnblogs.com/recall-lonely/p/10683419.html