安装inotify
- [[email protected] ~]# mkdir -p /home/oldboy/tools
安装inotify-tools-3.14.tar.gz
- [[email protected] tools]# ls -l /proc/sys/fs/inotify/ #出现下面三个表示支持inotify
- total 0
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_queued_events
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_instances
- -rw-r--r-- 1 root root 0 Feb 6 15:36 max_user_watches
解压：
- [[email protected] tools]# tar zxf inotify-tools-3.14.tar.gz
编译安装：
- [[email protected] tools]# cd inotify-tools-3.14
- [[email protected] inotify-tools-3.14]# ./configure --prefix=/usr/local/inotify-tools-3.14
- [[email protected] inotify-tools-3.14]# echo $?
- 0
- [[email protected] inotify-tools-3.14]# make && make install
- [[email protected] inotify-tools-3.14]# echo $? #0表示安装正确
- 0
- [[email protected] tools]# ln -s /usr/local/inotify-tools-3.14/ /usr/local/inotify-tools #创建软连接
- [[email protected] tools]# ls -l /usr/local/inotify-tools/
- total 16
- drwxr-xr-x. 2 root root 4096 Feb 6 15:43 bin #inotify执行命令（二进制）
- drwxr-xr-x. 3 root root 4096 Feb 6 15:42 include #inotify程序所需要的头文件
- drwxr-xr-x. 2 root root 4096 Feb 6 15:43 lib #动态链接的库文件
- drwxr-xr-x. 4 root root 4096 Feb 6 15:42 share #帮助文档
- [[email protected] inotify-tools]# ll bin/
- total 88
- -rwxr-xr-x. 1 root root 44287 Feb 6 15:43 inotifywait
- -rwxr-xr-x. 1 root root 41409 Feb 6 15:43 inotifywatch
inotifywait：在被监控的文件或目录上等待特定文件系统事件（open、close、delete等）发生，执行后处于阻塞转态，适合在shell脚本中使用。
inotifywatch：收集被监控的文件系统使用度统计数据，指文件系统事件发生的次数统计。
inotifywait命令常用参数
- [[email protected] inotify-tools]# ./bin/inotifywait --help
- inotifywait 3.14
- Wait for a particular event on a file or set of files.
- Usage: inotifywait [ options ] file1 [ file2 ] [ file3 ] [ ... ]
- Options:
- -h|--help Show this help text.
- @<file> Exclude the specified file from being watched.
- --exclude <pattern>
- Exclude all events on files matching the
- extended regular expression <pattern>.
- --excludei <pattern>
- Like --exclude but case insensitive. #排除文件或目录时，不区分大小写
- -m|--monitor Keep listening for events forever. Without
- this option, inotifywait will exit after one
- event
is received. #始终保持事件监听状态 - -d|--daemon Same as --monitor, except run in the background
- logging events to a file specified by --outfile.
- Implies --syslog.
- -r|--recursive Watch directories recursively. #递归查询目录
- --fromfile <file>
- Read files to watch from <file> or `-‘ for stdin.
- -o|--outfile <file>
- Print events to <file> rather than stdout.
- -s|--syslog Send errors to syslog rather than stderr.
- -q|--quiet Print less (only print events). #打印很少的信息，仅仅打印监控事件的信息
- -qq Print nothing (not even events).
- --format <fmt> Print using a specified printf-like format
- string; read the man page for more details.
- --timefmt <fmt> strftime-compatible format string
for use with - %T in --format string. #指定事件输出的格式
- -c|--csv Print events in CSV format.
- -t|--timeout <seconds>
- When listening for a single event, time out after
- waiting for an event
for <seconds> seconds. - If <seconds> is 0, inotifywait will never time out.
- -e|--event <event1> [ -e|--event <event2> ... ]
- Listen for specific event(s). If omitted, all events are
- listened for. #监控事件
- Exit status:
- 0 - An event you asked to watch for was received.
- 1 - An event you did not ask to watch for was received
- (usually delete_self or unmount), or some error occurred.
- 2 - The --timeout option was given and no events occurred
- in the specified interval of time.
- Events:
- access file or directory contents were read
- modify file or directory contents were written
- attrib file or directory attributes changed
- close_write file or directory closed, after being opened in
- writeable mode
- close_nowrite file or directory closed, after being opened in
- read-only mode
- close file or directory closed, regardless of read/write mode
- open file or directory opened
- moved_to file or directory moved to watched directory
- moved_from file or directory moved from watched directory
- move file or directory moved to or from watched directory
- create file or directory created within watched directory
- delete file or directory deleted within watched directory
- delete_self file or directory was deleted
- unmount file system containing file or directory unmounted
测试一：
窗口一：
- [[email protected] inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt ‘%d/%m/%y %H:%M‘ --format ‘%T %w%f‘ -e create /data #create创建（监控创建事件）
- 06/02/17 16:13 /data/test.txt
- 06/02/17 16:13 /data/test2.txt
- 06/02/17 16:14 /data/1.txt
- 06/02/17 16:14 /data/2.txt
- 06/02/17 16:14 /data/3.txt
- 06/02/17 16:14 /data/4.txt
窗口二：
- [[email protected] ~]# cd /data/
- [[email protected] data]# ls
- [[email protected] data]# touch test.txt
- [[email protected] data]# touch test2.txt
- [[email protected] data]# touch {1..4}.txt
测试二：
窗口一：
- [[email protected] inotify-tools]# /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt ‘%d/%m/%y %H:%M‘ --format ‘%T %w%f‘ -e delete /data #delete删除（监控删除事件）
- 06/02/17 16:17 /data/test.txt
- 06/02/17 16:17 /data/test2.txt
窗口二：
- [[email protected] data]# rm -f test.txt
- [[email protected] data]# rm -f test2.txt
同时监控多个事件，事件之间用逗号分隔。
- /usr/local/inotify-tools/bin/inotifywait -mrq --timefmt ‘%d/%m/%y %H:%M‘ --format ‘%T %w%f‘ -e delete,create,close_write /data
测试：
窗口一：
- [[email protected] scripts]# cat inotify.sh
- #!/bin/bash
- inotify=/usr/local/inotify-tools/bin/inotifywait
- $inotify -mrq --format ‘%w%f‘ -e create,close_write,delete /data
- |while read file
- do
- cd / &&
- rsync -az ./data --delete [email protected]::backup/
- --password-file=/etc/rsync.password
- done
- [[email protected] scripts]# sh -x inotify.sh
- + inotify=/usr/local/inotify-tools/bin/inotifywait
- + read file
- + /usr/local/inotify-tools/bin/inotifywait -mrq --format %w%f -e create,close_write,delete /data
- + cd /
- + rsync -az ./data --delete [email protected]::backup/ --password-file=/etc/rsync.password
- + read file
- + cd /
- + rsync -az ./data --delete [email protected]::backup/ --password-file=/etc/rsync.password
- + read file
窗口二：
- [[email protected] data]# touch oldboy.txt
- [[email protected] data]# touch test.log
备份服务器：
- [[email protected] backup]# ls
- data
- [[email protected] backup]# tree
- .
- └── data
- ├── 2.txt
- ├── 3.txt
- ├── 4.txt
- ├── oldboy.txt
- ├── test.log
- └── test.txt
- [[email protected] scripts]# /bin/sh /server/scripts/inotify.sh & #放入后台执行
写入rc.local
应用场景：10—300k小文件并发200—300，不会有延迟。
关键参数说明：
在/proc/sys/fs/inotify目录下有三个文件，对inotify机制有一定的限制。
max_user_watches：设置inotifywait或inotifywatch命令可以监视的文件数量（单进程）。
max_user_instances：设置每个用户可以运行的inotifywait或inotifywatch命令的进程数。
max_queued_events：设置inotify实例事件（event）队列可容纳的事件数量。
- [[email protected] scripts]# cat /proc/sys/fs/inotify/max_user_watches # 修改为50000000
- 8192
- [[email protected] scripts]# cat /proc/sys/fs/inotify/max_user_instances #
- 128
- [[email protected] scripts]# cat /proc/sys/fs/inotify/max_queued_events #修改为50000000
- 16384
每秒200个文件并发，数据同步几乎无延迟。
inotify优点：
实时数据同步。
inotify缺点：
1、并发如果大于200个文件（10—100K），同步会有延迟。
2、监控到事件后，调用rsync同步是单进程的（加&并发），sersync多进程同步。
sersync的功能：
1、配置文件。
2、真正的守护进程socker。
3、可以对失败文件定时重传（定时任务）。
4、第三方的http接口。
5、默认多线程同步。
高并发数据实时同步方案：
1、文件级别：inotify（sersync）+rsync。
2、文件系统级别：drbd。
3、第三方软件的同步功能：mysql同步、oracle、mongodb。
4、程序双写。
5、业务逻辑解决。
时间： 2024-08-01 21:08:10