zabbix自带的“Template OS Linux”模板支持监控已挂载的磁盘空间利用率,是利用LLD(Low-level discovery)实现的,却没有对磁盘IO监控。本篇文章就介绍利用zabbix LLD功能监控磁盘IO的方法。
思路:首先创建Discovery rules,在agent端配置对应的UserParameter,调用编写好的shell脚本,动态查找出已经挂载的磁盘分区;然后创建Item prototypes,使用vfs.dev.read[device,<type>,<mode>]和vfs.dev.write[device,<type>,<mode>]来监控磁盘分区的IO,包括磁盘读写速率、每秒读写扇区数、每秒读写操作数;然后创建Graph prototypes,生成磁盘IO图表。
效果图:
磁盘读写速率:
磁盘每秒读写扇区数:
磁盘每秒读写操作数:
下面给出具体监控步骤:
一、编写脚本查找已挂载的磁盘分区
1)在zabbix客户端创建查找磁盘分区的脚本mount_disk_discovery.sh,放到/etc/zabbix/monitor_scripts目录下,内容如下:
#!/bin/bash #Function: low-level discovery mounted disk #Script_name: mount_disk_discovery.sh mount_disk_discovery() { local regexp="\b(btrfs|ext2|ext3|ext4|jfs|reiser|xfs|ffs|ufs|jfs|jfs2|vxfs|hfs|ntfs|fat32|zfs)\b" local tmpfile="/tmp/mounts.tmp" :> "$tmpfile" # 过滤所有已挂载的文件系统 egrep "$regexp" /proc/mounts > "$tmpfile" local num=$(cat "$tmpfile" | wc -l) printf ‘{\n‘ printf ‘\t"data":[ ‘ while read line;do # 磁盘分区名称 DEV_NAME=$(echo $line | awk ‘{print $1}‘) # 文件系统名称,即磁盘分区的挂载点 FS_NAME=$(echo $line | awk ‘{print $2}‘) # blockdev命令获取扇区大小,用于计算磁盘读写速率 SEC_SIZE=$(sudo /sbin/blockdev --getss $DEV_NAME 2>/dev/null) printf ‘\n\t\t{‘ printf "\"{#DEV_NAME}\":\"${DEV_NAME}\"," printf "\"{#FS_NAME}\":\"${FS_NAME}\"," printf "\"{#SEC_SIZE}\":\"${SEC_SIZE}\"}" ((num--)) [ "$num" == 0 ] && break printf "," done < "$tmpfile" printf ‘\n\t]\n‘ printf ‘}\n‘ } case "$1" in mount_disk_discovery) "$1" ;; *) echo "Bad Parameter." echo "Usage: $0 mount_disk_discovery" exit 1 ;; esac
脚本首先过滤/proc/mounts中的磁盘类型,然后输出包含磁盘名称、文件系统名称和扇区大小的Json数据。
2)设置相关文件权限:
touch /tmp/mounts.tmp chown zabbix:zabbix /tmp/mounts.tmp chown -R zabbix:zabbix /etc/zabbix/monitor_scripts chmod 755 /etc/zabbix/monitor_scripts/mount_disk_discovery.sh
3)测试一下:
[[email protected] ~]# /etc/zabbix/monitor_scripts/mount_disk_discovery.sh mount_disk_discovery { "data":[ {"{#DEV_NAME}":"/dev/sda3","{#FS_NAME}":"/","{#SEC_SIZE}":"512"}, {"{#DEV_NAME}":"/dev/sda1","{#FS_NAME}":"/boot","{#SEC_SIZE}":"512"} ] }
上面的数据表示,磁盘分区/dev/sda3挂载到了根目录,扇区大小为512B;/dev/sda1挂载到了/boot目录,扇区大小为512B。
二、配置zabbix客户端
1)编辑zabbix agent配置文件,设置Include目录:
Include=/etc/zabbix/zabbix_agentd.d/
agent启动时会自动加载Include目录下的配置文件。
2)在/etc/zabbix/zabbix_agentd.d/目录下建立disk_lld.conf,内容如下:
UserParameter=mount_disk_discovery,/bin/bash /etc/zabbix/monitor_scripts/mount_disk_discovery.sh mount_disk_discovery
表示:建立用户自定义参数mount_disk_discovery,当zabbix server向agent请求这个item key时,agent会调用/etc/zabbix/monitor_scripts/mount_disk_discovery.sh脚本,返回json数据。
3)由于zabbix agent的启动用户是zabbix,所以要授权zabbix用户无密码运行/sbin/blockdev命令。使用visudo命令,在/etc/sudoers里注释掉Defaults requiretty:
#Defaults requiretty
并在最后面加上:
zabbix ALL=(ALL) NOPASSWD: /sbin/blockdev
这样在脚本里执行类似“sudo /sbin/blockdev --getss /dev/sda1”这样的指令就没问题了。
4)最后重启zabbix agent
/etc/init.d/zabbix-agent restart
5)测试一下:
我们从zabbix server上远程get一下mount_disk_discovery
如上图DEV_NAME、FS_NAME、SEC_SIZE都有值,说明客户端配置正确。
三、配置zabbix前台
1)登录zabbix前台,选择Configuration->Templates->Import,导入“Template Linux DiskIO”模板(见附件)。
2)进入主机配置界面,链接“Template Linux DiskIO”模板
至此,全部配置结束了,耐心等待监控数据的出现吧~
下面介绍下模板里LLD的配置。
3)模板的Discovery rule介绍
Discovery rule的Key为mount_disk_discovery,对应zabbix agent配置里的:
UserParameter=mount_disk_discovery,/bin/bash /etc/zabbix/monitor_scripts/mount_disk_discovery.sh mount_disk_discovery
每隔一段时间,server会向agent请求mount_disk_discovery,然后agent调用/etc/zabbix/monitor_scripts/mount_disk_discovery.sh脚本,返回json数据给server。
4)模板的Item prototypes介绍
其中Sectors和Operations的监控项都是Zabbix agent类型的。通过阅读zabbix C源码,我们会发现zabbix agent是根据/proc/diskstats文件计算每秒Sectors和Operations,默认取一分钟内的平均值。有兴趣的同学可以读一读src/libs/zbxsysinfo/linux/diskio.c。
磁盘读写speed监控项是Calculated类型的,通过每秒读写扇区数乘以扇区大小计算而来的,见下图:
5)模板的Graph prototypes介绍
模板对每个磁盘分区都建立三个图表,包括磁盘读写速率图表、读写扇区数图表、读写操作数图表。
附:客户端一键配置脚本
#!/bin/sh # # Filename: autoMonitorDiskIO.sh # Date: 2014/01/05 # Author: Qicheng # Website: http://qicheng0211.blog.51cto.com/ # Description: 部署zabbix low-level discovery 监控磁盘IO # Notes: 在被监控客户端运行此脚本,前提条件已经安装好zabbix agent # ROOT_UID=0 if [ "$UID" -ne "$ROOT_UID" ];then echo "Error: Please run this script as root user." exit 1 fi # 自行修改为你的zabbix agent配置文件路径 AGENT_CONF="/etc/zabbix/zabbix_agentd.conf" mkdir -p /etc/zabbix/monitor_scripts # 创建 low-level discovery mounted disk 脚本 cat > /etc/zabbix/monitor_scripts/mount_disk_discovery.sh << ‘EOF‘ #!/bin/bash #Function: low-level discovery mounted disk #Script_name: mount_disk_discovery.sh mount_disk_discovery() { local regexp="\b(btrfs|ext2|ext3|ext4|jfs|reiser|xfs|ffs|ufs|jfs|jfs2|vxfs|hfs|ntfs|fat32|zfs)\b" local tmpfile="/tmp/mounts.tmp" :> "$tmpfile" egrep "$regexp" /proc/mounts > "$tmpfile" local num=$(cat "$tmpfile" | wc -l) printf ‘{\n‘ printf ‘\t"data":[ ‘ while read line;do DEV_NAME=$(echo $line | awk ‘{print $1}‘) FS_NAME=$(echo $line | awk ‘{print $2}‘) SEC_SIZE=$(sudo /sbin/blockdev --getss $DEV_NAME 2>/dev/null) printf ‘\n\t\t{‘ printf "\"{#DEV_NAME}\":\"${DEV_NAME}\"," printf "\"{#FS_NAME}\":\"${FS_NAME}\"," printf "\"{#SEC_SIZE}\":\"${SEC_SIZE}\"}" ((num--)) [ "$num" == 0 ] && break printf "," done < "$tmpfile" printf ‘\n\t]\n‘ printf ‘}\n‘ } case "$1" in mount_disk_discovery) "$1" ;; *) echo "Bad Parameter." echo "Usage: $0 mount_disk_discovery" exit 1 ;; esac EOF touch /tmp/mounts.tmp chown zabbix:zabbix /tmp/mounts.tmp chown -R zabbix:zabbix /etc/zabbix/monitor_scripts chmod 755 /etc/zabbix/monitor_scripts/mount_disk_discovery.sh # 判断配置文件是否存在 [ -f "${AGENT_CONF}" ] || { echo "ERROR: File ${AGENT_CONF} does not exist.";exit 1;} include=`grep ‘^Include‘ ${AGENT_CONF} | cut -d‘=‘ -f2` # 在配置文件中添加自定义参数 if [ -d "$include" ];then cat > $include/disk_lld.conf << ‘EOF‘ UserParameter=mount_disk_discovery,/bin/bash /etc/zabbix/monitor_scripts/mount_disk_discovery.sh mount_disk_discovery EOF else grep -q ‘^UserParameter=mount_disk_discovery‘ ${AGENT_CONF} || cat >> ${AGENT_CONF} << ‘EOF‘ UserParameter=mount_disk_discovery,/bin/bash /etc/zabbix/monitor_scripts/mount_disk_discovery.sh mount_disk_discovery EOF fi # 授权zabbix用户无密码运行/sbin/blockdev命令 chmod +w /etc/sudoers sed -i ‘/^Defaults\s\+requiretty/s/^/#/‘ /etc/sudoers grep -q ‘^zabbix ALL=(ALL).*blockdev‘ /etc/sudoers || echo ‘zabbix ALL=(ALL) NOPASSWD: /sbin/blockdev‘ >> /etc/sudoers chmod 440 /etc/sudoers # 重启agent服务 [ -f ‘/etc/init.d/zabbix-agent‘ ] && /etc/init.d/zabbix-agent restart || echo "需手动重启zabbix agent服务."