Sysbench 基准压测 my.cnf
[TOC]
############################################
- line： V1.1
- mail: [email protected]
- date: 2017-11-10
###########################################
一、Sysench测试前准备
1.1、压测环境
| 配置 | 信息 |
|---|---|
| 主机 | Dell PowerEdge R730xd |
| CPU | 24 * Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz |
| 内存 | 64G (16G * 4) |
| RAID | RAID1 |
| 硬盘 | 7.2K RPM, 6000G SAS, 12G/s |
| 文件系统 | ext4 |
| 系统 | Red Hat Enterprise Linux Server release 7.3 (Maipo) |
| 内核 | 3.10.0-514.el7.x86_64 |
| MySQL | MySQL5.7.18 |
| Sysbench | 1.1.0-0167e45 |
1.2、压测基准值
| 测试工具 | sysbench |
|---|---|
| 每张表初始化的数据数量 | 10,000,000 |
| 测试表数 | 16 |
| 测试脚本 | oltp_update_index.lua、oltp_update_non_index.lua |
| run time(秒) | 3600 |
| 最大请求数 | 100,000,000 |
| 并发线程数 | 8 - 256 |
| 生成数据量 | 127.56G |
二、进行OLTP_update测试
2.1、安装压测工具sysbench
- 标准安装
[[email protected](252) /r2/soft/dbtest] curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash &&
sudo yum -y install sysbench
- my.cnf文件
# line V1.2
# mail:
# data: 2017-10-26
# file_name: my.cnf
# function: 无数据一致性要求，只要求update速度快
## optimized mysql configure file my.cnf
## 注意：建议参数可需要根据实际情况作调整
## 本配置文件主要适用于MySQL 5.7.18版本
# ********* 以下重要参数必须修改核对 *********
# 1.innodb_flush_log_at_trx_commit=2
# 2.sync_binlog = 0
# 3.innodb_strict_mode = OFF
# 4.innodb_flush_method = O_DIRECT
# 5.lower_case_table_names = 0
# 6.character-set-server = utf8
# 7.sql_mode 配置为空值
# 8.server-id =1 修改成对应数值
# 9.innodb_buffer_pool_size = 32G
# ********************************************
[client]
port = 3306
socket = /r2/soft/dbtest/mysql-5.7.18/mysqldata/mysql.sock
#=======================================================================
# # MySQL客户端配置
#=======================================================================
[mysql]
prompt="\[email protected] \R:\m:\s [\d]> "
#no-auto-rehash
#=======================================================================
# MySQL服务器全局配置
#=======================================================================
[mysqld]
user = mysql
port = 3306
server-id = 168
tmpdir = /r2/soft/dbtest/mysql-5.7.18/mysqldata
datadir = /r2/soft/dbtest/mysql-5.7.18/mysqldata
socket = /r2/soft/dbtest/mysql-5.7.18/mysqldata/mysql.sock
wait_timeout = 31536000
#interactive_timeout = 600
sql_mode =
#sql_mode 配置为空值
skip_name_resolve = 1
lower_case_table_names = 0
character-set-server = utf8
#auto_increment_increment = 1
#auto_increment_offset = 1
# init_connect =
######################### 性能参数 ####################
open_files_limit = 1024000
max_connections = 10000
max_user_connections=9990
max_connect_errors = 100000
table_open_cache = 1024
max_allowed_packet = 128M
thread_cache_size = 64
max_heap_table_size = 32M
query_cache_type = 0
###global cache ###
key_buffer_size = 16M
query_cache_size = 0
###session cache ###
sort_buffer_size = 8M #排序缓冲
join_buffer_size = 4M #表连接缓冲
read_buffer_size = 8M #顺序读缓冲
read_rnd_buffer_size = 8M #随机读缓冲
tmp_table_size = 32M #内存临时表
binlog_cache_size = 4M #二进制日志缓冲
thread_stack = 256KB #线程的堆栈的大小
######################### binlog设置 #####################
binlog_format = MIXED
log_bin = /r2/soft/dbtest/mysql-5.7.18/mysqldata/binlog
max_binlog_cache_size = 1G
max_binlog_size = 1G
expire_logs_days = 30
sync_binlog = 0 #重要参数必须修改为0
######################### 复制设置 ########################
log_slave_updates=1
#replicate-do-db=User
#binlog-ignore-db = test
#slave-skip-errors=1146,1032,1062
### GTID 配置 ###
gtid_mode=ON
enforce-gtid-consistency=true
######################### innodb ##########################
default_storage_engine = InnoDB
innodb_data_file_path = ibdata1:1G:autoextend
innodb_buffer_pool_size = 32G #系统内存50%
innodb_open_files = 1000000
innodb_flush_log_at_trx_commit = 2 #线上服务器必须配置为2
innodb_file_per_table = 1
innodb_lock_wait_timeout = 5
# 根据您的服务器IOPS能力适当调整innodb_io_capacity
# 一般配普通SSD盘的话，可以调整到 10000 - 20000
#innodb_io_capacity = 200
innodb_io_capacity = 600
innodb_io_capacity_max = 20000
innodb_flush_method = O_DIRECT
innodb_log_file_size = 2G
innodb_log_files_in_group = 2
innodb_large_prefix = 0
innodb_thread_concurrency = 64
innodb_strict_mode = OFF
innodb_sort_buffer_size = 4194304
######################### log 设置 #####################
log_error = /r2/soft/dbtest/mysql-5.7.18/mysqldata/error.log
slow_query_log = 1
long_query_time = 1
slow_query_log_file = /r2/soft/dbtest/mysql-5.7.18/mysqldata/slow.log
#=======================================================================
# MySQL mysqldump配置
#=======================================================================
[mysqldump]
quick
max_allowed_packet = 32M
#=======================================================================
# MySQL mysqld_safe配置
#=======================================================================
[mysqld_safe]
log_error = /r2/soft/dbtest/mysql-5.7.18/mysqldata/error.log
pid_file = /r2/soft/dbtest/mysql-5.7.18/mysqldata/mysqldb.pid
- Test 脚本
# line V1.2
# mail: [email protected]
# date: 2017-10-26
# file_name: mysql_oltp_sysbench.sh
# 修改符合公司环境
#通过sysbench测试mysql相关性能，并将关键数据存储于‘dbtest.sysbenc_test’表中
#----------自定义部分----------
#定义记录测试结果的mysql连接相关参数，本例我在测试机上记录测试结果
m_user=‘gcdb‘
m_passwd=‘iforgot‘
m_port=‘3306‘
m_host=‘localhost‘
#测试结果存储于哪个库
m_db=‘dbtest‘
#测试结果存储于哪个表
m_table=‘sysbench_test‘
m_sock=‘/r2/soft/dbtest/mysql-5.7.18/mysqldata/mysql.sock‘
#sysbench lua脚本目录
lua_dir=/usr/local/sysbench-1.1.0/share/sysbench
#sysbench 参数
SYSBENCH_PARAMETER="--table-size=10000000 --tables=32 --max-requests=100000000 --rand-type=uniform --report-interval=10 --max-time=1200 "
#SYSBENCH_PARAMETER="--mysql-table-engine=innodb --oltp-table-size=5000000 --oltp-tables-count=16 --oltp-test-mode=complex --rand-type=uniform --rand-init=on --report-interval=10 --max-time=1500 "
#画图维度（关注的六个指标，也就是会画六张图）
target="server_load request_read request_write request_per_second transactions_per_second 95_pct_time"
#定义错误日志文件
log=/tmp/mysql_oltp.log
#定义分析结果文件
data=/tmp/mysql_oltp.dat
#定义测试线程
threds_num=‘24 32 64 96 128 160 192 256‘
#每种场景的测试次数，分析时取平均值
times=1
#----------自定义部分结束----------
# sysbenc cleanup and perpare
#sysbench $lua_dir/oltp_read_only.lua --mysql-user=$6 --mysql-password=$7 --mysql-port=$5 --mysql-host=$4 $SYSBENCH_PARAMETER --threads=24 cleanup
#sysbench $lua_dir/oltp_read_only.lua --mysql-user=$6 --mysql-password=$7 --mysql-port=$5 --mysql-host=$4 $SYSBENCH_PARAMETER --threads=24 prepare
#测试函数
sb_test() {
if [ "$3" == "read-only" ];then read_only=‘on‘;else read_only=‘off‘;fi #根据脚本参数确定是否read-only
#创建记录测试信息的表
echo -e "\n---------------\n创建测测试结果表$m_db.$m_table\n---------------"
return=$(mysql -u$m_user -p$m_passwd -P$m_port -h$m_host -S $m_sock <<EOF 2>&1
CREATE TABLE IF NOT EXISTS $m_db.$m_table (
scenario varchar(30) NOT NULL DEFAULT ‘‘ COMMENT ‘测试场景‘,
server_name varchar(15) NOT NULL COMMENT ‘被测DB name‘,
test_type varchar(15) NOT NULL COMMENT ‘read-only,read-write,insert等‘,
sb_threads int(11) NOT NULL DEFAULT ‘0‘ COMMENT ‘sysbench 测试线程‘,
server_load decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘以当前线程测试完后立刻记录一分钟负载值‘,
request_read int(11) NOT NULL DEFAULT ‘0‘,
request_write int(11) NOT NULL DEFAULT ‘0‘,
transactions_per_second decimal(12,2) NOT NULL DEFAULT ‘0.00‘,
request_per_second decimal(12,2) NOT NULL DEFAULT ‘0.00‘,
95_pct_time decimal(12,2) NOT NULL DEFAULT ‘0.00‘ COMMENT ‘单位毫秒‘
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
EOF
)
if [ $? -ne 0 ];then
echo $return|sed ‘s/[Ww]arning:.*password on the command.*insecure\.//‘
#echo "create table $m_db.$m_table failed"
exit -1
fi
#开始测试,每种条件测$times次,分析时取平均值
echo -e "\n---------------\n场景:$2 模式:$3\n---------------"
for i in `seq $times`;do
for sb_threds in $threds_num;do #按照指定的sysbench线程测试
printf " %-10s %s\n" $sb_threds线程 第$i次运行...
#result 作为每次最小测试单元的结果，根据sysbench测试结果各参数的出现顺序
#以request_read、request_write、request_total、request_per_second、total_time、95_pct_time为顺序插入表中
#下条命令中，egerp之后的操作是为了对sysbench的输出做筛选和格式化，以便插入数据库
#sysbench 参数
#--oltp_table_count=1：指定测试过程中表的个数，0.5新增，0.4整个测试过程只有一个表。
#--oltp-table-size=：指定表的大小，如果指定1000，那么它会往表里初始化1000条数据
#--rand-init=on：是否随机初始化数据，如果不随机化那么初始好的数据每行内容除了主键不同外其他完全相同。
#--num-threads=：测试过程中并发线程数，看测试要求来定并发压力。
#--otlp-read-only=off：知否只读测试
#--report-interval=10：每隔多久打印一次统计信息，单位秒，0.5新增
#--rand-type=special：数据分布模式，special表示存在热点数据，uniform表示非热点数据模式，还有其他几个选项。
#--rand-spec-pct=5：这个与上面那个选项相关，热点数据的百分比，我们公司的一个应用测试出来是4.9%的热点数据。
#--mysql-table-engine=$type：表的存储引擎类型，innodb/myisam/tokudb/这些都可以。
#--max-time=8000：这个命令跑多长时间，单位秒，与之相反的是指定请求数--max-requests
# sysbench $lua_dir/oltp_update_non_index.lua --mysql-user=$6 --mysql-password=$7 --mysql-port=$5 --mysql-host=$4 sysbench $lua_dir/oltp_update_index.lua --mysql-user=$6 --mysql-password=$7 --mysql-port=$5 --mysql-host=$4 --num-threads=$sb_threds $SYSBENCH_PARAMETER run &> $log
if [ $? -ne 0 ];then
echo -e "\nSysbench error! For more information see $log"
exit -1
fi
result=$(cat $log | egrep "read:|write:|transactions:|queries:|95th\ percentile:" | sed -r -e "s/[0-9]+ \(//g" -e "s/\ per sec\.\)//g" -e "s/m?s$//g"| awk ‘{printf("%s ",$NF)}‘ | sed "s/\ /,/g" | sed "s/,$//g" | sed "s/(//g")
#测试完成后立刻记录系统一分钟负载值，可近似认为测试过程中proxy的负载抽样
load=`uptime|awk -F: ‘{print $NF}‘| awk -F , ‘{print $1}‘`
if [ -s $load ];then
load=0.00
fi
return=$(mysql -u$m_user -p$m_passwd -P$m_port -h$m_host -S $m_sock $m_db <<EOF 2>&1
INSERT INTO $m_table (scenario,server_name,test_type,sb_threads,server_load,request_read,
request_write,transactions_per_second,request_per_second,95_pct_time)
VALUES (‘$2‘,‘$4‘,‘$3‘,‘$sb_threds‘,‘$load‘,$result);
EOF
)
# echo "INSERT INTO $m_table (scenario,server_name,test_type,sb_threads,server_load,request_read,request_write,transactions_per_second,request_per_second,95_pct_time) VALUES (‘$2‘,‘$4‘,‘$3‘,‘$sb_threds‘,‘$load‘,$result)"
# exit 1
if [ $? -ne 0 ];then
echo -e "\n----------$sb_threds线程测试，第$i次插入数据库时失败----------"
#错误输出中排除mysql安全提示
echo $return|sed ‘s/[Ww]arning:.*password on the command.*insecure\.//‘
#echo "INSERT VALUES (‘$2‘,‘$4‘,‘$3‘,$sb_threds,$load,$result)"
exit -2
fi
sleep 60 #让库歇一会，也让一分钟负载能够恢复到测试前的值
done
done
}
#结果分析函数
sb_analyse() {
#2>&1|grep部分为避免安全提示,使用该技巧就无法获取SQL执行的返回码了
mysql -u$m_user -p$m_passwd -h$m_host -S $m_sock <<EOF 2>&1|grep -v ‘password on the command line‘
SELECT
scenario,
server_name,
test_type,
sb_threads,
convert(avg(server_load),decimal(12,2)) as server_load,
convert(avg(request_read),decimal(12,0)) as request_read,
convert(avg(request_write),decimal(12,0)) as request_write,
convert(avg(transactions_per_second),decimal(12,2)) as transactions_per_second,
convert(avg(request_per_second),decimal(12,2)) as request_per_second,
convert(avg(95_pct_time),decimal(12,2)) as 95_pct_time
FROM $m_db.$m_table group by scenario,server_name,test_type,sb_threads
EOF
}
#画图函数
sb_chart() {
sb_analyse >$data
for chart_type in $target;do
col_num=0 #该行及下面这个for循环用于取得三个指标在数据中的列号
for col_name in `cat $data |awk ‘NR<2 {print}‘`;do
let col_num++
if [ $col_name == $chart_type ];then break;fi
done
if [ $chart_type == "transactions_per_second" ];then #根据图表特点为不同的chart_type设置gunplot不同的key position
key_pos="top left"
unit="TPS"
elif [ $chart_type == "request_per_second" ];then #根据图表特点为不同的chart_type设置gunplot不同的key position
key_pos="top left"
#key_pos="bottom right"
unit="QPS"
elif [ $chart_type == "95_pct_time" ];then
key_pos="top left"
unit="(95_pstms)"
elif [ $chart_type == "request_write" ];then
key_pos="top left"
unit="request_write"
elif [ $chart_type == "request_read" ];then
key_pos="top left"
unit="(only_read)"
elif [ $chart_type == "server_load" ];then
key_pos="top left"
unit="(CPU_load_1_min)"
fi
plot_cmd="set term png size 800,600; set output ‘/tmp/$chart_type.png‘;set title ‘$unit‘; set ylabel ‘$chart_type‘; set grid;set key $key_pos;set ylabel ‘$chart_type‘;set xlabel ‘threads‘;plot "
# plot_cmd="set term png size 800,600;set output ‘/tmp/$chart_type.png‘;set title ‘$chart_type $unit‘;set grid;set key $key_pos;plot "
if [ $# -eq 0 ];then
#对分析结果中所有场景进行画图
for scenario in `mysql -u$m_user -p$m_passwd -h$m_host -P$m_port -s -e "select distinct(scenario) from $m_db.$m_table" 2>/dev/null`;do
sb_analyse | awk -v scenario=$scenario ‘$1 == scenario {print}‘ > /tmp/"$scenario.dat"
plot_cmd=${plot_cmd}"‘/tmp/"$scenario.dat"‘ using $col_num:xtic(4) title ‘$scenario‘ with linespoints lw 2,"
done
plot_cmd=$(echo $plot_cmd | sed ‘s/,$//g‘)
echo $plot_cmd | gnuplot
else
#只绘制指定的场景
for scenario in $*;do
sb_analyse | awk -v scenario=$scenario ‘$1 == scenario {print}‘ > /tmp/"$scenario.dat"
plot_cmd=${plot_cmd}"‘/tmp/"$scenario.dat"‘ using $col_num:xtic(4) title ‘$scenario‘ with linespoints lw 2,"
done
plot_cmd=$(echo $plot_cmd | sed ‘s/,$//g‘)
echo "$plot_cmd" | gnuplot
fi
done
}
#脚本使用说明/参数判断
if [ $# -eq 1 ] && [ $1 == "-h" -o $1 == "--help" ];then
echo -e "\nUsage: $0 test (test_scenario) (test_type) (mysql_host) (mysql_port) (mysql_user) (mysql_password)\n $0 analyse\n $0 chart [scenario]...\n"
echo ----------
echo -e "测试: 子命令test"
echo -e " test_scenario: 自定义的测试场景名"
echo -e " test_type: read-only 或 read-write, 表示测试模式"
echo -e " 其余4参数表示待测试MySQL连接相关信息,密码若包含特殊字符,将其置于单引号内"
echo -e "----------"
echo -e "分析: 子命令analyse"
echo -e "----------"
echo -e "画图: 子命令chart"
echo -e " 会在/tmp/下生成request_per_second.png transactions_per_second.png 95_pct_time.png server_load.png request_read.png request_write.png 六张图"
echo -e " chart (对分析结果中的所有测试场景画图)"
echo -e " chart scenario ... (对指定的测试场景画图，场景名依据先前自定义的名称)\n"
exit -1
elif [ "$1" == "test" -a $# -eq 7 ];then
sb_test $1 $2 $3 $4 $5 $6 $7
elif [ "$1" == "analyse" -a $# -eq 1 ];then
sb_analyse
elif [ "$1" == "chart" ];then
#chart函数可不接参数,也可接任意个‘测试场景‘作为参数
arg=($*)
arg_len=${#arg[@]}
sb_chart ${arg[@]:1:$arg_len-1}
else
echo -e "\nUsage: $0 test (test_scenario) (test_type) (mysql_host) (mysql_port) (mysql_user) (mysql_password)\n $0 analyse\n $0 chart [scenario]...\n"
fi
2.2、执行压测
- 依据测试环境修改脚本的mysql参数和sysbench参数，然后执行
- 注意：脚本会把执行结果写入到dbtest库下面的sysbench_test表里
Usage: ./sysbench_t1.sh test (test_scenario) (test_type) (mysql_host) (mysql_port) (mysql_user) (mysql_password)
./sysbench_t1.sh analyse
./sysbench_t1.sh chart [scenario]...
----------
测试: 子命令test
test_scenario: 自定义的测试场景名
test_type: read-only 或 read-write, 表示测试模式
其余4参数表示待测试MySQL连接相关信息,密码若包含特殊字符,将其置于单引号内
----------
分析: 子命令analyse
----------
画图: 子命令chart
会在/tmp/下生成request_per_second.png transactions_per_second.png 95_pct_time.png 三张图
chart (对分析结果中的所有测试场景画图)
chart scenario ... (对指定的测试场景画图，场景名依据先前自定义的名称)
[[email protected](252) /r2/soft]# ll /usr/local/sysbench/share/sysbench/
total 64
-rwxr-xr-x 1 root root 1452 11月 2 17:14 bulk_insert.lua
-rw-r--r-- 1 root root 13918 11月 2 17:14 oltp_common.lua
-rwxr-xr-x 1 root root 1290 11月 2 17:14 oltp_delete.lua
-rwxr-xr-x 1 root root 2415 11月 2 17:14 oltp_insert.lua
-rwxr-xr-x 1 root root 1265 11月 2 17:14 oltp_point_select.lua
-rwxr-xr-x 1 root root 1649 11月 2 17:14 oltp_read_only.lua
-rwxr-xr-x 1 root root 1824 11月 2 17:14 oltp_read_write.lua
-rwxr-xr-x 1 root root 1118 11月 2 17:14 oltp_update_index.lua
-rwxr-xr-x 1 root root 1127 11月 2 17:14 oltp_update_non_index.lua
-rwxr-xr-x 1 root root 1440 11月 2 17:14 oltp_write_only.lua
-rwxr-xr-x 1 root root 1919 11月 2 17:14 select_random_points.lua
-rwxr-xr-x 1 root root 2118 11月 2 17:14 select_random_ranges.lua
drwxr-xr-x 4 root root 4096 11月 2 17:14 tests
[[email protected](252) /r2/soft]# nohup ./mysql_oltp_sysbench.sh test scenario_oltp_update_non_index read-write localhost 3306 gcdb ‘iforgot‘ &
[[email protected](252) /r2/soft]# tail -f /tmp/mysql_oltp.log
[ 10s ] thds: 192 tps: 255.30 qps: 255.30 (r/w/o: 0.00/255.30/0.00) lat (ms,95%): 2728.81 err/s: 0.00 reconn/s: 0.00
三、执行结果
3.1 sysbench 执行结果
SQL statistics:
queries performed:
read: 0
write: 367252 --写总数
other: 0
total: 367252
transactions: 367252 (203.97 per sec.) -- 总事务数(每秒事务数)
queries: 367252 (203.97 per sec.) -- 读写务数(每秒事务数)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)
Throughput:
events/s (eps): 203.9661 --每秒事务数
time elapsed: 1800.5544s --总耗时
total number of events: 367252 --总事务数
Latency (ms):
min: 0.12 --最小耗时
avg: 470.60 --平均耗时
max: 48410.54 --最大耗时
95th percentile: 846.57 --超过95%平均耗时
sum: 172829974.82 --总耗时
Threads fairness:
events (avg/stddev): 3825.5417/76.81 --事件(平均值/偏差)
execution time (avg/stddev): 1800.3122/0.10 --执行时间(平均值/偏差)
3.2 查看脚本执行记录
- 查询dbtest.sysbench_test表
mysql> SELECT scenario,sb_threads,server_load,request_read,request_write,transactions_per_second,request_per_second,95_pct_time FROM dbtest.sysbench_test;
+--------------------------------+------------+-------------+--------------+---------------+-------------------------+--------------------+-------------+
| scenario | sb_threads | server_load | request_read | request_write | transactions_per_second | request_per_second | 95_pct_time |
+--------------------------------+------------+-------------+--------------+---------------+-------------------------+--------------------+-------------+
| scenario_oltp_update_index | 24 | 13.35 | 0 | 745072 | 413.84 | 413.84 | 277.21 |
| scenario_oltp_update_index | 32 | 7.81 | 0 | 249685 | 138.69 | 138.69 | 493.24 |
| scenario_oltp_update_index | 64 | 8.20 | 0 | 333937 | 185.50 | 185.50 | 733.00 |
| scenario_oltp_update_index | 96 | 20.03 | 0 | 343386 | 190.69 | 190.69 | 1032.01 |
| scenario_oltp_update_index | 128 | 6.72 | 0 | 326057 | 177.64 | 177.64 | 1479.41 |
| scenario_oltp_update_index | 160 | 7.23 | 0 | 342488 | 187.00 | 187.00 | 1869.60 |
| scenario_oltp_update_index | 192 | 9.93 | 0 | 331669 | 184.15 | 184.15 | 2493.86 |
| scenario_oltp_update_index | 256 | 6.63 | 0 | 322515 | 177.02 | 177.02 | 2680.11 |
| scenario_oltp_update_non_index | 24 | 24.78 | 0 | 1003021 | 557.19 | 557.19 | 189.93 |
| scenario_oltp_update_non_index | 32 | 17.21 | 0 | 547258 | 303.78 | 303.78 | 601.29 |
| scenario_oltp_update_non_index | 64 | 27.47 | 0 | 454084 | 251.93 | 251.93 | 1618.78 |
| scenario_oltp_update_non_index | 96 | 18.86 | 0 | 452345 | 250.91 | 250.91 | 2120.76 |
| scenario_oltp_update_non_index | 128 | 16.62 | 0 | 437129 | 242.45 | 242.45 | 2778.39 |
| scenario_oltp_update_non_index | 160 | 22.01 | 0 | 421289 | 233.70 | 233.70 | 3511.19 |
| scenario_oltp_update_non_index | 192 | 16.81 | 0 | 398962 | 221.19 | 221.19 | 4358.09 |
| scenario_oltp_update_non_index | 256 | 18.24 | 0 | 407225 | 225.80 | 225.80 | 5813.24 |
| scenario_oltp_update_non_index | 24 | 16.45 | 0 | 406238 | 225.59 | 225.59 | 746.32 |
| scenario_oltp_update_non_index | 32 | 16.82 | 0 | 419489 | 232.93 | 232.93 | 1032.01 |
| scenario_oltp_update_non_index | 64 | 23.02 | 0 | 426404 | 236.83 | 236.83 | 1836.24 |
| scenario_oltp_update_non_index | 96 | 20.04 | 0 | 457630 | 253.94 | 253.94 | 2045.74 |
| scenario_oltp_update_non_index | 128 | 24.04 | 0 | 460669 | 255.80 | 255.80 | 2632.28 |
| scenario_oltp_update_non_index | 160 | 21.59 | 0 | 454371 | 252.04 | 252.04 | 3267.19 |
| scenario_oltp_update_non_index | 192 | 18.05 | 0 | 450096 | 249.65 | 249.65 | 3911.79 |
| scenario_oltp_update_non_index | 256 | 19.35 | 0 | 437986 | 243.02 | 243.02 | 5507.54 |
| scenario_oltp_update_non_index | 24 | 15.51 | 0 | 427093 | 237.23 | 237.23 | 733.00 |
| scenario_oltp_update_non_index | 32 | 17.80 | 0 | 433449 | 240.72 | 240.72 | 977.74 |
| scenario_oltp_update_non_index | 64 | 17.65 | 0 | 432506 | 240.08 | 240.08 | 1803.47 |
| scenario_oltp_update_non_index | 96 | 18.55 | 0 | 426209 | 236.44 | 236.44 | 2279.14 |
| scenario_oltp_update_non_index | 128 | 17.06 | 0 | 415146 | 230.27 | 230.27 | 2880.27 |
| scenario_oltp_update_non_index | 160 | 19.04 | 0 | 416016 | 230.63 | 230.63 | 3511.19 |
| scenario_oltp_update_non_index | 192 | 19.73 | 0 | 411943 | 228.56 | 228.56 | 4280.32 |
| scenario_oltp_update_non_index | 256 | 17.48 | 0 | 418163 | 231.88 | 231.88 | 5709.50 |
| scenario_oltp_update_index | 24 | 11.59 | 0 | 490759 | 272.63 | 272.63 | 411.96 |
| scenario_oltp_update_index | 32 | 21.27 | 0 | 560223 | 311.19 | 311.19 | 458.96 |
| scenario_oltp_update_index | 64 | 29.03 | 0 | 357102 | 198.36 | 198.36 | 601.29 |
| scenario_oltp_update_index | 96 | 19.62 | 0 | 403699 | 222.30 | 222.30 | 846.57 |
| scenario_oltp_update_index | 128 | 15.11 | 0 | 380223 | 211.14 | 211.14 | 1170.65 |
| scenario_oltp_update_index | 160 | 16.88 | 0 | 391103 | 216.96 | 216.96 | 1506.29 |
| scenario_oltp_update_index | 192 | 8.91 | 0 | 378316 | 209.84 | 209.84 | 2045.74 |
| scenario_oltp_update_index | 256 | 8.18 | 0 | 377455 | 206.79 | 206.79 | 2932.60 |
| scenario_oltp_update_index | 24 | 8.76 | 0 | 375782 | 208.75 | 208.75 | 257.95 |
| scenario_oltp_update_index | 32 | 5.58 | 0 | 356393 | 197.99 | 197.99 | 356.70 |
| scenario_oltp_update_index | 64 | 5.52 | 0 | 377180 | 205.27 | 205.27 | 559.50 |
| scenario_oltp_update_index | 96 | 8.48 | 0 | 367252 | 203.97 | 203.97 | 846.57 |
| scenario_oltp_update_index | 128 | 11.43 | 0 | 374881 | 205.08 | 205.08 | 1149.76 |
| scenario_oltp_update_index | 160 | 9.46 | 0 | 366998 | 200.27 | 200.27 | 1618.78 |
| scenario_oltp_update_index | 192 | 5.50 | 0 | 351154 | 194.18 | 194.18 | 2159.29 |
| scenario_oltp_update_index | 256 | 11.49 | 0 | 367447 | 202.42 | 202.42 | 2449.36 |
+--------------------------------+------------+-------------+--------------+---------------+-------------------------+--------------------+-------------+
3.3 最后一次压测MySQL各种状态
- CPU MEM
- IO
- 线程数和update
- Innodb buffer
3.4 统计三次循环压测结果并画图
3.5 测试结果
- 当线程并发到64个时TPS在203左右，事务95%的平均耗时也在不断攀升，在线程数达并发到96个时候，耗时在1000ms即1s,系统不可以用
四、关于测试后的分析和建议
- 4.1、从上面io和cpu图可能执行并发压测是CPU负载上升并未达到瓶颈，但是IO使用率100%并出现io延迟，观察看到iops一直在200之下，是否受配置参数innodb_io_capacity=200限制？
- 4.2、进行sysbench fileio测试
[[email protected](252) /r2/soft/sysbench]# sysbench fileio > --time=1800 \ --30分钟
> --events=100000000 \ --最大随机数请求是100000000次
> --threads=24 \ --线程数24
> --file-num=16 \ --文件数16个
> --file-total-size=2G \ --文件总大小2G
> --file-test-mode=rndrw \ --文件测试模式，包含seqwr（顺序写）、seqrewr（顺序读写）、seqrd（顺序读）、rndrd（随机读）、rndwr（随机写）和rndrw（随机读写）
> --file-extra-flags=direct \ --以哪种模式打开O_SYNC，O_DSYNC，O_DIRECT
> --file-fsync-freq=0 \ --执行fsync函数的频率
> --file-block-size=16384 \ --块的大小，mysql默认16K
> run
sysbench 1.1.0-0167e45 (using bundled LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 24
Initializing random number generator from current time
Extra file open flags: 3
16 files, 128MiB each
2GiB total file size
Block size 16KiB
Number of IO requests: 100000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Initializing worker threads...
Threads started!
Throughput:
read: IOPS=0.00 0.00 MiB/s (0.00 MB/s)
write: IOPS=529.69 8.28 MiB/s (8.68 MB/s)
fsync: IOPS=0.00
Latency (ms):
min: 0.12
avg: 45.30
max: 1212.98
95th percentile: 287.38
sum: 43198787.07
- 分别进行rndwr（随机写）和rndrw（随机读写）各30分钟，如上图R2目录（RAID1）IOPS:
- rndwr（随机写）平均600IOPS
- rndrw（随机读写）平均写306iops，平均读460iops
4.3、建议用/r2目录所在磁盘(RAID1 SAS 7.2K 3.5 12G/s 6T)， 进行磁盘升级换成(RAID10)，目前瓶颈在io上面，内存和cpu都满是需求，把日志分开保存有效利用io
时间： 2024-10-10 04:13:22