以前做过一个性能测试项目,基于nginx+Hbase+Hadoop,第一次接触开源的东西,遇到各种问题,印象深的是Hbase和Hadoop的监控,当时也搜索到可以用开源的监控工具或写代码通过JMX取JVM的信息。在摸索的过程中发现一种更简单方便的办法,不用监控工具和写大量代码,直接用loadrunner脚本(该办法可能不专业,但是能用,可以让项目尽快进行,仅供参考)。
我们知道Hbase开放了60010和60030端口,Hadoop开放的50070端口,以web的方式查看master、region和hadoop的性能状况(比如查看master只需要在浏览器输入http://IP:60010/),可以查看进程的heap、request、queue使用情况,这些指标正是性能测试需要关注的。而这些信息是通过JMX传递,在浏览器中输入http://IP:port/jmx,会以json的格式返回此刻进程JVM的情况,效果如截图。
master信息
json信息
看到这些信息之后,我的第一反应就是这些返回的信息可用loadrunner的web_reg_save_param函数保存起来,然后写到txt里面。
所以决定Hbase和Hadoop的资源监控流程:
1、loadrunner定时获取json
2、讲关键指标保存到txt
3、用excel生成趋势图
loadrunner脚本:
Action() { int filename; int i; int len; char * item; //hbase region lr_think_time(10); lr_start_transaction("heap"); web_reg_save_param_ex( "ParamName=regionHeap", "LB= : ", "RB=\n },", SEARCH_FILTERS, "Scope=Body", "IgnoreRedirections=Yes", "RequestUrl=*/jmx*", LAST); web_url("jmx", "URL=http://IP:60030/jmx", "Resource=1", "RecContentType=application/json", "Referer=", "Snapshot=t1.inf", LAST); lr_error_message("region is %s", lr_eval_string("{regionHeap}")); //hadoop namenode web_reg_save_param_ex( "ParamName=nameNodeHeap", "LB= : ", "RB=\n },", SEARCH_FILTERS, "Scope=Body", "IgnoreRedirections=Yes", "RequestUrl=*/jmx*", LAST); web_url("jmx", "URL=http://IP:50070/jmx", "Resource=1", "RecContentType=application/json", "Referer=", "Snapshot=t1.inf", LAST); lr_error_message("nameNodeHeap is %s", lr_eval_string("{nameNodeHeap}")); //hbase master web_reg_save_param_ex( "ParamName=masterHeap", "LB= : ", "RB=\n },", SEARCH_FILTERS, "Scope=Body", "IgnoreRedirections=Yes", "RequestUrl=*/jmx*", LAST); web_url("jmx", "URL=http://IP:60010/jmx", "Resource=1", "RecContentType=application/json", "Referer=", "Snapshot=t1.inf", LAST); lr_error_message("master is %s", lr_eval_string("{regionHeap}")); lr_end_transaction("heap",LR_AUTO); //write text filename=fopen("E:\\cowen\\B2C\\monitor\\master.txt","a");//w-write,r-read,a-append fprintf(filename,"%s",lr_eval_string("{masterHeap}")); fprintf(filename,"%s","\n"); fclose(filename); filename=fopen("E:\\cowen\\B2C\\monitor\\region.txt","a");//w-write,r-read,a-append fprintf(filename,"%s",lr_eval_string("{regionHeap}")); fprintf(filename,"%s","\n"); fclose(filename); filename=fopen("E:\\cowen\\B2C\\monitor\\namenode.txt","a");//w-write,r-read,a-append fprintf(filename,"%s",lr_eval_string("{nameNodeHeap}")); fprintf(filename,"%s","\n"); fclose(filename); return 0; }
效果图
时间: 2024-10-25 15:24:38