目前我们的监控,可以发现消耗较高CPU的进程(阀值为3个CPU),通过监控我们可以找到消耗较高CPU的进程号;
通过进程号pid,我们在linux上可以通过top –H –p <pid>命令,显示该进程中每个线程的CPU资源消耗情况。
然后通过threadump命令,可以打印出某个应用JVM中某时刻所有线程的调用情况,通过线程号我们就可以对应找到线程调用的程序了。
另:
使用jrcmd命令也可以进行threadump和进程执行线程情况的查看,但该命令为jrockit 5.0 新带的命令,在wls81用户下,我们没有执行权限,但可以联系中间件帮忙执行。
1. 进入到jrockit/bin目录下
2. 打./jrcmd 回车,会看到当前所有在执行中的java进程的PID号
3. 打./jrcmd [PID] help 回车,会看到指定java进程支持的操作
4. 打./jrcmd [PID] print_threads > xxxFile,可以把当前java进程中的所有线程堆栈轨迹打印到指定文件中
示例:
[[email protected] wls]$ top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
228961 wls81 15 0 714m 637m 1804 S 755.7 0.2 383:24.53 java
[[email protected] wls]$ top -H -p 228961
top - 13:37:01 up 156 days, 16:11, 8 users, load average: 14.24, 13.82, 12.88
Tasks: 140 total, 23 running, 117 sleeping, 0 stopped, 0 zombie
Cpu(s): 25.6%us, 1.6%sy, 0.0%ni, 72.6%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 264120292k total, 217896772k used, 46223520k free, 4547804k buffers
Swap: 3145720k total, 464k used, 3145256k free, 153619224k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
229077
wls81 15 0 714m 637m 1804 R 32.6 0.2 7:11.79
ExecuteThread:
229093
wls81 15 0 714m 637m 1804 R 32.6 0.2 23:41.47
ExecuteThread:
229073
wls81 15 0 714m 637m 1804 R 32.2 0.2 2:24.30
ExecuteThread:
229076
wls81 15 0 714m 637m 1804 R 32.2 0.2 2:22.75
ExecuteThread:
229078
wls81 15 0 714m 637m 1804 R 32.2 0.2 7:21.38
ExecuteThread:
229082
wls81 15 0 714m 637m 1804 R 32.2 0.2 10:14.74
ExecuteThread:
229084
wls81 15 0 714m 637m 1804 R 32.2 0.2 8:43.72
ExecuteThread:
229085
wls81 15 0 714m 637m 1804 R 32.2 0.2 12:45.62
ExecuteThread:
229087
wls81 15 0 714m 637m 1804 R 32.2 0.2 19:11.79
ExecuteThread:
229089
wls81 15 0 714m 637m 1804 R 32.2 0.2 23:22.67
ExecuteThread:
229075
wls81 15 0 714m 637m 1804 R 31.9 0.2 5:06.52 ExecuteThread:
229088
wls81 15 0 714m 637m 1804 R 31.9 0.2 13:19.55
ExecuteThread:
229090
wls81 15 0 714m 637m 1804 R 31.9 0.2 23:53.55
ExecuteThread:
229092
wls81 15 0 714m 637m 1804 R 31.9 0.2 24:20.10
ExecuteThread:
229094
wls81 15 0 714m 637m 1804 R 31.9 0.2 33:25.41
ExecuteThread:
229095
wls81 15 0 714m 637m 1804 R 31.9 0.2 33:41.20
ExecuteThread:
229096
wls81 15 0 714m 637m 1804 R 31.9 0.2 36:22.72
ExecuteThread:
229081
wls81 15 0 714m 637m 1804 R 31.6 0.2 7:28.80
ExecuteThread:
229083
wls81 15 0 714m 637m 1804 R 31.6 0.2 11:05.48
ExecuteThread:
229086
wls81 15 0 714m 637m 1804 R 31.6 0.2 11:40.36
ExecuteThread:
229091
wls81 15 0 714m 637m 1804 R 31.6 0.2 17:53.92
ExecuteThread:
229097
wls81 15 0 714m 637m 1804 R 31.6 0.2 32:41.28
ExecuteThread:
229079 wls81 15 0 714m 637m 1804 R 31.2 0.2 7:33.44 ExecuteThread:
229080
wls81 15 0 714m 637m 1804 S 1.3 0.2 0:14.13
ExecuteThread:
228963
wls81 15 0 714m 637m 1804 S 0.7 0.2 0:13.44 (GC Main
Thread
229070
wls81 15 0 714m 637m 1804 S 0.7 0.2 0:05.97
ExecuteThread:
229664 wls81 15 0 714m 637m 1804 S 0.7 0.2 0:08.69 pool-1-thread-7
Threadump信息片段:
229083:
"ExecuteThread: ‘35‘ for queue: ‘weblogic.kernel.Default‘" id=44 idx=0xd4 tid=229083 prio=5 alive, daemon
at java/util/HashMap.get(HashMap.java:319)
at com/paic/abbs/biz/service/common/impl/HardCodeServiceImpl.getString(HardCodeServiceImpl.java:115)
at com/paic/abbs/biz/service/util/impl/UtilServicesImpl.isExpected(UtilServicesImpl.java:45)
at com/paic/abbs/biz/service/common/impl/AbbsInternetServiceImpl.policyIssuing(AbbsInternetServiceImpl.java:3156)
at com/paic/abbs/biz/service/common/impl/AbbsInternetServiceImpl.procRequest(AbbsInternetServiceImpl.java:218)
at com/paic/abbs/biz/service/common/impl/AbbsInternetServiceImpl.handleRequest(AbbsInternetServiceImpl.java:122)
at
com/paic/abbs/biz/service/common/impl/AbbsInternetService_gwxgqo_EOImpl.handleRequest(AbbsInternetService_gwxgqo_EOImpl.java:262)
at sun/reflect/GeneratedMethodAccessor287.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodA