来源于:
ORAAGENT Managed by CRSD Consuming Higher CPU and Leaking Memory (文档 ID 2021627.1)
APPLIES TO:
Oracle Database - Enterprise Edition - Version 11.2.0.4 and later
Information in this document applies to any platform.
This document has been generated related to NCC analysis
SYMPTOMS
o After applying 11.2.0.4.5GIPSU, oraagent started to leak memory and use high cpu.
o ‘ps‘ and ‘svmon‘ shows the memory used by the process is increasing over the time. The CPU usage is much higher than other processes.
zzz ***Wed May 20 11:00:25 CDT 2015
USER PID PPID PRI %CPU %MEM VSZ RSS WCHAN S ELAPSED TIME COMMAND
oracle 13041828 1 60 1.6 9.0 3972632 4278256 * A 5-08:39:08 1-08:26:35 /opt/oracrs/gridsw/grid11204/bin/oraagent.bin
oracle 12124326 1 60 0.1 1.0 73824 379592 * A 5-08:40:28 01:02:51 /opt/oracrs/gridsw/grid11204/bin/oraagent.bin
....
zzz ***Wed May 20 11:22:27 CDT 2015
USER PID PPID PRI %CPU %MEM VSZ RSS WCHAN S ELAPSED TIME COMMAND
oracle 13041828 1 60 1.6 6.0 2552380 2858060 * A 5-09:01:10 1-08:34:41 /opt/oracrs/gridsw/grid11204/bin/oraagent.bin
oracle 12124326 1 60 0.1 1.0 73824 379592 * A 5-09:02:30 01:03:02 /opt/oracrs/gridsw/grid11204/bin/oraagent.bin
note that oraagent with PID 13041828 (from CRSD) is growing in memory and the cumulative CPU usage is considerable higher 1-08:34:41, compared with the other oraagent with PID 12124326 (from OHASD).
svmon output:
Wed May 20 00:01:08 CDT 2015
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
13041828 oraagent.bin 1659692 83538 0 1582356 Y Y Y
Wed May 20 11:32:20 CDT 2015
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd 16MB
13041828 oraagent.bin 1781494 83538 0 1703011 Y Y Y
o ‘svmon‘ shows the additional memory ‘work text data BSS heap‘ segments created in oraagent.
o The log for oraagent shows the following messages constantly:
InstConnection:connectInt connected In DedicatedThread::generateLocalListener DbAgent::DedicatedThread::generateLocalListener getLocalVipAddress CrsCmd::ClscrsCmdData::stat entity 1 statflag 1 useFilter 0 CrsCmd::ClscrsCmdData::destroy CrsCmd::ClscrsCmdData::stat entity 1 statflag 1 useFilter 0 CrsCmd::ClscrsCmdData::destroy CrsCmd::ClscrsCmdData::stat entity 1 statflag 1 useFilter 0 CrsCmd::ClscrsCmdData::destroy Local VIP address is lsidp4822.co.com CrsCmd::destroy vipVector[0]=lsidp4822.co.com m_crsHome = /opt/oracrs/gridsw/grid11204 DbAgent::DedicatedThread::generateLocalListener getResNameByTypes CrsCmd::ClscrsCmdData::destroy DbAgent::DedicatedThread::generateLocalListener getAttrValueFromResource CrsCmd::ClscrsCmdData::stat entity 5 statflag 48 useFilter 0 CrsCmd::ClscrsCmdData::destroy CrsCmd::ClscrsCmdData::stat entity 1 statflag 33 useFilter 0 CrsCmd::ClscrsCmdData::destroy ORACLE_HOME = , START_DEPENDENCIES = hard(type:ora.cluster_vip_net1.type) pullup(type:ora.cluster_vip_net1.type) for ora.LISTENER.lsnr CrsCmd::destroy
o The log information shows oraagent is checking the local listener, but also reveals ORACLE_HOME is not set.
CHANGES
Customer updated from 11.2.0.3 to 11.2.0.4 and applied the GIPSU 11.2.0.4.5
CAUSE
Before updating to 11.2.0.4, customer had modified the listener resource, removing the attribute ORACLE_HOME:
crsctl modify res ora.LISTENER.lsnr -attr "ORACLE_HOME= "
SOLUTION
Oracle Clusterware resources ‘ora.* ‘ must not be modified, this is an unsupported operation.
For this particular situation, add back the ORACLE_HOME, running:
crsctl modify resource ora.LISTENER.lsnr -attr "ORACLE_HOME=%CRS_HOME%"
After setting back the ORACLE_HOME, oraagent stopped using higher memory and cpu.