Apply Bug10010310 On Oracle RAC 10.2.0.5

9月24日数据库上频繁出现如下错误

Errors in file /u04/admin/njord/udump/njord_ora_25895.trc:

ORA-27300: OS system dependent operation:invalid_process_id failed with status: 0

ORA-27301: OS failure message: Error 0

ORA-27302: failure occurred at: skgpalive1

该错误将影响客户端的连接，经过查阅资料证明是Oracle 10.2.0.5的bug，以下是mos上面的说明

ORA-27300 ORA-27301 ORA-27302:
failure occurred at: skgpalive1 (文档 ID 1252265.1)		 转到底部  

In this Document

Symptoms
  Changes
  Cause
  Solution
  References

APPLIES TO:

Oracle Database – Enterprise Edition – Version 10.2.0.5 and later

Information in this document applies to any platform.

SYMPTOMS

Database alert_<ORACLE_SID>.log shows following error and keeps running:

Errors in file /u04/admin/njord/udump/njord_ora_25895.trc:

ORA-27300: OS system dependent operation:invalid_process_id failed with status: 0

ORA-27301: OS failure message: Error 0

ORA-27302: failure occurred at: skgpalive1

CHANGES

Applied 10.2.0.5 patch set.

CAUSE

This is caused by bug 10010310

SOLUTION

bug 10010310 is fixed in 11.2.0.3, 12.1 and above. Please check patch
10010310 for existing interim patches.

REFERENCES

BUG:10207340 - ORA-27302 SKGPALIVE1 AFTER UPGRADE TO 10.2.0.5.0

BUG:10010310 - ORA-27300: INVALID_PROCESS_ID FAILED WITH STATUS: 0; ORA-27302: SKGPALIVE1

BUG:10154137 - ORA-27300: OS SYSTEM DEPENDENT OPERATION:INVALID_PROCESS_ID FAILED WITH STATUS 0
 

生产环境：

OS：IBM AIX power 7

Oracle Version:10.2.0.5

Storage:Raw device

Opatch Version:10.2.0.4.9

测试环境：

OS：redhat 5.8 x86-64

Oracle Version:10.2.0.5

Storage:Raw device

Opatch Version:10.2.0.4.9

操作步骤：

1、下载补丁、上传补丁、解压补丁

2、配置环境变量

3、关闭oracle数据库

4、停止crs

5、在一节点应用补丁

6、在二节点应用补丁

7、start crs

8、开启数据库

在My Support Oracle 上下载Bug 10010310

[[email protected] ~]$ unzip p10010310_10205_Linux-x86-64.zip

配置opatch环境变量：

export OPatch=/u01/app/oracle/product/10.2.0/db_1/OPatch

export PATH=$HOME/bin:$ORACLE_HOME/bin:$OPatch:/sbin:$PATH

opatch 版本

[[email protected] 10010310]$ opatch version

Invoking OPatch 10.2.0.4.9

OPatch Version: 10.2.0.4.9

OPatch succeeded.

停止数据库：

srvctl stop instance -d zhdb -i "zhdb1,zhdb2" -o immediate

停止 crs:

crsctl stop crs

一节点：

[[email protected] 10010310]$ opatch apply

Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9

Copyright (c) 2009, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/10.2.0/db_1

Central Inventory : /u01/app/oracle/oraInventory

from           : /etc/oraInst.loc

OPatch version    : 10.2.0.4.9

OUI version       : 10.2.0.5.0

OUI location      : /u01/app/oracle/product/10.2.0/db_1/oui

Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-09-29_05-05-15AM.log

Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

ApplySession applying interim patch ‘10010310‘ to OH ‘/u01/app/oracle/product/10.2.0/db_1‘

Running prerequisite checks…

OPatch detected the node list and the local node from the inventory.  OPatch will patch the local system then propagate the patch to the remote nodes.

This node is part of an Oracle Real Application Cluster.

Remote nodes: ‘rac2‘

Local node: ‘rac1‘

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.

(Oracle Home = ‘/u01/app/oracle/product/10.2.0/db_1‘)

Is the local system ready for patching? [y|n]

y

User Responded with: Y

Backing up files and inventory (not for auto-rollback) for the Oracle Home

Backing up files affected by the patch ‘10010310‘ for restore. This might take a while…

Backing up files affected by the patch ‘10010310‘ for rollback. This might take a while…

Patching component oracle.rdbms, 10.2.0.5.0…

Updating archive file "/u01/app/oracle/product/10.2.0/db_1/lib/libserver10.a"  with "lib/libserver10.a/ksu.o"

Running make for target ioracle

ApplySession adding interim patch ‘10010310‘ to inventory

Verifying the update…

Inventory check OK: Patch ID 10010310 is registered in Oracle Home inventory with proper meta-data.

Files check OK: Files from Patch ID 10010310 are present in Oracle Home.

The local system has been patched.  You can restart Oracle instances on it.

Patching in rolling mode.

The node ‘rac2‘ will be patched next.

Please shutdown Oracle instances running out of this ORACLE_HOME on ‘rac2‘.

(Oracle Home = ‘/u01/app/oracle/product/10.2.0/db_1‘)

Is the node ready for patching? [y|n]

y

User Responded with: Y

Updating nodes ‘rac2‘

Apply-related files are:

FP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt"

DP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt"

MP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt"

RC = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remote_cmds.txt"

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt"
with actual path.

Propagating files to remote nodes…

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt"
with actual path.

Propagating directories to remote nodes…

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt"
with actual path.

Running command on remote node ‘rac2‘:

cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1 || echo REMOTE_MAKE_FAILED::>&2

The node ‘rac2‘ has been patched.  You can restart Oracle instances on it.

There were relinks on remote nodes.  Remember to check the binary size and timestamp on the nodes ‘rac2‘ .

The following make commands were invoked on remote nodes:

‘cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1

‘

OPatch succeeded.

[[email protected] 10010310]$ opatch lsinventory

Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9

Copyright (c) 2009, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/10.2.0/db_1

Central Inventory : /u01/app/oracle/oraInventory

from           : /etc/oraInst.loc

OPatch version    : 10.2.0.4.9

OUI version       : 10.2.0.5.0

OUI location      : /u01/app/oracle/product/10.2.0/db_1/oui

Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-09-29_05-08-40AM.log

Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2014-09-29_05-08-40AM.txt

——————————————————————————–

Installed Top-level Products (2):

Oracle Database 10g                                                  10.2.0.1.0

Oracle Database 10g Release 2 Patch Set 4                            10.2.0.5.0

There are 2 products installed in this Oracle Home.

Interim patches (1) :

Patch  10010310     : applied on Mon Sep 29 05:06:10 EDT 2014

Unique Patch ID:  13106370

Created on 3 Nov 2010, 02:11:57 hrs PST8PDT

Bugs fixed:

10010310

Rac system comprising of multiple nodes

Local node = rac1

Remote node = rac2

——————————————————————————–

OPatch succeeded.

[[email protected] 10010310]$

节点二：

[[email protected] 10010310]$ opatch  apply

Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9

Copyright (c) 2009, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/10.2.0/db_1

Central Inventory : /u01/app/oracle/oraInventory

from           : /etc/oraInst.loc

OPatch version    : 10.2.0.4.9

OUI version       : 10.2.0.5.0

OUI location      : /u01/app/oracle/product/10.2.0/db_1/oui

Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-09-29_05-10-16AM.log

Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

ApplySession applying interim patch ‘10010310‘ to OH ‘/u01/app/oracle/product/10.2.0/db_1‘

Interim patch 10010310 is a superset of the patch(es) [  10010310 ] in the Oracle Home

OPatch will rollback the subset patches and apply the given patch.

Running prerequisite checks…

OPatch detected the node list and the local node from the inventory.  OPatch will patch the local system then propagate the patch to the remote nodes.

This node is part of an Oracle Real Application Cluster.

Remote nodes: ‘rac1‘

Local node: ‘rac2‘

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.

(Oracle Home = ‘/u01/app/oracle/product/10.2.0/db_1‘)

Is the local system ready for patching? [y|n]

y

User Responded with: Y

Backing up files and inventory (not for auto-rollback) for the Oracle Home

Backing up files affected by the patch ‘10010310‘ for restore. This might take a while…

Backing up files affected by the patch ‘10010310‘ for restore. This might take a while…

ApplySession rolling back interim patch ‘10010310‘ from OH ‘/u01/app/oracle/product/10.2.0/db_1‘

Patching component oracle.rdbms, 10.2.0.5.0…

Updating archive file "/u01/app/oracle/product/10.2.0/db_1/lib/libserver10.a"  with "lib/libserver10.a/ksu.o"

RollbackSession removing interim patch ‘10010310‘ from inventory

OPatch back to application of the patch ‘10010310‘ after auto-rollback.

Backing up files affected by the patch ‘10010310‘ for rollback. This might take a while…

Patching component oracle.rdbms, 10.2.0.5.0…

Updating archive file "/u01/app/oracle/product/10.2.0/db_1/lib/libserver10.a"  with "lib/libserver10.a/ksu.o"

Running make for target ioracle

ApplySession adding interim patch ‘10010310‘ to inventory

Verifying the update…

Inventory check OK: Patch ID 10010310 is registered in Oracle Home inventory with proper meta-data.

Files check OK: Files from Patch ID 10010310 are present in Oracle Home.

The local system has been patched.  You can restart Oracle instances on it.

Patching in rolling mode.

The node ‘rac1‘ will be patched next.

Please shutdown Oracle instances running out of this ORACLE_HOME on ‘rac1‘.

(Oracle Home = ‘/u01/app/oracle/product/10.2.0/db_1‘)

Is the node ready for patching? [y|n]

y

User Responded with: Y

Updating nodes ‘rac1‘

Apply-related files are:

FP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt"

DP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt"

MP = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt"

RC = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remote_cmds.txt"

Auto-Rollback-related files are:

FR = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remove_files.txt"

DR = "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remove_dirs.txt"

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remove_dirs.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/remove_dirs.txt"
with actual path.

Removing directories on remote nodes…

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_files.txt"
with actual path.

Propagating files to remote nodes…

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/copy_dirs.txt"
with actual path.

Propagating directories to remote nodes…

Instantiating the file "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt.instantiated" by replacing $ORACLE_HOME in "/u01/app/oracle/product/10.2.0/db_1/.patch_storage/10010310_Nov_3_2010_02_11_57/rac/make_cmds.txt"
with actual path.

Running command on remote node ‘rac1‘:

cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1 || echo REMOTE_MAKE_FAILED::>&2

The node ‘rac1‘ has been patched.  You can restart Oracle instances on it.

There were relinks on remote nodes.  Remember to check the binary size and timestamp on the nodes ‘rac1‘ .

The following make commands were invoked on remote nodes:

‘cd /u01/app/oracle/product/10.2.0/db_1/rdbms/lib; /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1

‘

——————————————————————————–

The following warnings have occurred during OPatch execution:

1) OUI-67620:Interim patch 10010310 is a superset of the patch(es) [  10010310 ] in the Oracle Home

——————————————————————————–

OPatch Session completed with warnings.

OPatch completed with warnings.

[[email protected] 10010310]$

[[email protected] 10010310]$

[[email protected] 10010310]$ opatch lsinventory

Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9

Copyright (c) 2009, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/10.2.0/db_1

Central Inventory : /u01/app/oracle/oraInventory

from           : /etc/oraInst.loc

OPatch version    : 10.2.0.4.9

OUI version       : 10.2.0.5.0

OUI location      : /u01/app/oracle/product/10.2.0/db_1/oui

Log file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-09-29_05-13-18AM.log

Patch history file: /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2014-09-29_05-13-18AM.txt

——————————————————————————–

Installed Top-level Products (2):

Oracle Database 10g                                                  10.2.0.1.0

Oracle Database 10g Release 2 Patch Set 4                            10.2.0.5.0

There are 2 products installed in this Oracle Home.

Interim patches (1) :

Patch  10010310     : applied on Mon Sep 29 05:11:58 EDT 2014

Unique Patch ID:  13106370

Created on 3 Nov 2010, 02:11:57 hrs PST8PDT

Bugs fixed:

10010310

Rac system comprising of multiple nodes

Local node = rac2

Remote node = rac1

——————————————————————————–

OPatch succeeded.

[[email protected] 10010310]$

[[email protected] oracle]# crsctl start crs

Attempting to start CRS stack

The CRS stack will be started shortly

[[email protected] oracle]# crs_stat -t

CRS-0184: Cannot communicate with the CRS daemon.

[[email protected] oracle]# crs_stat -t

Name           Type           Target    State     Host

————————————————————

ora….C1.lsnr application    ONLINE    ONLINE    rac1

ora.rac1.gsd   application    ONLINE    ONLINE    rac1

ora.rac1.ons   application    ONLINE    ONLINE    rac1

ora.rac1.vip   application    ONLINE    ONLINE    rac1

ora….C2.lsnr application    ONLINE    ONLINE    rac2

ora.rac2.gsd   application    ONLINE    ONLINE    rac2

ora.rac2.ons   application    ONLINE    ONLINE    rac2

ora.rac2.vip   application    ONLINE    ONLINE    rac2

ora.zhdb.db    application    ONLINE    ONLINE    rac1

ora….b1.inst application    ONLINE    ONLINE    rac1

ora….b2.inst application    OFFLINE   OFFLINE

测试环境需要宕机时间大概为10分钟。生产环境保守估计30分钟

时间： 2024-10-01 12:54:26

Apply Bug10010310 On Oracle RAC 10.2.0.5的相关文章

Oracle RAC 10.2.0.5 Install For redhat 5.8(use raw device)

因为客户数据库需要打补丁,所以安装相似测试环境.需要先安装10.2.0.1 然后再升级到10.2.0.5.因为我有很多篇关于安装的文章,所以这篇只说关键步骤,以及遇见的bug安装错误 注意:我已经在redhat 6.3测试过,不能完成安装,主要是在跑root.sh脚本的时候无法通过.换成redhat 5.8不会出现这种问题. Steps: 1.防火墙and selinux 2.kernel parameter 3.source limit 4.create user and group 5.建立

Oracle 10g 10.2.0.1 在Oracle Linux 5.4 32Bit RAC安装手冊（一抹曦阳）

Oracle 10g 10.2.0.1 在Oracle Linux 5.4 32Bit RAC安装手冊(一抹曦阳).pdf下载地址 ,step by step http://download.csdn.net/detail/rlhua/7699223

【Oracle】RAC 10.2.0.1升级10.2.0.5

环境: OS:OEL5.6 RAC:10.2.0.1.0 相关环境变量: CRS_HOME /u01/app/oracle/product/10.2.0/db_1 ORACLE_HOME   /u01/app/oracle/product/10.2.0/db_2 crs_stop -all关闭所有资源 [[email protected] ~]$ crs_stat -t Name           Type           Target    State     Host --------

Oracle 10g 10.2.0.1 在Oracle Linux 5.4 32Bit RAC安装手册（一抹曦阳）

Oracle 10g 10.2.0.1 在Oracle Linux 5.4 32Bit RAC安装手册(一抹曦阳).pdf下载地址 ,step by step http://download.csdn.net/detail/rlhua/7699223 Oracle 10g 10.2.0.1 在Oracle Linux 5.4 32Bit RAC安装手册(一抹曦阳)

Oracle 10g(10.2.0.4)升级到10.2.0.5.19

一.将数据库版本从10.2.0.4 升级到 10.2.0.5,再升级到10.2.0.5.19 (1) 备份等过程略过,一个老库的升级过程,记录之.   (2) 一致性关闭数据库及监听 sqlplus / as sysdba;   shutdown immediate    lsnrctl stop 二.升级数据库软件 1,解压p8202632_10205_Linux-x86-64.zip   2,直接采用安装方式安装,覆盖原安装目录 xhost+   su - oracle    cd Disk

RAC 10.2.0.5，客户端登陆间断遭遇ORA-12545

试验环境: 服务端:OEL 5.7 + Oracle 10.2.0.5 RAC 客户端:Windows 7 + Oracle 11.2.0.1 Client 1.客户端登陆间断遭遇ORA-12545,现象如下: C:\Users\xiaoyu>sqlplus system/[email protected]192.168.1.171/jy.oracle.com SQL*Plus: Release 11.2.0.1.0 Production on 星期二 5月 20 19:43:52 2014 C

oracle database 10.2.0.4 升级到 10.2.0.5

某发票开发测试库升级      升级前准备,此次升级只是很对测试环境数据库升级,所以没有事先一个月来获取系统,数据库的统计信息,机器性能比对 为了加快升级只是清理了以下信息 01.截断SYS.AUD$基表: SQL>TRUNCATE TABLE SYS.AUD$; 02.清理DBA回收站: SQL>purge DBA_RECYCLEBIN; 1.升级开始,升级前首先断开测试环境的中间件应用 查看本机的ORACLE_HOME [[email protected]_10 ~]$ echo $ORA

Oracle Rac 11.2.0.3迁移OCR和VOTEDISK

环境:AIX7.1+Oracle Rac 11.2.0.3 迁移描述:今天在装Oracle Rac的时候,错误的将500G的数据盘用作OCRDG了,遂后续比较麻烦,只能讲ocr和votedisk迁移到新建的OCRDG上,并把DATADG删除并格式化该盘.(OCRDG为normal 冗余) 操作如下: [email protected]:/home/grid>/oraapp/grid/gridhome/bin/ocrcheck Status of Oracle Cluster Registry i

OEM无法启动：oracle/product/10.2.0/dbhome_1/oc4j/j2ee/OC4J_DBConsole_localhost.localdomain_orcl not found.

配置OEM环境变量 ORACLE_UNQNAME=$ORACLE_SID 要想打开OEM的web管理程序,必须先启动Database listener,再启动Datebase Control端口监控程序,但是碰到下面的情况          问题:    [[email protected] OC4J_DBConsole]$ emctl start dbconsole    TZ set to PRC    OC4J Configuration issue. /u01/app/oracle/pr