分类目录归档:ORACLE 19C

19c 打RU patch遇到oui-patch.xml (Permission denied)问题

有一段时间没有做实施的活,今天安装一套19c rac,并且计划安装Patch 32545008 – GI Release Update 19.11.0.0.210420,在安装过程中遇到oui-patch.xml问题,记录下来供参考:
1. OPATCHAUTO-72088

[root@jbsbdb1 ~]# opatchauto apply /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 14:20:24 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_02-20-29PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_02-20-57PM.log
The id for this session is N2PG

Wrong OPatch software installed in following homes:
Host:jbsbdb2, Home:/u01/app/oracle/product/19.0/db_1

Host:jbsbdb2, Home:/u01/app/19.0/grid

OPATCHAUTO-72088: OPatch version check failed.
OPATCHAUTO-72088: OPatch software version in homes selected for patching are different.
OPATCHAUTO-72088: Please install same OPatch software in all homes.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 14:21:15 2021
Time taken to complete the session 0 minute, 51 seconds

 opatchauto failed with error code 42

故障原因,只是升级了节点1的grid和oracle的opatch,把节点2的opatch也升级之后该问题解决

2. oui-patch.xml (Permission denied)文件处理

[root@jbsbdb2 soft]# opatchauto apply /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 14:52:29 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_02-52-32PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_02-52-58PM.log
The id for this session is T6ST

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1
Patch applicability verified successfully on home /u01/app/19.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Executing patch validation checks on home /u01/app/19.0/grid
Patch validation checks successfully completed on home /u01/app/19.0/grid


Executing patch validation checks on home /u01/app/oracle/product/19.0/db_1
Patch validation checks successfully completed on home /u01/app/oracle/product/19.0/db_1


Verifying SQL patch applicability on home /u01/app/oracle/product/19.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Preparing to bring down database service on home /u01/app/oracle/product/19.0/db_1
Successfully prepared home /u01/app/oracle/product/19.0/db_1 to bring down database service


Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/19.0/grid
Prepatch operation log file location: 
/u01/app/grid/crsdata/jbsbdb2/crsconfig/crs_prepatch_apply_inplace_jbsbdb2_2021-04-28_02-54-34PM.log
CRS service brought down successfully on home /u01/app/19.0/grid


Performing prepatch operation on home /u01/app/oracle/product/19.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Start applying binary patch on home /u01/app/oracle/product/19.0/db_1
Failed while applying binary patches on home /u01/app/oracle/product/19.0/db_1

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : jbsbdb2->/u01/app/oracle/product/19.0/db_1 Type[rac]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19.0/db_1, host: jbsbdb2.
Command failed:  /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto 
 apply /u01/soft/32545008 -oh /u01/app/oracle/product/19.0/db_1 -target_type rac_database
 -binary -invPtrLoc /u01/app/19.0/grid/oraInst.loc -jre /u01/app/19.0/grid/OPatch/jre -persistresult
/u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_jbsbdb2_rac_2.ser -analyzedresult
 /u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_jbsbdb2_rac_2.ser
Command failure output: 
==Following patches FAILED in apply:

Patch: /u01/soft/32545008/32545013
Log: /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-28_15-00-49PM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: 
ApplySession failed in system modification phase... 
'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: 
java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)' 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 15:05:29 2021
Time taken to complete the session 13 minutes, 0 second

 opatchauto failed with error code 42

在节点1打patch成功之后,对节点2进行打patch,遇到类似:ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException:
java.io.FileNotFoundException: /u01/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)问题,通过查询mos发现类似文章opatchauto apply Results java.io.FileNotFoundException: /ContentsXML/oui-patch.xml (Permission denied) Error in Non-OUI Nodes (Doc ID 2582139.1),确认是bug 29859410 在20.1版本中修复.检查系统中对应oui-patch.xml信息.

[root@jbsbdb2 soft]# ls -l /u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-r----- 1 grid oinstall 174 Apr 28 14:04 /u01/app/oraInventory/ContentsXML/oui-patch.xml

确实文件权限不对,对其授权处理

[root@jbsbdb2 soft]# chmod 660 /u01/app/oraInventory/ContentsXML/oui-patch.xml
[root@jbsbdb2 soft]# ls -l /u01/app/oraInventory/ContentsXML/oui-patch.xml
-rw-rw---- 1 grid oinstall 174 Apr 28 14:04 /u01/app/oraInventory/ContentsXML/oui-patch.xml

处理oui-patch.xml文件异常之后,尝试回滚操作

[root@jbsbdb2 soft]# opatchauto rollback /u01/soft/32545008

OPatchauto session is initiated at Wed Apr 28 15:13:58 2021

System initialization log file is /u01/app/19.0/grid/cfgtoollogs/opatchautodb/systemconfig2021-04-28_03-14-00PM.log.

Session log file is /u01/app/19.0/grid/cfgtoollogs/opatchauto/opatchauto2021-04-28_03-14-28PM.log
The id for this session is KFYI

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.0/grid
Patch applicability verified successfully on home /u01/app/19.0/grid

Patch applicability verification failed on home /u01/app/oracle/product/19.0/db_1

Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : jbsbdb2->/u01/app/oracle/product/19.0/db_1 Type[rac]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/oracle/product/19.0/db_1, host: jbsbdb2.
Command failed:  /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto  rollback
 /u01/soft/32545008 -oh /u01/app/oracle/product/19.0/db_1 
-target_type rac_database -binary -invPtrLoc /u01/app/19.0/grid/oraInst.loc 
-jre /u01/app/19.0/grid/OPatch/jre -persistresult 
/u01/app/oracle/product/19.0/db_1/opatchautocfg/db/sessioninfo/sessionresult_analyze_jbsbdb2_rac_2.ser
 -analyze -online -prepare_home
Command failure output: 
==Following patches FAILED in analysis for rollback:

Patch: /u01/soft/32545008/32579761
Log: 
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: 
Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/32545013 is corrupted. PatchObject constructor: 
Input file "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/actions" 
or "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/inventory" does not exist.


Patch: /u01/soft/32545008/32545013
Log: 
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: 
Unable to create patchObject
Possible causes are:
   ORACLE_HOME/inventory/oneoffs/32545013 is corrupted. PatchObject constructor:
 Input file "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/actions" 
or "/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013/etc/config/inventory" does not exist. 

After fixing the cause of failure Run opatchauto resume

]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.

OPatchauto session completed at Wed Apr 28 15:14:50 2021
Time taken to complete the session 0 minute, 53 seconds

 opatchauto failed with error code 42

检查发现/u01/app/oracle/product/19.0/db_1/inventory/oneoffs/32545013确实不存在,从节点1把该文件夹tar过来,然后再次回滚

[root@jbsbdb2 ~]# /u01/app/oracle/product/19.0/db_1/OPatch/opatchauto rollback /u01/soft/32545008 -oh 
>/u01/app/oracle/product/19.0/db_1

OPatchauto session is initiated at Wed Apr 28 16:12:33 2021

System initialization log file is 
/u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchautodb/systemconfig2021-04-28_04-12-36PM.log.

Session log file is /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/opatchauto2021-04-28_04-12-53PM.log
The id for this session is JRS3

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Executing patch validation checks on home /u01/app/oracle/product/19.0/db_1
Patch validation checks successfully completed on home /u01/app/oracle/product/19.0/db_1


Verifying SQL patch applicability on home /u01/app/oracle/product/19.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.0/db_1


Preparing to bring down database service on home /u01/app/oracle/product/19.0/db_1
Successfully prepared home /u01/app/oracle/product/19.0/db_1 to bring down database service


Bringing down database service on home /u01/app/oracle/product/19.0/db_1
Following database(s) and/or service(s) are stopped and will be restarted later during the session: racdb
Database service successfully brought down on home /u01/app/oracle/product/19.0/db_1


Performing prepatch operation on home /u01/app/oracle/product/19.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Start rolling back binary patch on home /u01/app/oracle/product/19.0/db_1
Binary patch rolled back successfully on home /u01/app/oracle/product/19.0/db_1


Performing postpatch operation on home /u01/app/oracle/product/19.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.0/db_1


Starting database service on home /u01/app/oracle/product/19.0/db_1
Database service successfully started on home /u01/app/oracle/product/19.0/db_1


Preparing home /u01/app/oracle/product/19.0/db_1 after database service restarted
No step execution required.........
 

Trying to roll back SQL patch on home /u01/app/oracle/product/19.0/db_1
SQL patch rolled back successfully on home /u01/app/oracle/product/19.0/db_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:jbsbdb2
RAC Home:/u01/app/oracle/product/19.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /u01/soft/32545008/32576499
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32585572
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32584670
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /u01/soft/32545008/32579761
Reason: This Patch does not exist in the home, it cannot be rolled back.


==Following patches were SUCCESSFULLY rolled back:

Patch: /u01/soft/32545008/32545013
Log: /u01/app/oracle/product/19.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-04-28_16-15-05PM_1.log

回滚成功,直接使用opatchauto apply /u01/soft/32545008打节点2成功
3. RU安装成功结果

[grid@jbsbdb2 ~]$ opatch lspatches
32585572;DBWLM RELEASE UPDATE 19.0.0.0.0 (32585572)
32584670;TOMCAT RELEASE UPDATE 19.0.0.0.0 (32584670)
32579761;OCW RELEASE UPDATE 19.11.0.0.0 (32579761)
32576499;ACFS RELEASE UPDATE 19.11.0.0.0 (32576499)
32545013;Database Release Update : 19.11.0.0.210420 (32545013)

OPatch succeeded.

[oracle@jbsbdb2 ~]$ opatch lspatches
32579761;OCW RELEASE UPDATE 19.11.0.0.0 (32579761)
32545013;Database Release Update : 19.11.0.0.210420 (32545013)

OPatch succeeded.
[grid@jbsbdb2 ~]$ crsctl query crs softwarepatch
Oracle Clusterware patch level on node jbsbdb2 is [3331580692].
[grid@jbsbdb2 ~]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. 
The cluster upgrade state is [NORMAL]. The cluster active patch level is [3331580692].

SQL> select PATCH_ID,DESCRIPTION from dba_registry_sqlpatch;

  PATCH_ID
----------
DESCRIPTION
--------------------------------------------------------------------------------
  29517242
Database Release Update : 19.3.0.0.190416 (29517242)

  32545013
Database Release Update : 19.11.0.0.210420 (32545013)

在打patch之前已经知晓该问题,比较粗心的把oui-patch.xml文件从节点1scp拷贝到节点2,没有确认oui-patch.xml权限(拷贝过来是640)从而引起后续很多麻烦

发表在 ORACLE 19C, Oracle安装升级 | 标签为 , | 评论关闭

无法启动此程序,因为计算机中丢失api-ms-win-crt-runtime-l1-1-0.dll

在新安装的win操作操作系统中安装oracle 19.3版本数据库,结果不幸遭遇到如下错误
api-mi-win-crt-runtime1-1-0


查询oracle 19c对于win操作系统认证列表
19c-win-certification

本机操作系统为win 2012,在此认证列表中
2012
通过上述认证查询以及是新安装的原版系统,很可能是由于19c安装特殊之处导致,通过查询mos,确认是由于19c的数据库在安装之时perl需要VS 2017的运行库进行一些操作的依赖,因此安装Microsoft Visual C++ Redistributable for Visual Studio 2017或者更高版本即可.下载link:https://aka.ms/vs/16/release/vc_redist.x64.exe
具体参考MOS:Oracle DB 19C Install fails with the error – “THE PROCEDURE ENTRY POINT _REGISTER_ONEXIT_FUNCTION COULD NOT BE LOCATED IN THE DYNAMIC LINK LIBRARY \PERL\BIN\PERL.EXE” (Doc ID 2658357.1)

发表在 ORACLE 19C, Oracle安装升级 | 标签为 , | 评论关闭

Oracle 19c故障恢复

有客户找到我们,他们的oracle 19c数据库由于异常断电,导致启动异常,经过一系列恢复之后,依旧无法解决问题,请求我们给予支持.通过我们的Oracle数据库异常恢复检查脚本(Oracle Database Recovery Check),获取数据库当前信息如下:
数据库版本为19C并且安装了19.5.0.0.191015 (30125133)补丁
20200310220453
20200310220748


数据库使用pdb
20200310220610

数据库启动成功后,一会就crash掉

2020-03-10T01:44:41.018032+08:00
Pluggable database RACBAK opened read write
2020-03-10T01:44:41.018996+08:00
Pluggable database RAC opened read write
2020-03-10T01:44:51.244050+08:00
Completed: ALTER PLUGGABLE DATABASE ALL OPEN
Starting background process CJQ0
Completed: ALTER DATABASE OPEN
2020-03-10T01:44:51.317085+08:00
CJQ0 started with pid=224, OS id=32581 
2020-03-10T01:44:56.067043+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j001_32588.trc  (incident=1095281) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [27733], [27754], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095281/XFF_j001_32588_i1095281.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:44:56.073112+08:00
RAC(4):*****************************************************************
RAC(4):An internal routine has requested a dump of selected redo.
RAC(4):This usually happens following a specific internal error, when
RAC(4):analysis of the redo logs will help Oracle Support with the
RAC(4):diagnosis.
RAC(4):It is recommended that you retain all the redo logs generated (by
RAC(4):all the instances) during the past 12 hours, in case additional
RAC(4):redo dumps are required to help with the diagnosis.
RAC(4):*****************************************************************
2020-03-10T01:44:56.079228+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j002_32590.trc  (incident=1095289) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [2633], [2638], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095289/XFF_j002_32590_i1095289.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:44:56.085068+08:00
RAC(4):*****************************************************************
RAC(4):An internal routine has requested a dump of selected redo.
RAC(4):This usually happens following a specific internal error, when
RAC(4):analysis of the redo logs will help Oracle Support with the
RAC(4):diagnosis.
RAC(4):It is recommended that you retain all the redo logs generated (by
RAC(4):all the instances) during the past 12 hours, in case additional
RAC(4):redo dumps are required to help with the diagnosis.
RAC(4):*****************************************************************
2020-03-10T01:44:56.115765+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_j004_32594.trc  (incident=1095305) (PDBNAME=RAC):
ORA-00600: internal error code, arguments: [4193], [63532], [63537], [], [], [], [], [], [], [], [], []
RAC(4):Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1095305/XFF_j004_32594_i1095305.trc
RAC(4):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2020-03-10T01:46:48.202213+08:00
RAC(4):Recovery of Online Redo Log: Thread 1 Group 2 Seq 2 Reading mem 0
RAC(4):  Mem# 0: /opt/oracle/oradata/XFF/redo02.log
RAC(4):Block recovery completed at rba 0.0.0, scn 0x0000000d3675e48e
RAC(4):DDE: Problem Key 'ORA 600 [4193]' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes
2020-03-10T01:46:48.384040+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_clmn_31741.trc:
ORA-00600: internal error code, arguments: [4193], [27733], [27754], [], [], [], [], [], [], [], [], []
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_clmn_31741.trc  (incident=1093505) (PDBNAME=CDB$ROOT):
ORA-501 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /opt/oracle/diag/rdbms/XFF/XFF/incident/incdir_1093505/XFF_clmn_31741_i1093505.trc
2020-03-10T01:46:49.264624+08:00
USER (ospid: 31741): terminating the instance due to ORA error 501
2020-03-10T01:46:49.280664+08:00
System state dump requested by (instance=1, osid=31741 (CLMN)), summary=[abnormal instance termination].
System State dumped to trace file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_diag_31759.trc
2020-03-10T01:46:53.156926+08:00
ORA-00501: CLMN process terminated with error
2020-03-10T01:46:53.157103+08:00
Errors in file /opt/oracle/diag/rdbms/XFF/XFF/trace/XFF_diag_31759.trc:
ORA-00501: CLMN process terminated with error
2020-03-10T01:46:53.157211+08:00
Dumping diagnostic data in directory=[cdmp_20200310014649], requested by (instance=1, osid=31741 (CLMN)), 
summary=[abnormal instance termination].

通过报错信息判断,数据库open之后(特别是pdb 4 open之后),开始报ORA-600 4193错误.然后由于CLMN进程异常,最后数据库crash.对于这类故障,因为使用的pdb,而且是由于pdb的undo异常导致数据库启动之后crash,可以通过对于pdb进行特殊处理,从而实现数据库启动之后不再crash.

发表在 ORACLE 19C, Oracle备份恢复 | 标签为 , , , | 评论关闭