标签云
asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2131 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (102)
- 数据库 (1,683)
- DB2 (22)
- MySQL (73)
- Oracle (1,545)
- Data Guard (52)
- EXADATA (8)
- GoldenGate (24)
- ORA-xxxxx (159)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (15)
- ORACLE 21C (3)
- Oracle 23ai (7)
- Oracle ASM (68)
- Oracle Bug (8)
- Oracle RAC (53)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (28)
- Oracle备份恢复 (565)
- Oracle安装升级 (92)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (79)
- PostgreSQL (18)
- PostgreSQL恢复 (6)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (37)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (20)
-
最近发表
- ORA-15411: Failure groups in disk group DATA have different number of disks.
- 断电引起的ORA-08102: 未找到索引关键字, 对象号 39故障处理
- ORA-00227: corrupt block detected in control file
- 手工删除19c rac
- 解决oracle数据文件路径有回车故障
- .wstop扩展名勒索数据库恢复
- Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障(文件offline)
- OGG-02771 Input trail file format RELEASE 19.1 is different from previous trail file form at RELEASE 11.2.
- OGG-02246 Source redo compatibility level 19.0.0 requires trail FORMAT 12.2 or higher
- GoldenGate 19安装和打patch
- dd破坏asm磁盘头恢复
- 删除asmlib磁盘导致磁盘组故障恢复
- Kylin Linux 安装19c
- ORA-600 krse_arc_complete.4
- Oracle 19c 202410补丁(RUs+OJVM)
- ntfs MFT损坏(ntfs文件系统故障)导致oracle异常恢复
- .mkp扩展名oracle数据文件加密恢复
- 清空redo,导致ORA-27048: skgfifi: file header information is invalid
- A_H_README_TO_RECOVER勒索恢复
- 通过alert日志分析客户自行对一个数据库恢复的来龙去脉和点评
标签归档:kdfReserveSingle_1
aix平台tab$被删除可能出现ORA-600 [16703], [1403], [28]错误
以前在恢复过程中遇到过ORA-00600: internal error code, arguments: [16703], [1403], [28]错误(10g数据库遭遇ORA-600 16703)以为是因为10g版本的tab$记录被删除的原因导致报错和最常见的ORA-00600: internal error code, arguments: [16703], [1403], [20]不完全一致(警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703),最近又遇到一个ORA-600 16703 1403 28错误的case,而且数据库版本是11.2.0.4 for aix平台,进一步说明该问题不是由于10g和11g的tab$被删除的区别导致,更多可能是由于操作系统不一样,数据库启动基表访问顺序不一致导致,特此进行说明.
数据库启动成功后报错
Wed Apr 01 22:36:19 2020 Completed: ALTER DATABASE OPEN /* db agent *//* {2:54387:2} */ Wed Apr 01 22:36:19 2020 Starting background process CJQ0 Wed Apr 01 22:36:19 2020 CJQ0 started with pid=53, OS id=7078224 Wed Apr 01 22:36:21 2020 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7668220.trc (incident=40337): ORA-00600: internal error code, arguments: [kzrini:!uprofile], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_40337/orcl2_ora_7668220_i40337.trc Wed Apr 01 22:36:21 2020 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_6881718.trc (incident=40369): ORA-00600: internal error code, arguments: [kzrini:!uprofile], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_40369/orcl2_ora_6881718_i40369.trc Setting Resource Manager plan SCHEDULER[0x32DB]:DEFAULT_MAINTENANCE_PLAN via scheduler window Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter Wed Apr 01 22:51:16 2020 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc: ORA-00604: error occurred at recursive SQL level 1 ORA-00957: duplicate column name Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc: ORA-00604: error occurred at recursive SQL level 1 ORA-00957: duplicate column name Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc: ORA-00604: error occurred at recursive SQL level 1 ORA-00957: duplicate column name Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_m000_4850312.trc (incident=48045): ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_48045/orcl2_m000_4850312_i48045.trc Thu Apr 02 00:59:35 2020 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_smon_7078802.trc: ORA-00604: error occurred at recursive SQL level 1 ORA-00957: duplicate column name Thu Apr 02 00:59:36 2020 DDE: Problem Key 'ORA 600 [kzrini:!uprofile]' was completely flood controlled (0x6) Further messages for this problem key will be suppressed for up to 10 minutes Thu Apr 02 00:59:37 2020 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_m000_4850312.trc (incident=48046): ORA-00600: internal error code, arguments: [kewrose_1], [600], [ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], []
数据库再次重启报错
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:25340:2} */ ALTER DATABASE OPEN /* db agent *//* {1:25340:2} */ This instance was first to open Picked broadcast on commit scheme to generate SCNs Thu Apr 02 02:17:59 2020 Thread 2 opened at log sequence 13485 Current log# 3 seq# 13485 mem# 0: +DATA/orcl/onlinelog/group_3.265.1003137665 Current log# 3 seq# 13485 mem# 1: +FLASH/orcl/onlinelog/group_3.259.1003137677 Successful open of redo thread 2 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Thu Apr 02 02:17:59 2020 SMON: enabling cache recovery Instance recovery: looking for dead threads Instance recovery: lock domain invalid but no dead threads Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc (incident=48395): ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl2/incident/incdir_48395/orcl2_ora_7799020_i48395.trc Thu Apr 02 02:18:01 2020 Thu Apr 02 02:18:01 2020 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], [] Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl2/trace/orcl2_ora_7799020.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [28], [], [], [], [], [], [], [], [], [] Error 704 happened during db open, shutting down database USER (ospid: 7799020): terminating the instance due to error 704 Instance terminated by USER, pid = 7799020 ORA-1092 signalled during: ALTER DATABASE OPEN /* db agent *//* {1:25340:2} */...
对于此类问题,通过分析,确定也是由于DBMS_SUPPORT_DBMONITORP恶意脚本导致tab$记录被删除,导致数据库启动异常,处理方法基本上就是对tab$进行恢复,然后open数据库.
tab$被恶意删除sys用户之外记录
数据库open成功,但是alert日志报大量ORA-00600错误
Sun Apr 14 14:30:46 2019 SMCO started with pid=53, OS id=6761 Completed: ALTER DATABASE OPEN /* db agent *//* {1:65047:2} */ Sun Apr 14 14:30:49 2019 Starting background process CJQ0 Sun Apr 14 14:30:49 2019 CJQ0 started with pid=54, OS id=6776 Setting Resource Manager plan SCHEDULER[0x32DF]:DEFAULT_MAINTENANCE_PLAN via scheduler window Setting Resource Manager plan DEFAULT_MAINTENANCE_PLAN via parameter Sun Apr 14 14:30:52 2019 Starting background process VKRM Sun Apr 14 14:30:52 2019 VKRM started with pid=37, OS id=6809 Sun Apr 14 14:30:54 2019 Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j000_6811.trc (incident=288633): ORA-00600: internal error code, arguments: [kkpo_rcinfo_defstg:delseg], [84638], [], [], [], [], [], [], [], [], [], [] Incident details in: /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_288633/xifenfei1_j000_6811_i288633.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j000_6811.trc: ORA-00600: internal error code, arguments: [kkpo_rcinfo_defstg:delseg], [84638], [], [], [], [], [], [], [], [], [], [] ORA-06512: at "APEX_030200.WWV_FLOW_MAIL", line 695 ORA-06512: at line 1 Sun Apr 14 14:30:57 2019 Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j000_7491.trc (incident=288658): ORA-00600: 内部错误代码, 参数: [16659], [kqldtu], [INS], [0], [206196], [], [], [], [], [], [], [] Incident details in: /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_288658/xifenfei1_j000_7491_i288658.trc Sun Apr 14 14:34:10 2019 Dumping diagnostic data in directory=[cdmp_20190414143410], requested by (instance=1, osid=7491 (J000)), summary=[incident=288658]. Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j000_7491.trc: ORA-00600: 内部错误代码, 参数: [16659], [kqldtu], [INS], [0], [206196], [], [], [], [], [], [], [] ORA-06512: 在 "WEBCSMS.P_YGERROR", line 3 Sun Apr 14 14:39:08 2019 Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j000_8515.trc (incident=288593): ORA-00600: 内部错误代码, 参数: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Incident details in: /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_288593/xifenfei1_j000_8515_i288593.trc ORA-06512: 在 line 1 Sun Apr 14 14:52:14 2019 Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_j001_11062.trc (incident=288705): ORA-00600: internal error code, arguments: [16607], [0x3CFB04C90], [257], [9], [0x000000000], [], [], [], [], [], [], [] Incident details in: /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_288705/xifenfei1_j001_11062_i288705.trc Errors in file /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_16712.trc (incident=288597): ORA-00600: 内部错误代码, 参数: [16607], [0x3C7CEA678], [1281], [9], [0x000000000], [], [], [], [], [], [], [] Incident details in: /oracle/oracle/oracle/diag/rdbms/xifenfei/xifenfei1/incident/incdir_288597/xifenfei1_ora_16712_i288597.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details.
报错比较多,客户还反馈登录数据库之后,发现所有的表都丢失。第一反应可能数据字典损坏了,然后让客户查看备库,现在dg的备库也一样表都丢失了,进一步确认字典可能异常,让客户提供system文件进行本地分析.发现DBMS_SUPPORT_DBMONITOR触发器调用DBMS_SUPPORT_DBMONITORP存储过程,和警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703中的名称非常类似,但是有点不一样,以前的恶意脚本中都是被注入并且触发之后,数据库无法正常启动,这次数据库能够正常open成功.分析恶意脚本,确认原因
确实这次的恶意脚本是在2016年8月份被创建在库中,在600天之后重启被触发,而且是删除非sys的tab$中记录.知道了恶意脚本的源头,那恢复就比较容易,直接通过批量bbed程序对tab$反删除可以实现比较完美恢复.原则上这样的故障可以实现数据库完美恢复,原库继续使用.
12C数据库遭遇ORA-600 16703
关于tab$异常的问题,我们在10G(10g数据库遭遇ORA-600 16703),11G(警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703)中有发现,近期在12C版本中也发现该问题.再次提醒请注意检查安装介质的md5值和现在运行的库中是否有相关恶意程序
数据库版本
alert日志报错
************************************************************ Warning: The SCN headroom for this database is only 45 days! ************************************************************ ************************************************************ Warning: The SCN headroom for this database is only 45 days! ************************************************************ Picked broadcast on commit scheme to generate SCNs Fri Jan 25 11:34:55 2019 Starting background process TMON Fri Jan 25 11:34:55 2019 TMON started with pid=104, OS id=47282 Fri Jan 25 11:34:55 2019 Thread 1 advanced to log sequence 11942 (thread open) Thread 1 opened at log sequence 11942 Current log# 5 seq# 11942 mem# 0: +DATA/xifenfei/ONLINELOG/redo01_01.log Successful open of redo thread 1 Fri Jan 25 11:34:55 2019 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Fri Jan 25 11:34:55 2019 Network Resource Management enabled for Process LG00 (pid 451491) for Exadata I/O Fri Jan 25 11:34:55 2019 SMON: enabling cache recovery Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_46443.trc (incident=394486): ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Fri Jan 25 11:34:56 2019 Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_46443.trc: ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Fri Jan 25 11:34:56 2019 Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_46443.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Fri Jan 25 11:34:56 2019 Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_46443.trc: ORA-00704: bootstrap process failure ORA-00704: bootstrap process failure ORA-00600: internal error code, arguments: [16703], [1403], [20], [], [], [], [], [], [], [], [], [] Fri Jan 25 11:34:56 2019 Error 704 happened during db open, shutting down database USER (ospid: 46443): terminating the instance due to error 704 Fri Jan 25 11:34:57 2019 Instance terminated by USER, pid = 46443
处理思路基本上和其他两个版本一样,都是对tab$进行修复,然后直接open数据库,但是在这次的恢复中遇到ORA-00600 kdfReserveSingle_1错误
ORA-600 kdfReserveSingle_1
SQL> alter database open ; alter database open * ERROR at line 1: ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Process ID: 319852 Session ID: 2401 Serial number: 24555 ---alert日志 Starting background process TMON Fri Jan 25 20:51:41 2019 TMON started with pid=94, OS id=322554 Fri Jan 25 20:51:41 2019 Thread 1 advanced to log sequence 11943 (thread open) Thread 1 opened at log sequence 11943 Current log# 6 seq# 11943 mem# 0: +DATA/xifenfei/ONLINELOG/redo02_01.log Successful open of redo thread 1 Fri Jan 25 20:51:41 2019 MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set Fri Jan 25 20:51:41 2019 Network Resource Management enabled for Process LG00 (pid 319754) for Exadata I/O Fri Jan 25 20:51:41 2019 SMON: enabling cache recovery Fri Jan 25 20:51:41 2019 [319852] Successfully onlined Undo Tablespace 2. Undo initialization finished serial:0 start:88159754 end:88160264 diff:510 ms (0.5 seconds) Verifying minimum file header compatibility (11g) for tablespace encryption.. Verifying 11g file header compatibility for tablespace encryption completed Fri Jan 25 20:51:41 2019 SMON: enabling tx recovery Fri Jan 25 20:51:41 2019 Database Characterset is ZHS16GBK Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_319852.trc (incident=466446): ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Fri Jan 25 20:51:42 2019 Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_319852.trc: ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Fri Jan 25 20:51:42 2019 Errors in file /u01/oracle/app/oracle/diag/rdbms/xifenfei/xifenfei1/trace/xifenfei1_ora_319852.trc: ORA-00600: internal error code, arguments: [kdfReserveSingle_1], [0], [65280], [], [], [], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 319852): terminating the instance due to error 600 Fri Jan 25 20:51:43 2019 Instance terminated by USER, pid = 319852 ORA-1092 signalled during: alter database open… opiodr aborting process unknown ospid (319852) as a result of ORA-1092 Fri Jan 25 20:51:49 2019 ORA-1092 : opitsk aborting process
这个错误主要是由于12c的tab$和以前版本比有一些特殊,导致开始恢复不全,通过继续修复tab$之后数据库正常open.
相关文章汇总
tab$恢复错误汇总
12C数据库遭遇ORA-600 16703
10G数据库遭遇ORA-600 16703
ORA-600 16703故障解析—tab$表被清空
警告:互联网中有oracle介质被注入恶意程序导致—ORA-600 16703
发表在 Oracle
标签为 12c tab$, delete tab$, kdfReserveSingle_1, ORA-600 16703, ORA-600 kdfReserveSingle_1, tab$清空
评论关闭