联系：手机/微信(+86 17813235971) QQ(107644445)
标题：ORA-600 kcvent_internal_02故障处理
作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]
数据库启动报ORA-00600: internal error code, arguments: [kcvent_internal_02]错,无法正常open
Reconfiguration complete parallel recovery started with 32 processes Started redo scan Completed redo scan read 22775 KB redo, 5055 data blocks need recovery Started redo application at Thread 2: logseq 166395, block 88 Recovery of Online Redo Log: Thread 2 Group 3 Seq 166395 Reading mem 0 Mem# 0: +DATA/orcl/onlinelog/group_3.283.1036687245 Mem# 1: +FLASH/orcl/onlinelog/group_3.264.1036687257 Recovery of Online Redo Log: Thread 2 Group 4 Seq 166396 Reading mem 0 Mem# 0: +DATA/orcl/onlinelog/group_4.284.1036687257 Mem# 1: +FLASH/orcl/onlinelog/group_4.265.1036687257 Completed redo application of 15.97MB Completed instance recovery at Thread 2: logseq 166396, block 15854, scn 27533037896 5055 data blocks read, 5055 data blocks written, 22775 redo k-bytes read Thread 2 advanced to log sequence 166397 (thread recovery) Redo thread 2 internally disabled at seq 166397 Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_35652472.trc (incident=195549): ORA-00600: internal error code, arguments: [kcvent_internal_02], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_195549/orcl1_ora_35652472_i195549.trc
对应的trace文件信息
Dump continued from file: /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_35652472.trc
ORA-00600: internal error code, arguments: [kcvent_internal_02], [], [], [], [], [], [], [], [], [], [], []
========= Dump for incident 195549 (ORA 600 [kcvent_internal_02]) ========
*** 2022-06-06 22:17:48.743
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=5fmpzya54p4hf) -----
ALTER DATABASE OPEN /* db agent *//* {1:38339:2} */
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
skdstdst()+40 bl 0000000109B1E77C 000000000 ? 000000001 ?
000000003 ? 000000000 ?
000000000 ? 000000001 ?
000000003 ? 000000000 ?
ksedst1()+112 call skdstdst() 16F60DC8B26FAB02 ?
4846284100000000 ?
FFFFFFFFFFE46D0 ?
283C6E7C6A9A6 ? 10A6B923C ?
000000000 ? 110737880 ?
2050033FFFE46D8 ?
ksedst()+40 call ksedst1() 000000000 ? 00000000A ?
07FFFFFFF ? 700000000003670 ?
000000000 ? 000000000 ?
000002004 ? 000000001 ?
dbkedDefDump()+1516 call ksedst() 000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 300000003 ?
ksedmp()+72 call dbkedDefDump() 310737880 ? 110000D40 ?
FFFFFFFFFFE4EE0 ? 1106AB740 ?
100124BB8 ? 000000000 ?
700011D7387FF08 ? 1106AB740 ?
ksfdmp()+100 call ksedmp() 000000002 ? 000000000 ?
000000002 ? 10AF01CA8 ?
10A041C38 ? 000000000 ?
11073C760 ? 110737880 ?
dbgexPhaseII()+1904 call ksfdmp() 000000000 ? 00000000A ?
000000002 ? 000000000 ?
000000002 ? 10A041C30 ?
000000000 ? 001050005 ?
dbgexProcessError() call dbgexPhaseII() 110737880 ? 11073A970 ?
+1556 00002FBDD ? 200000000 ?
FFFFFFFFFFE5DF8 ? 00000006C ?
200000000 ? 1000000000 ?
dbgeExecuteForError call dbgexProcessError() 110737880 ? 11073C760 ?
()+72 100000703 ? 000004000 ?
000000000 ? FFFFFFFFFFE9608 ?
000000001 ? 11073E4A8 ?
dbgePostErrorKGE()+ call dbgeExecuteForError FFFFFFFFFFE92B0 ?
2044 () 700011D61558BB8 ? 102878B5C ?
000000000 ? 000000000 ?
FFFFFFFFFFE9608 ? 000000000 ?
000000000 ?
dbkePostKGE_kgsf()+ call dbgePostErrorKGE() 07FFFFFFF ? 700000000003670 ?
68 25800000001 ? 109E4A618 ?
000000000 ? 000000000 ?
FFFFFFFFFFEA0B0 ? 1109C0040 ?
kgeadse()+380 call dbkePostKGE_kgsf() 000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 069186EAB ?
kgerinv_internal()+ call kgeadse() 000000002 ? 000000002 ?
48 000000001 ? FFFFFFFFFFEAB58 ?
10A4E02F0 ? 000000002 ?
FFFFFFFFFFE9FE0 ? 000000000 ?
kgerinv()+48 call kgerinv_internal() 200000002 ? 000000002 ?
FFFFFFFFFFEA060 ? 000000000 ?
102860EB0 ? FFFFFFFFFFEA458 ?
10285CE74 ? FFFFFFFFFFEA358 ?
kgeasnmierr()+72 call kgerinv() 38400000001 ? 000000000 ?
10A4E0D20 ? 497F0A29CAE0 ?
000000001 ? FFFFFFFFFFEA1C0 ?
10A4E0D20 ? 110000D78 ?
kcvent_internal()+1 call kgeasnmierr() FFFFFFFFFFEA1C0 ? 200000002 ?
532 1F0410001F041 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000004 ?
kctenb_internal()+2 call kcvent_internal() FFFFFFFFFFEB378 ? 200000002 ?
772 FFFFFFFFFFEB448 ?
FFFFFFFFFFEB2E8 ?
41F6C57900000000 ?
000000000 ? FFFFFFFFFFEB330 ?
1106AB740 ?
kcfopd()+1508 call kctenb_internal() 07FFFFFFF ? 000000000 ?
000000018 ? FFFFFFFFFFEC380 ?
000000000 ? 110A39050 ?
FFFFFFFFFFEC390 ? 000000000 ?
adbdrv()+8028 call kcfopd() 081F0AD00 ? 00000000F ?
0FFFED4C0 ? 000000000 ?
FFFFFFFFFFED548 ? 100000000 ?
000000000 ? 1000100000000 ?
opiexe()+16048 call adbdrv() 2300000023 ? 100000001 ?
000000000 ? FFFFFFFFFFF6960 ?
000000000 ? FFFFFFFFFFF6B60 ?
FFFFFFFFFFF6A98 ? 200000002 ?
opiosq0()+3984 call opiexe() 700011E117B3B20 ? 000000000 ?
FFFFFFFFFFF7ED8 ? 110000D78 ?
000000001 ? 1109FA438 ?
FFFFFFFFFFF7E70 ?
2216414400000001 ?
kpooprx()+316 call opiosq0() 300000000 ? 000000000 ?
000000000 ? A4000000000000 ?
000000000 ? FFFFFFFFFFF87F0 ?
28104221FFFF86F0 ?
1109FAB08 ?
kpoal8()+872 call kpooprx() 1000CE68C ? 000000001 ?
FFFFFFFFFFFAD14 ? 100000001 ?
000000000 ? A40000000000A4 ?
109EB6D00 ? 000000000 ?
opiodr()+908 call kpoal8() 100000000 ? 9001000A0091108 ?
000000FFF ? 07FFFFFF8 ?
FFFFFFFFFFF8F10 ? 000000018 ?
000000000 ? 000072FFF ?
ttcpip()+1028 call opiodr() 5EFFFFA480 ? 1C00200048 ?
FFFFFFFFFFFA9F8 ? 000530058 ?
1108BEE30 ? 000000028 ?
FFFFFFFFFFFA3A0 ? 1108BEC70 ?
opitsk()+1612 call ttcpip() 110135440 ? 000002078 ?
000000000 ? 110000D78 ?
110005210 ? 000000000 ?
FFFFFFFFFFFAA20 ?
2222208009EF13C0 ?
opiino()+940 call opitsk() 110024C58 ? 000000000 ?
11079B550 ? 1107A0850 ?
110737880 ? FFFFFFFFFFFCAE0 ?
FFFFFFFFFFFEB3C ? 000000101 ?
opiodr()+908 call opiino() 3C006C787C ?
BFF0000000000000 ?
FFFFFFFFFFFEF60 ?
FFFFFFFFFFFD5E9 ?
FFFFFFFFFFFD630 ? 1106AB740 ?
FFFFFFFFFFFD650 ?
9FFFFFFF000E608 ?
opidrv()+1132 call opiodr() 3C0AFBC600 ? 410134340 ?
FFFFFFFFFFFEF60 ? 07530312F ?
108820CE4 ? 1106AB740 ?
7264626D732F6F72 ?
1106AB740 ?
sou2o()+136 call opidrv() 3C0882A9D0 ? 41170031F ?
FFFFFFFFFFFEF60 ?
110017002A0000 ? 0E0DDF00D ?
1106AB740 ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
opimai_real()+560 call sou2o() FFFFFFFFFFFEFD0 ?
BADC0FFEE0DDF00D ?
90000000008BE3C ?
BADC0FFEE0DDF00D ?
000000002 ? 9001000A0091108 ?
A0000000A000000 ? 10B671248 ?
ssthrdmain()+276 call opimai_real() 10B6B1D74 ? 9001000A0095260 ?
FFFFFFFFFFFF0B0 ? 10B6B1598 ?
FFFFFFFFFFFF0D0 ?
FFFFFFFFFFFF428 ?
900000000100968 ?
9001000A0091108 ?
main()+204 call ssthrdmain() 240000000 ? FFFFFFFFFFFF418 ?
8FFFFFFF0000090 ? 000000000 ?
000000000 ? 000000000 ?
BADC0FFEE0DDF00D ?
BADC0FFEE0DDF00D ?
__start()+112 call main() 000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
000000000 ? 000000000 ?
--------------------- Binary Stack Dump ---------------------
该错误在mos,互联网上没有任何信息,不过在alert日志中发现类似信息
Mon Jun 06 23:03:58 2022
Error: Controlfile sequence number in file header is different from the one in memory
Please check that the correct mount options are used if controlfile is located on NFS
初步判断可能和这个错误有关系,解决相关问题后,尝试open库
SQL> recover database;
ORA-00279: change 27533037896 generated at 06/06/2022 22:17:46 needed for
thread 2
ORA-00289: suggestion :
+FLASH/orcl/archivelog/2022_06_06/thread_2_seq_166396.6532.1106691471
ORA-00280: change 27533037896 for thread 2 is in sequence #166396
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
Log applied.
Media recovery complete.
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-01216: thread 2 is expected to be disabled after CREATE CONTROLFILE
SQL> !oerr ora 01216
01216, 00000, "thread %s is expected to be disabled after CREATE CONTROLFILE"
// *Cause: A thread that was given during CREATE CONTROLFILE is enabled, but
// the datafiles indicate that it should be disabled. This is
// probably because the logs supplied to the CREATE CONTROLFILE
// are old (from before the disabling of the thread).
// *Action: This thread is not required to run the database. The CREATE
// CONTROLFILE statement can be reissued without the problem thread,
// and, if desired, the thread can be recreated after the database
// is open.
ORA-01216这个错误比较也比较少见,但是感觉和thread有关系,大概的意思是thread 被disable了
SQL> select thread#,STATUS FROM V$THREAD;
THREAD# STATUS
---------- ------------------
1 CLOSED
2 CLOSED
通过人工强制把thread个open,然后数据库启动成功
SQL> select thread#,status from v$thread;
THREAD# STATUS
---------- ------------------
1 OPEN
2 CLOSED
SQL> alter database open;
Database altered.
然后启动thread 2,open 第二个节点
--需要open节点 QL> startup ORACLE instance started. Total System Global Area 1.2961E+11 bytes Fixed Size 2262400 bytes Variable Size 3.3018E+10 bytes Database Buffers 9.6368E+10 bytes Redo Buffers 221818880 bytes ORA-01618: redo thread 2 is not enabled - cannot mount --已经open节点 SQL> ALTER DATABASE ENABLE THREAD 2; Database altered. --需要open节点 SQL> ALTER DATABASE MOUNT; Database altered. SQL> ALTER DATABASE OPEN; Database altered.
xifenfei1:/home/grid$crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.FLASH.dg
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.LISTENER.lsnr
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.OCR.dg
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.asm
ONLINE ONLINE xifenfei1 Started
ONLINE ONLINE xifenfei2 Started
ora.gsd
OFFLINE OFFLINE xifenfei1
OFFLINE OFFLINE xifenfei2
ora.net1.network
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.ons
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
ora.registry.acfs
ONLINE ONLINE xifenfei1
ONLINE ONLINE xifenfei2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE xifenfei1
ora.cvu
1 OFFLINE OFFLINE
ora.xifenfei1.vip
1 ONLINE ONLINE xifenfei1
ora.xifenfei2.vip
1 ONLINE ONLINE xifenfei2
ora.oc4j
1 ONLINE ONLINE xifenfei2
ora.orcl.db
1 ONLINE ONLINE xifenfei1 Open
2 ONLINE ONLINE xifenfei2 Open
ora.scan1.vip
1 ONLINE ONLINE xifenfei1
加我QQ
