联系：手机/微信(+86 17813235971) QQ(107644445)
作者：惜分飞©版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]
看到dbsnake关于UPDATE GLOBAL_NAME为空之后的恢复写的不是很完整,自己通过试验和对bbed的研究,完善他的blog内容(泄露一点内部的东西)
模拟错误
SQL> select * from v$version; BANNER ------------------------------------------------------------------------ Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - Production PL/SQL Release 11.2.0.3.0 - Production CORE 11.2.0.3.0 Production TNS for Linux: Version 11.2.0.3.0 - Production NLSRTL Version 11.2.0.3.0 - Production SQL> select * from global_name ; GLOBAL_NAME -------------------------------------------------------------------- ORA11G SQL> update global_name set global_name=''; 1 row updated. SQL> COMMIT; Commit complete. SQL> SHUTDOWN IMMEDIATE Database closed. Database dismounted. ORACLE instance shut down. SQL> STARTUP ORACLE instance started. Total System Global Area 313860096 bytes Fixed Size 1344652 bytes Variable Size 260049780 bytes Database Buffers 46137344 bytes Redo Buffers 6328320 bytes Database mounted. ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Process ID: 28306 Session ID: 125 Serial number: 5
alert日志
Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_28306.trc (incident=20556): ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /u01/oracle/diag/rdbms/ora11g/ora11g/incident/incdir_20556/ora11g_ora_28306_i20556.trc Wed Aug 08 23:21:48 2012 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_28306.trc: ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Errors in file /u01/oracle/diag/rdbms/ora11g/ora11g/trace/ora11g_ora_28306.trc: ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 28306): terminating the instance due to error 600 Instance terminated by USER, pid = 28306 ORA-1092 signalled during: ALTER DATABASE OPEN... opiodr aborting process unknown ospid (28306) as a result of ORA-1092 Wed Aug 08 23:21:48 2012 ORA-1092 : opitsk aborting process
找出global_name相关信息(另外库中)
SQL> SET LONG 1000
SQL> select dbms_metadata.get_ddl('VIEW','GLOBAL_NAME','SYS') from dual;
DBMS_METADATA.GET_DDL('VIEW','GLOBAL_NAME','SYS')
--------------------------------------------------------------------------------
CREATE OR REPLACE FORCE VIEW "SYS"."GLOBAL_NAME" ("GLOBAL_NAME") AS
select value$ from sys.props$ where name = 'GLOBAL_DB_NAME'
SQL> select * from props$ where value$='XIFENFEI';
NAME VALUE$
------------------------------ ----------------------------------------
COMMENT$
--------------------------------------------------------------------------
GLOBAL_DB_NAME XIFENFEI
Global database name
SQL> select dump('GLOBAL_DB_NAME','16') from dual;
DUMP('GLOBAL_DB_NAME','16')
--------------------------------------------------------
Typ=96 Len=14: 47,4c,4f,42,41,4c,5f,44,42,5f,4e,41,4d,45
--得出GLOBAL_DB_NAME存储字16进制字符串为:0e474c4f42414c5f44425f4e414d45
bbed操作
--通过第三方工具定位props$表中的GLOBAL_DB_NAME列在数据块0x00400321的31行上
[oracle@xifenfei ~]$ bbed listfile=listfile mode=edit password=blockedit
BBED: Release 2.0.0.0.0 - Limited Production on Thu Aug 9 00:26:12 2012
Copyright (c) 1982, 2007, Oracle. All rights reserved.
************* !!! For Oracle Internal Use only !!! ***************
BBED> info
File# Name Size(blks)
----- ---- ----------
1 /u01/oracle/oradata/ora11g/system01.dbf 0
2 /u01/oracle/oradata/ora11g/sysaux01.dbf 0
3 /u01/oracle/oradata/ora11g/undotbs01.dbf 0
4 /u01/oracle/oradata/ora11g/users01.dbf 0
BBED> set block 801
BLOCK# 801
BBED> map
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Dba:0x00400321
------------------------------------------------------------
KTB Data Block (Table/Cluster)
struct kcbh, 20 bytes @0
struct ktbbh, 72 bytes @20
struct kdbh, 14 bytes @92
struct kdbt[1], 4 bytes @106
sb2 kdbr[37] @110
ub1 freespace[5771] @184
ub1 rowdata[2233] @5955
ub4 tailchk @8188
BBED> p *kdbr[31]
rowdata[0]
----------
ub1 rowdata[0] @5955 0x2c
BBED> x /rccc
rowdata[0] @5955
----------
flag@5955: 0x2c (KDRHFL, KDRHFF, KDRHFH)
lock@5956: 0x02
cols@5957: 3
col 0[14] @5958: GLOBAL_DB_NAME
col 1[0] @5973: *NULL*
col 2[20] @5974: Global database name
BBED> set count 64
COUNT 64
BBED> f /x 0e474c4f
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5958 to 6021 Dba:0x00400321
------------------------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d45ff 14476c6f 62616c20 64617461 62617365
206e616d 652c0003 0e474c4f 42414c5f 44425f4e 414d4506 4f524131 31471447
<32 bytes per line>
BBED> f
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5998 to 6061 Dba:0x00400321
------------------------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d4506 4f524131 31471447 6c6f6261 6c206461
74616261 7365206e 616d652c 00030a44 4254494d 455a4f4e 45053030 3a30300c
<32 bytes per line>
BBED> f
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 6460 to 6523 Dba:0x00400321
------------------------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d4508 53454544 44415441 14476c6f 62616c20
64617461 62617365 206e616d 652c0003 114e4c53 5f524442 4d535f56 45525349
<32 bytes per line>
BBED> f
BBED-00212: search string not found
BBED> d /v offset 5958
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5958 to 6021 Dba:0x00400321
-------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d45ff l .GLOBAL_DB_NAME.
14476c6f 62616c20 64617461 62617365 l .Global database
206e616d 652c0003 0e474c4f 42414c5f l name,...GLOBAL_
44425f4e 414d4506 4f524131 31471447 l DB_NAME.ORA11G.G
<16 bytes per line>
BBED> d /v offset 5998
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5998 to 6061 Dba:0x00400321
-------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d4506 l .GLOBAL_DB_NAME.
4f524131 31471447 6c6f6261 6c206461 l ORA11G.Global da
74616261 7365206e 616d652c 00030a44 l tabase name,...D
4254494d 455a4f4e 45053030 3a30300c l BTIMEZONE.00:00.
<16 bytes per line>
BBED> d /v offset 6460
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 6460 to 6523 Dba:0x00400321
-------------------------------------------------------
0e474c4f 42414c5f 44425f4e 414d4508 l .GLOBAL_DB_NAME.
53454544 44415441 14476c6f 62616c20 l SEEDDATA.Global
64617461 62617365 206e616d 652c0003 l database name,..
114e4c53 5f524442 4d535f56 45525349 l .NLS_RDBMS_VERSI
<16 bytes per line>
--通过时上面的查找可以知道数据库对GLOBAL_DB_NAME有进行两次update操作
--GLOBAL_DB_NAME初始化值SEEDDATA,第一次更新为ORA11G,第二次更新为null
BBED> p kdbr[31]
sb2 kdbr[31] @172 5863
--这里可以发现我们看到offset 5995开始有值,但是row directory却指向了5863
BBED> d offset 5863 count 128
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5863 to 5990 Dba:0x00400321
------------------------------------------------------------------------
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 2c02030e
474c4f42 414c5f44 425f4e41 4d45ff14 476c6f62 616c2064 61746162 61736520
<32 bytes per line>
--通过dump看到row directory指向的值和实际的数据相差(5995-5863),都是0
BBED> d /v offset 5900
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5900 to 6027 Dba:0x00400321
-------------------------------------------------------
00000000 00000000 00000000 00000000 l ................
00000000 00000000 00000000 00000000 l ................
00000000 00000000 00000000 00000000 l ................
00000000 0000002c 02030e47 4c4f4241 l .......,...GLOBA
4c5f4442 5f4e414d 45ff1447 6c6f6261 l L_DB_NAME..Globa
6c206461 74616261 7365206e 616d652c l l database name,
00030e47 4c4f4241 4c5f4442 5f4e414d l ...GLOBAL_DB_NAM
45064f52 41313147 14476c6f 62616c20 l E.ORA11G.Global
<16 bytes per line>
--需要指定的新值前面也存在同样的0,所以模仿的处理方法,让row directory同样向前偏移92
BBED> m /x 0f17 offset 172
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 172 to 299 Dba:0x00400321
------------------------------------------------------------------------
0f176018 16189a17 5e173d17 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<32 bytes per line>
BBED> p kdbr[31]
sb2 kdbr[31] @172 5903
BBED> p *kdbr[31]
rowdata[40]
-----------
ub1 rowdata[40] @5995 0x2c
BBED> x /rccc
rowdata[40] @5995
-----------
flag@5995: 0x2c (KDRHFL, KDRHFF, KDRHFH)
lock@5996: 0x00
cols@5997: 3
col 0[14] @5998: GLOBAL_DB_NAME
col 1[6] @6013: ORA11G
col 2[20] @6020: Global database name
修改lock信息
BBED> m /x 02 offset 5996
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5996 to 6123 Dba:0x00400321
------------------------------------------------------------------------
02030e47 4c4f4241 4c5f4442 5f4e414d 45064f52 41313147 14476c6f 62616c20
64617461 62617365 206e616d 652c0003 0a444254 494d455a 4f4e4505 30303a30
300c4442 2074696d 65207a6f 6e652c00 02174e4f 5f555345 5249445f 56455249
46494552 5f53414c 54203633 39364335 38414231 37414530 30374539 41373238
<32 bytes per line>
BBED> d offset 5955
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5955 to 6082 Dba:0x00400321
------------------------------------------------------------------------
2c02030e 474c4f42 414c5f44 425f4e41 4d45ff14 476c6f62 616c2064 61746162
61736520 6e616d65 2c02030e 474c4f42 414c5f44 425f4e41 4d45064f 52413131
4714476c 6f62616c 20646174 61626173 65206e61 6d652c00 030a4442 54494d45
5a4f4e45 0530303a 30300c44 42207469 6d65207a 6f6e652c 0002174e 4f5f5553
<32 bytes per line>
BBED> m /x 2c00
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 5955 to 6082 Dba:0x00400321
------------------------------------------------------------------------
2c00030e 474c4f42 414c5f44 425f4e41 4d45ff14 476c6f62 616c2064 61746162
61736520 6e616d65 2c02030e 474c4f42 414c5f44 425f4e41 4d45064f 52413131
4714476c 6f62616c 20646174 61626173 65206e61 6d652c00 030a4442 54494d45
5a4f4e45 0530303a 30300c44 42207469 6d65207a 6f6e652c 0002174e 4f5f5553
<32 bytes per line>
--验证块
BBED> sum apply
Check value for File 1, Block 801:
current = 0xe836, required = 0xe836
BBED> verify
DBVERIFY - Verification starting
FILE = /u01/oracle/oradata/ora11g/system01.dbf
BLOCK = 801
Block Checking: DBA = 4195105, Block Type = KTB-managed data block
data header at 0xb677b25c
kdbchk: the amount of space used is not equal to block size
used=2056 fsc=6 avsp=6040 dtl=8096
Block 801 failed with check code 6110
DBVERIFY - Verification complete
Total Blocks Examined : 1
Total Blocks Processed (Data) : 1
Total Blocks Failing (Data) : 1
Total Blocks Processed (Index): 0
Total Blocks Failing (Index): 0
Total Blocks Empty : 0
Total Blocks Marked Corrupt : 0
Total Blocks Influx : 0
--修改_ktbitfsc信息，让其通过块验证
BBED> p ktbbhitl
struct ktbbhitl[0], 24 bytes @44
struct ktbitxid, 8 bytes @44
ub2 kxidusn @44 0x0004
ub2 kxidslt @46 0x000e
ub4 kxidsqn @48 0x000001d4
struct ktbituba, 8 bytes @52
ub4 kubadba @52 0x00c00a93
ub2 kubaseq @56 0x0083
ub1 kubarec @58 0x33
ub2 ktbitflg @60 0x8000 (KTBFCOM)
union _ktbitun, 2 bytes @62
b2 _ktbitfsc @62 0
ub2 _ktbitwrp @62 0x0000
ub4 ktbitbas @64 0x000c78fe
struct ktbbhitl[1], 24 bytes @68
struct ktbitxid, 8 bytes @68
ub2 kxidusn @68 0x000a
ub2 kxidslt @70 0x0016
ub4 kxidsqn @72 0x000001eb
struct ktbituba, 8 bytes @76
ub4 kubadba @76 0x00c0015b
ub2 kubaseq @80 0x008f
ub1 kubarec @82 0x0d
ub2 ktbitflg @84 0x0001 (NONE)
union _ktbitun, 2 bytes @86
b2 _ktbitfsc @86 6
ub2 _ktbitwrp @86 0x0006
ub4 ktbitbas @88 0x00000000
BBED> m /x 00 offset 86
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 86 to 213 Dba:0x00400321
------------------------------------------------------------------------
00000000 00000001 25000600 5c00e716 98179e17 00002500 701f571e 92189c1e
101efb1d ffff981d 551d0f1d d91cb71c 941c731c 4a1c1a1c ef1bc51b 9c1b701b
471b191b d61a931a 3c1a101a ee19bc19 86194e19 0d190f17 60181618 9a175e17
3d170000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<32 bytes per line>
BBED> sum apply
Check value for File 1, Block 801:
current = 0xe830, required = 0xe830
BBED> verify
DBVERIFY - Verification starting
FILE = /u01/oracle/oradata/ora11g/system01.dbf
BLOCK = 801
Block Checking: DBA = 4195105, Block Type = KTB-managed data block
data header at 0xb677b25c
kdbchk: space available on commit is incorrect
tosp=6046 fsc=0 stb=0 avsp=6040
Block 801 failed with check code 6111
DBVERIFY - Verification complete
Total Blocks Examined : 1
Total Blocks Processed (Data) : 1
Total Blocks Failing (Data) : 1
Total Blocks Processed (Index): 0
Total Blocks Failing (Index): 0
Total Blocks Empty : 0
Total Blocks Marked Corrupt : 0
Total Blocks Influx : 0
--修改kdbhtosp信息
BBED> p kdbh
struct kdbh, 14 bytes @92
ub1 kdbhflag @92 0x00 (NONE)
b1 kdbhntab @93 1
b2 kdbhnrow @94 37
sb2 kdbhfrre @96 6
sb2 kdbhfsbo @98 92
sb2 kdbhfseo @100 5863
b2 kdbhavsp @102 6040
b2 kdbhtosp @104 6046
BBED> d offset 102
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 102 to 229 Dba:0x00400321
------------------------------------------------------------------------
98179e17 00002500 701f571e 92189c1e 101efb1d ffff981d 551d0f1d d91cb71c
941c731c 4a1c1a1c ef1bc51b 9c1b701b 471b191b d61a931a 3c1a101a ee19bc19
86194e19 0d190f17 60181618 9a175e17 3d170000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<32 bytes per line>
BBED> d offset 104
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 104 to 231 Dba:0x00400321
------------------------------------------------------------------------
9e170000 2500701f 571e9218 9c1e101e fb1dffff 981d551d 0f1dd91c b71c941c
731c4a1c 1a1cef1b c51b9c1b 701b471b 191bd61a 931a3c1a 101aee19 bc198619
4e190d19 0f176018 16189a17 5e173d17 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<32 bytes per line>
BBED> m /x 98
File: /u01/oracle/oradata/ora11g/system01.dbf (1)
Block: 801 Offsets: 104 to 231 Dba:0x00400321
------------------------------------------------------------------------
98170000 2500701f 571e9218 9c1e101e fb1dffff 981d551d 0f1dd91c b71c941c
731c4a1c 1a1cef1b c51b9c1b 701b471b 191bd61a 931a3c1a 101aee19 bc198619
4e190d19 0f176018 16189a17 5e173d17 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<32 bytes per line>
--至此修改row directory指针完成
BBED> sum apply
Check value for File 1, Block 801:
current = 0xe836, required = 0xe836
BBED> verify
DBVERIFY - Verification starting
FILE = /u01/oracle/oradata/ora11g/system01.dbf
BLOCK = 801
DBVERIFY - Verification complete
Total Blocks Examined : 1
Total Blocks Processed (Data) : 1
Total Blocks Failing (Data) : 0
Total Blocks Processed (Index): 0
Total Blocks Failing (Index): 0
Total Blocks Empty : 0
Total Blocks Marked Corrupt : 0
Total Blocks Influx : 0
重新启动数据库
SQL> startup ORACLE instance started. Total System Global Area 313860096 bytes Fixed Size 1344652 bytes Variable Size 260049780 bytes Database Buffers 46137344 bytes Redo Buffers 6328320 bytes Database mounted. Database opened. SQL> select * from global_name ; GLOBAL_NAME -------------------------------------------------------- ORA11G SQL>
加我QQ
是指针，关于数据块的校验部分，自己慢慢去折磨，一句话讲不清楚
惜分飞,
谢谢大神，受益非浅！还有点疑问：p kdbr中值最小的就是数据位置的指针么？ 还有数据校验这一块还有点不清楚。能给指点指点么 ? 感谢感谢！
对数据库启动过程进行跟踪，可以定位到sql，也可以定位到block，然后dump block分析
请问大神，你怎么知道修改lock信息 时应该改为哪些值？
还有如果不知道global_name被改为空，怎么定位这个问题？
Ora-00600 [18062] During Database Startup
Applies to: Oracle Server - Enterprise Edition - Version: 11.2.0.2.0 and later [Release: 11.2 and later ] Information in this document applies to any platform. Symptoms After updating the GLOBAL_NAME and setting it to NULL: UPDATE GLOBAL_NAME SET GLOBAL_NAME = ''; COMMIT; The database is not starting up: SQL> shutdown immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 422670336 bytes Fixed Size 1344616 bytes Variable Size 318770072 bytes Database Buffers 96468992 bytes Redo Buffers 6086656 bytes Database mounted. ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Process ID: 3552 Session ID: 1 Serial number: 5 Cause The steps that can cause the error: SQL> select * from global_name; GLOBAL_NAME -------------------------------------------------------------------------------- ORCL1.LOCALDOMAIN SQL> UPDATE GLOBAL_NAME SET GLOBAL_NAME = ''; 1 row updated. SQL> commit; Commit complete. SQL> shutdown immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 422670336 bytes Fixed Size 1344616 bytes Variable Size 318770072 bytes Database Buffers 96468992 bytes Redo Buffers 6086656 bytes Database mounted. ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [18062], [], [], [], [], [], [], [], [], [], [], [] Process ID: 3552 Session ID: 1 Serial number: 5 Solution Steps to resolve the error: 1) Mount the DB SQL> conn / as sysdba Connected to an idle instance. SQL> startup mount; ORACLE instance started. Total System Global Area 422670336 bytes Fixed Size 1344616 bytes Variable Size 318770072 bytes Database Buffers 96468992 bytes Redo Buffers 6086656 bytes Database mounted. 2) Find the PID of the Oracle process oracle@localhost:[oracle@localhost ~]$ ps -ef | grep LOCAL | grep -v grep oracle 4030 3417 0 11:51 ? 00:00:00 oracleorcl1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) 3) Here the PID is 4030 which we will use in the GDB oracle@localhost:[oracle@localhost ~]$ gdb $ORACLE_HOME/bin/oracle 4030 GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/bin/oracle...(no debugging symbols found)...done. Attaching to program: /home/oracle/app/oracle/product/11.2.0/dbhome_2/bin/oracle, process 4030 warning: .dynamic section for "/lib/librt.so.1" is not at the expected address warning: difference appears to be caused by prelink, adjusting expectations Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libodm11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libodm11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libcell11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libcell11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libskgxp11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libskgxp11.so Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libnnz11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libnnz11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libclsra11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libclsra11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libdbcfg11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libdbcfg11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libhasgen11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libhasgen11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libskgxn2.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libskgxn2.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocr11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocr11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocrb11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocrb11.so Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocrutl11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libocrutl11.so Reading symbols from /usr/lib/libaio.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libaio.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib/libnuma.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libnuma.so.1 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libnque11.so...(no debugging symbols found)...done. Loaded symbols for /home/oracle/app/oracle/product/11.2.0/dbhome_2/lib/libnque11.so 0x00d24402 in __kernel_vsyscall () 4) Insert a breakpoint at kokiasg (gdb) break kokiasg Breakpoint 1 at 0x99f4d04 5) Continue the debugging (Gdb) continue Continuing. 6) Open the DB SQL> alter database open The hang will happen at the following point in the alert log due to the above break point Database Characterset is WE8MSWIN1252 The gdb output will show: Breakpoint 1 , 0x0a3d404d in kokiasg () 7) Open a new SQLPLUS session as the first one is currently hanging due to the break-point and restore the GLOBAL_NAME to its original value: SQLPLUS / as sysdba SQL> select status from v$instance; STATUS ----------- OPEN SQL > update global_name set global_name = 'ORCL1.LOCALDOMAIN'; 1 row updated. SQL > commit; 8) In another session connect to SQLPLUS and check the GLOBAL_NAME: SQL> select * from global_name; GLOBAL_NAME ---------------------- ORCL1.LOCALDOMAIN 9) Kill the GDB session which is currently suspending the database OPEN: (Gdb) kill Kill the program being debugged? (y or n) y (Gdb) quit At this moment, sqlplus will prompt: alter database open * ERROR at line 1 : ORA - 03113 : end - of - file on communication channel 10) Restart the DB.