惜分飞

有客户sql server数据库由于硬件故障,导致dbcc的时候报类似GAM、SGAM 或 PFS 页上存在页错误

kzj2025的 DBCC 结果。
Service Broker 消息 9675，状态 1: 已分析的消息类型: 14。
Service Broker 消息 9676，状态 1: 已分析的服务约定: 6。
Service Broker 消息 9667，状态 1: 已分析的服务: 3。
Service Broker 消息 9668，状态 1: 已分析的服务队列: 3。
Service Broker 消息 9669，状态 1: 已分析的会话端点: 0。
Service Broker 消息 9674，状态 1: 已分析的会话组: 0。
Service Broker 消息 9670，状态 1: 已分析的远程服务绑定: 0。
Service Broker 消息 9605，状态 1: 已分析的会话优先级: 0。
消息 8939，级别 16，状态 98，第 1 行
表错误: 对象 ID 0，索引 ID -1，分区 ID 0，分配单元 ID -2958221917649371136 
(类型为 Unknown)，页 (12337:808857908)。测试(IS_OFF (BUF_IOERR, pBUF->bstat))失败。值为 12716041 和 -10。
消息 8998，级别 16，状态 2，第 1 行
GAM、SGAM 或 PFS 页上存在页错误，无法对数据库 ID 6 中从 (1:186024) 到 (1:194111) 
 的页继续进行分配完整性检查。原因请参阅其他错误消息。
CHECKDB 发现有 2 个分配错误和 0 个一致性错误与任何单个的对象都没有关联。
sys.sysrscols的 DBCC 结果。
对象 'sys.sysrscols' 的 171 页中有 16248 行。
sys.sysrowsets的 DBCC 结果。
……………………
对象 'gspz_pmaintainidx' 的 12 页中有 1658 行。
DiseasesMedicationsIndex_temp的 DBCC 结果。
对象 'DiseasesMedicationsIndex_temp' 的 0 页中有 0 行。
GSP_BackBill的 DBCC 结果。
对象 'GSP_BackBill' 的 0 页中有 0 行。
Gsp_L_KwGradeIndex的 DBCC 结果。
对象 'Gsp_L_KwGradeIndex' 的 0 页中有 0 行。
Web_T_Message的 DBCC 结果。
对象 'Web_T_Message' 的 0 页中有 0 行。
CHECKDB 在数据库 'kzj2025' 中发现 2 个分配错误和 0 个一致性错误。
DBCC 执行完毕。如果 DBCC 输出了错误信息，请与系统管理员联系。

完成时间: 2025-03-29T19:53:21.6977003+08:00

由于GAM、SGAM 或 PFS是和sql数据库文件的空间分配有关系,对于这种情况,一般无法直接修复,需要对库进行重构的方法进行修复,我们处理之后,dbcc检查一切正常

最近一个客户咨询一个问题,他正常的drop tbs,结果触发ORA-600 krhpfh_03-1208 错误,导致数据库crash

Wed Mar 26 14:33:20 2025
Thread 1 cannot allocate new log, sequence 478485
Checkpoint not complete
  Current log# 2 seq# 478484 mem# 0: /apps/data/oracle/orcl/redo02.log
Thread 1 advanced to log sequence 478485 (LGWR switch)
  Current log# 3 seq# 478485 mem# 0: /apps/data/oracle/orcl/redo03.log
Wed Mar 26 14:35:06 2025
Wed Mar 26 14:35:06 2025
drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint
Read of datafile '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf' (fno 17) header failed with ORA-01208
Rereading datafile 17 header failed with ORA-01208
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7677):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7677/orcl_ora_188213_i7677.trc
Wed Mar 26 14:35:07 2025
Trace dumping is performing id=[cdmp_20250326143507]
ORA-600 signalled during: drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint...
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7678):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7678/orcl_ora_188213_i7678.trc
Wed Mar 26 14:35:08 2025
Sweep [inc][7678]: completed
Sweep [inc][7677]: completed
Sweep [inc2][7677]: completed
Wed Mar 26 14:35:09 2025
Thread 1 cannot allocate new log, sequence 478486
Checkpoint not complete
  Current log# 3 seq# 478485 mem# 0: /apps/data/oracle/orcl/redo03.log
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=7679):
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7679/orcl_ora_188213_i7679.trc
Trace dumping is performing id=[cdmp_20250326143511]
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc:
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: 内部错误代码, 参数: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935],[],[],[],[],[]
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: 数据库文件 17 验证失败
ORA-01110: 数据文件 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: 数据文件是旧的版本 - 不能访问当前版本
Thread 1 advanced to log sequence 478486 (LGWR switch)
  Current log# 1 seq# 478486 mem# 0: /apps/data/oracle/orcl/redo01.log
Wed Mar 26 14:35:13 2025
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_188213.trc  (incident=15551):
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: database file 17 failed verification check
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: data file is an old version - not accessing current version
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_15551/orcl_ora_188213_i15551.trc
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_15551/orcl_ora_188213_i15551.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01122: database file 17 failed verification check
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-01208: data file is an old version - not accessing current version
Trace dumping is performing id=[cdmp_20250326143514]
Wed Mar 26 14:35:15 2025
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_pmon_139367.trc  (incident=7224):
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
Incident details in: /apps/svr/oracle/diag/rdbms/orcl/orcl/incident/incdir_7224/orcl_pmon_139367_i7224.trc
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_pmon_139367.trc:
ORA-00600: internal error code, arguments: [krhpfh_03-1208],[fno =],[17],[fecpc =],[454709],[fhcpc =],[402935]
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
Wed Mar 26 14:35:19 2025
drop tablespace XFF_MON_2016 including contents and datafiles cascade constraint
Wed Mar 26 14:35:20 2025
DBW0 (ospid: 139390): terminating the instance due to error 472
Instance terminated by DBW0, pid = 139390

这个报错信息看,但是发起drop tbs之后,数据库应该是检查file 17号文件的状态,发现这个版本状态过旧(ORA-01208: 数据文件是旧的版本),由于某种原因报出来了krhpfh_03-1208,导致数据库crash了,然后他尝试启动数据库报ORA-01113: file 17 needs media recovery

Wed Mar 26 17:11:00 2025
Starting ORACLE instance (normal)
Wed Mar 26 17:11:17 2025
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
Wed Mar 26 17:11:28 2025
Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST
Autotune of undo retention is turned on. 
IMODE=BR
ILAT =168
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining,
Oracle Database Vault and Real Application Testing options.
Using parameter settings in server-side spfile /apps/svr/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora
System parameters with non-default values:
  processes                = 1000
  sga_target               = 0
  memory_target            = 66048M
  memory_max_target        = 66048M
  control_files            = "/apps/data/oracle/orcl/control01.ctl"
  control_files            = "/apps/svr/oracle/flash_recovery_area/orcl/control02.ctl"
  db_block_size            = 8192
  compatible               = "11.2.0.0.0"
  db_recovery_file_dest    = "/apps/svr/oracle/flash_recovery_area"
  db_recovery_file_dest_size= 3882M
  undo_tablespace          = "UNDOTBS1"
  remote_login_passwordfile= "EXCLUSIVE"
  db_domain                = ""
  dispatchers              = "(PROTOCOL=TCP) (SERVICE=orclXDB)"
  audit_file_dest          = "/apps/svr/oracle/admin/orcl/adump"
  audit_trail              = "DB"
  db_name                  = "orcl"
  open_cursors             = 300
  pga_aggregate_target     = 0
  diagnostic_dest          = "/apps/svr/oracle"
Wed Mar 26 17:11:29 2025
PMON started with pid=2, OS id=28315 
Wed Mar 26 17:11:29 2025
VKTM started with pid=3, OS id=28317 at elevated priority
VKTM running at (10)millisec precision with DBRM quantum (100)ms
Wed Mar 26 17:11:29 2025
GEN0 started with pid=4, OS id=28324 
Wed Mar 26 17:11:29 2025
DIAG started with pid=5, OS id=28326 
Wed Mar 26 17:11:29 2025
DBRM started with pid=6, OS id=28328 
Wed Mar 26 17:11:29 2025
PSP0 started with pid=7, OS id=28330 
Wed Mar 26 17:11:29 2025
DIA0 started with pid=9, OS id=28334 
Wed Mar 26 17:11:29 2025
MMAN started with pid=8, OS id=28336 
Wed Mar 26 17:11:29 2025
DBW0 started with pid=10, OS id=28338 
Wed Mar 26 17:11:29 2025
DBW1 started with pid=11, OS id=28340 
Wed Mar 26 17:11:29 2025
DBW2 started with pid=12, OS id=28342 
Wed Mar 26 17:11:29 2025
DBW3 started with pid=13, OS id=28344 
Wed Mar 26 17:11:29 2025
LGWR started with pid=14, OS id=28346 
Wed Mar 26 17:11:29 2025
CKPT started with pid=15, OS id=28348 
Wed Mar 26 17:11:29 2025
SMON started with pid=16, OS id=28350 
Wed Mar 26 17:11:29 2025
RECO started with pid=17, OS id=28352 
Wed Mar 26 17:11:29 2025
MMON started with pid=18, OS id=28354 
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
Wed Mar 26 17:11:29 2025
MMNL started with pid=19, OS id=28356 
starting up 1 shared server(s) ...
ORACLE_BASE from environment = /apps/svr/oracle
Wed Mar 26 17:11:29 2025
ALTER DATABASE   MOUNT
Wed Mar 26 17:11:32 2025
Sweep [inc][7679]: completed
Sweep [inc][7224]: completed
Sweep [inc][15551]: completed
Sweep [inc2][7679]: completed
Sweep [inc2][7678]: completed
Sweep [inc2][7224]: completed
Sweep [inc2][15551]: completed
Successful mount of redo thread 1, with mount id 1724539585
Database mounted in Exclusive Mode
Lost write protection disabled
Completed: ALTER DATABASE   MOUNT
Wed Mar 26 17:11:34 2025
ALTER DATABASE OPEN
Errors in file /apps/svr/oracle/diag/rdbms/orcl/orcl/trace/orcl_ora_28406.trc:
ORA-01113: file 17 needs media recovery
ORA-01110: data file 17: '/apps/data/oracle/XFF_MON/XFF_MON_2016.dbf'
ORA-1113 signalled during: ALTER DATABASE OPEN...

由于现场已经破坏,无法分析当时库的情况和17号文件的具体情况做进一步判断,只能通过日志记录下这个类型的错误.
在oracle中关于ORA-600 krhpfh_03的bug也比较多