ORA-600 krse_arc_complete.4

11.2.0.4版本数据库对于虚拟化环境中的备库进行克隆,然后尝试failover方式激活备库,结果遭遇ORA-600 krse_arc_complete.4错误

SQL> ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE;
ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE
*
ERROR at line 1:
ORA-01196: file 1 is inconsistent due to a failed media recovery session
ORA-01110: data file 1: '/u01/app/oracle/oradata/hisdb/system.256.975233377'

alert日志显示

Wed Oct 23 21:45:46 2024
ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE
ALTER DATABASE ACTIVATE [PHYSICAL] STANDBY DATABASE (hisdb)
Begin: Standby Redo Logfile archival
End: Standby Redo Logfile archival
Wed Oct 23 21:45:46 2024
ARC0: Detected ARCH process failure
ARC0: STARTING ARCH PROCESSES
Wed Oct 23 21:45:46 2024
ARC3 started with pid=22, OS id=28848 
ARC3: Archival started
ARC0: STARTING ARCH PROCESSES COMPLETE
Beginning Standby Crash Recovery.
Serial Media Recovery started
Managed Standby Recovery starting Real Time Apply
Errors in file /u01/app/oracle/diag/rdbms/hisdbdg/hisdb/trace/hisdb_arc0_28695.trc  (incident=528155):
ORA-00600: internal error code, arguments: [krse_arc_complete.4], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/hisdbdg/hisdb/incident/incdir_528155/hisdb_arc0_28695_i528155.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Master archival failure: 600
Wed Oct 23 21:45:47 2024
Deleted Oracle managed file /fra/fast_recovery_area/HISDBDG/archivelog/2024_10_23/o1_mf_2_158418_mkkzjbqt_.arc
Warning: Datafile 1 (/u01/app/oracle/oradata/hisdb/system.256.975233377) is infinitely media recovery fuzzy
Standby database will not open with this datafile online!
Standby Crash Recovery aborted due to error 10554.
Errors in file /u01/app/oracle/diag/rdbms/hisdbdg/hisdb/trace/hisdb_ora_28842.trc:
ORA-10554: Media recovery failed to bring datafile 1 to a consistent point
ORA-01110: data file 1: '/u01/app/oracle/oradata/hisdb/system.256.975233377'
Completed Standby Crash Recovery.
Wed Oct 23 21:45:47 2024
Errors in file /u01/app/oracle/diag/rdbms/hisdbdg/hisdb/trace/hisdb_arc2_28840.trc  (incident=528172):
ORA-00600: internal error code, arguments: [krse_arc_complete.4], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/hisdbdg/hisdb/incident/incdir_528172/hisdb_arc2_28840_i528172.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Master archival failure: 600
ORA-1196 signalled during: ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE...
ARC3: Detected ARCH process failure

查询mos 发现是尝试应用standby log,并且arc进程尝试对其进行归档,发现归档失败从而报该错误,可以尝试对standby log进行clear,然后再激活备库

SQL> select group#,status from v$standby_log;

          GROUP# STATUS
---------------- ----------
              11 UNASSIGNED
              12 UNASSIGNED
              13 UNASSIGNED
              14 ACTIVE
              15 UNASSIGNED
              16 UNASSIGNED
              17 UNASSIGNED
              18 UNASSIGNED

8 rows selected.

SQL> alter database clear unarchived logfile group 14;

Database altered.

SQL> ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE;

Database altered.

Wed Oct 23 21:55:12 2024
ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE
ALTER DATABASE ACTIVATE [PHYSICAL] STANDBY DATABASE (hisdb)
Begin: Standby Redo Logfile archival
End: Standby Redo Logfile archival
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
Standby terminal recovery start SCN: 35738445353
RESETLOGS after incomplete recovery UNTIL CHANGE 35738178180
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST
Resetting resetlogs activation ID 1887849281 (0x70864b41)
Online log /u01/app/oracle/oradata/hisdb/group_5.272.976991793: Thread 1 Group 5 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_5.2338.976991793: Thread 1 Group 5 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_6.273.976991805: Thread 1 Group 6 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_6.2339.976991805: Thread 1 Group 6 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_7.274.976991825: Thread 2 Group 7 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_7.2336.976991825: Thread 2 Group 7 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_8.275.976991863: Thread 2 Group 8 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_8.2337.976991863: Thread 2 Group 8 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_9.276.976991877: Thread 1 Group 9 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_9.2334.976991877: Thread 1 Group 9 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_10.277.976991891: Thread 2 Group 10 was previously cleared
Online log /u01/app/oracle/oradata/hisdb/group_10.2333.976991893: Thread 2 Group 10 was previously cleared
Standby became primary SCN: 35738445352
Wed Oct 23 21:55:21 2024
Setting recovery target incarnation to 3
ACTIVATE STANDBY: Complete - Database mounted as primary
Completed: ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE

参考:Activate Standby Database failed with ORA-00600: [krse_arc_complete.4] (Doc ID 2409336.1)

发表在 Data Guard | 标签为 , | 留下评论

Oracle 19c 202410补丁(RUs+OJVM)

19.0.0.0
 Description  Database Update  GI Update  Windows Bundle Patch
 OCT2024 (19.25.0.0.0) 36912597  36916690  36878821
 JUL2024 (19.24.0.0.0) 36582781  36582629  36521936
 APR2024 (19.23.0.0.0) 36233263  36233126  36219938
 JAN2024 (19.22.0.0.0) 35943157  35940989  35962832
 OCT2023 (19.21.0.0.0) 35643107  35642822  35681552
 JUL2023 (19.20.0.0.0) 35320081  35319490  35348034
 APR2023 (19.19.0.0.0) 35042068  35037840  35046439
 JAN2023 (19.18.0.0.0) 34765931  34762026  34750795
 Oct2022 (19.17.0.0.0) 34419443  34416665  34468114
 JUL2022 (19.16.0.0.0) 34133642  34130714  34110685
 APR2022 (19.15.0.0.0) 33806152  33803476  33829175
 JAN2022 (19.14.0.0.0) 33515361  33509923  33575656
 OCT2021(19.13.0.0.0) 33192793  33182768  33155330
 JUL2021 (19.12.0.0.0) 32904851  32895426  32832237
 APR2021 (19.11.0.0.0) 32545013  32545008  32409154
 JAN2021 (19.10.0.0.0) 32218454  32226239  32062765
 OCT2020 (19.9.0.0.0) 31771877  31750108  31719903
 JUL2020  (19.8.0.0.0) 31281355  31305339  31247621
 APR2020 (19.7.0.0.0) 30869156  30899722  30901317
 JAN2020 (19.6.0.0.0) 30557433  30501910  30445947
 OCT2019 (19.5.0.0.0) 30125133  30116789  30151705
 JUL2019 (19.4.0.0.0) 29834717  29708769   NA
 APR2019 (19.3.0.0.0) 29517242  29517302   NA
19.0.0.0
 Description  OJVM Update  OJVM + DB Update  OJVM + GI Update
 OCT2024 (19.25.0.0.241015)  36878697  36866623  36866740
 JUL2024 (19.24.0.0.240716)  36414915  36522340  36522439
 APR2024 (19.23.0.0.240416)  36199232  36209492  36209493
 JAN2024 (19.22.0.0.240116)  35926646  36031426  36031453
 OCT2023 (19.21.0.0.231017)  35648110  35742413  35742441
 JUL2023 (19.20.0.0.230718)  35354406  35370174  35370167
 APR2023 (19.19.0.0.230418)  35050341  35058163  35058172
 JAN2023 (19.18.0.0.230117)  34786990  34773489  34773504
 OCT2022 (19.17.0.0.221018)  34411846  34449114  34449117
 JUL2022 (19.16.0.0.220719)  34086870  34160831  34160854
 APR2022 (19.15.0.0.220419)  33808367  33859194  33859214
 JAN2022 (19.14.0.0.220118)  33561310  33567270  33567274
 OCT2021 (19.13.0.0.211019)  33192694  33248420  33248471
 JUL2021 (19.12.0.0.210720)  32876380  32900021  32900083
 APR2021 (19.11.0.0.210420)  32399816  32578972  32578973
 JAN2021 (19.10.0.0.210119)  32067171  32126828  32126842
 OCT2020 (19.9.0.0.201020)  31668882  31720396  31720429
 JUL2020 (19.8.0.0.200714)  31219897  31326362  31326369
 APR2020 (19.7.0.0.200414)  30805684  30783543  30783556
 JAN2020 (19.6.0.0.200114)  30484981  30463595  30463609
 OCT2019 (19.5.0.0.191015)  30128191  30133124  30133178
 JUL2019 (19.4.0.0.190716)  29774421  29699079  29699097
 APR2019 (19.3.0.0.190416)  29548437  29621253  29621299

参考:Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (Doc ID 2118136.2)

发表在 Oracle安装升级 | 标签为 | 留下评论

ntfs MFT损坏(ntfs文件系统故障)导致oracle异常恢复

客户虚拟化环境,由于断电,启动数据库报ORA-01157错误,通过操作系统层面查看,发现文件是存在的,但是dbv检测报不可访问
ora-01157


感觉是文件系统损坏了,尝试把该文件拷贝到其他磁盘
221509

查看操作系统事件,确认是ntfs文件系统的MFT损坏
mft

基于这种情况,通过文件系统恢复工具进行恢复该文件尝试,提示恢复文件大小和实际元数据中记录大小不一致
214712

通过对比实际恢复大小和文件本身大小,发现7811899392-7791460352,几乎等于20M大小(也就是说恢复出来的数据文件少了20M),通过分析数据库alert日志,确认该系统在前端时间刚好扩展了20M(增加数据文件之时指定了每次扩展20m)

2023-08-11T11:29:21.397236+08:00
ALTER TABLESPACE "HSHIS" ADD DATAFILE
'D:\APP\ADMINISTRATOR\ORADATA\HIS\HSHIS01.DBF' SIZE 10M AUTOEXTEND ON NEXT 20M MAXSIZE 8001M
Completed: ALTER TABLESPACE "HSHIS" ADD DATAFILE
'D:\APP\ADMINISTRATOR\ORADATA\HIS\HSHIS01.DBF' SIZE 10M AUTOEXTEND ON NEXT 20M MAXSIZE 8001M

2024-10-09T00:18:31.058537+08:00
Resize operation completed for file# 66, old size 7608320K, new size 7628800K

通过对该文件底层block分析,确认最终丢失block就是最后20M(直接的数据文件的block的rdba均正确),对于这种故障,通过填补数据文件尾部,欺骗数据库完成该文件的恢复(最后20M中如果写入了业务数据,可能会丢失),做好该文件修复工作之后,尝试打开数据库,结果很不乐观,redo也损坏
recover-error


屏蔽一致性,强制打开库成功

2024-10-18T04:24:43.911107+08:00
ALTER DATABASE RECOVER    CANCEL  
2024-10-18T04:24:47.098637+08:00
Errors in file E:\TRACE\diag\rdbms\his\his\trace\his_pr00_2608.trc:
ORA-01547: 警告: RECOVER 成功但 OPEN RESETLOGS 将出现如下错误
ORA-01194: 文件 1 需要更多的恢复来保持一致性
ORA-01110: 数据文件 1: 'E:\ORADATA\SYSTEM01.DBF'
2024-10-18T04:24:47.114278+08:00
ORA-1547 signalled during: ALTER DATABASE RECOVER    CANCEL  ...
ALTER DATABASE RECOVER CANCEL 
ORA-1112 signalled during: ALTER DATABASE RECOVER CANCEL ...
2024-10-18T04:25:03.989398+08:00
alter database open resetlogs
2024-10-18T04:25:05.598781+08:00
RESETLOGS is being done without consistancy checks. This may result
in a corrupted database. The database should be recreated.
RESETLOGS after incomplete recovery UNTIL CHANGE 2666786639 time 
Resetting resetlogs activation ID 3659241623 (0xda1b9897)
2024-10-18T04:25:12.380089+08:00
Setting recovery target incarnation to 3
2024-10-18T04:25:15.052071+08:00
Ping without log force is disabled:
  instance mounted in exclusive mode.
Endian type of dictionary set to little
2024-10-18T04:25:15.458286+08:00
Assigning activation ID 3703362676 (0xdcbcd474)
2024-10-18T04:25:15.505102+08:00
TT00 (PID:4092): Gap Manager starting
2024-10-18T04:25:15.551992+08:00
Redo log for group 1, sequence 1 is not located on DAX storage
2024-10-18T04:25:17.833250+08:00
Thread 1 opened at log sequence 1
  Current log# 1 seq# 1 mem# 0: E:\ORADATA\REDO01.LOG
Successful open of redo thread 1
2024-10-18T04:25:17.848888+08:00
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
stopping change tracking
2024-10-18T04:25:22.052035+08:00
Undo initialization recovery: err:0 start: 24275578 end: 24276578 diff: 1000 ms (1.0 seconds)
Undo initialization online undo segments: err:0 start: 24276578 end: 24276593 diff: 15 ms (0.0 seconds)
Undo initialization finished serial:0 start:24275578 end:24276640 diff:1062 ms (1.1 seconds)
Dictionary check beginning
Dictionary check complete
Verifying minimum file header compatibility for tablespace encryption..
Verifying file header compatibility for tablespace encryption completed for pdb 0
2024-10-18T04:25:23.114610+08:00
Database Characterset is AL32UTF8
No Resource Manager plan active
2024-10-18T04:25:29.036475+08:00
replication_dependency_tracking turned off (no async multimaster replication found)
2024-10-18T04:25:32.833386+08:00
LOGSTDBY: Validating controlfile with logical metadata
LOGSTDBY: Validation complete
Starting background process AQPC
2024-10-18T04:25:33.145881+08:00
AQPC started with pid=37, OS id=5560 
2024-10-18T04:25:35.677167+08:00
Starting background process CJQ0
2024-10-18T04:25:35.708430+08:00
CJQ0 started with pid=39, OS id=2728 
2024-10-18T04:25:36.724036+08:00
Completed: alter database open resetlogs

然后导出数据到新库,其中遇到了file# 66号文件最后丢失的20M引起的数据无法正常导出的问题处理(丢弃损坏部分数据,把剩余好的表中数据恢复到新库中)

发表在 Oracle备份恢复 | 标签为 | 留下评论