标签云
asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2131 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (102)
- 数据库 (1,671)
- DB2 (22)
- MySQL (73)
- Oracle (1,533)
- Data Guard (52)
- EXADATA (8)
- GoldenGate (21)
- ORA-xxxxx (159)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (14)
- ORACLE 21C (3)
- Oracle 23ai (7)
- Oracle ASM (65)
- Oracle Bug (8)
- Oracle RAC (52)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (28)
- Oracle备份恢复 (560)
- Oracle安装升级 (92)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (78)
- PostgreSQL (18)
- PostgreSQL恢复 (6)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (37)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (20)
-
最近发表
- Kylin Linux 安装19c
- ORA-600 krse_arc_complete.4
- Oracle 19c 202410补丁(RUs+OJVM)
- ntfs MFT损坏(ntfs文件系统故障)导致oracle异常恢复
- .mkp扩展名oracle数据文件加密恢复
- 清空redo,导致ORA-27048: skgfifi: file header information is invalid
- A_H_README_TO_RECOVER勒索恢复
- 通过alert日志分析客户自行对一个数据库恢复的来龙去脉和点评
- ORA-12514: TNS: 监听进程不能解析在连接描述符中给出的SERVICE_NAME
- ORA-01092 ORA-00604 ORA-01558故障处理
- ORA-65088: database open should be retried
- Oracle 19c异常恢复—ORA-01209/ORA-65088
- ORA-600 16703故障再现
- 数据库启动报ORA-27102 OSD-00026 O/S-Error: (OS 1455)
- .[metro777@cock.li].Elbie勒索病毒加密数据库恢复
- 应用连接错误,初始化mysql数据库恢复
- RAC默认服务配置优先节点
- Oracle 19c RAC 替换私网操作
- 监听报TNS-12541 TNS-12560 TNS-00511错误
- drop tablespace xxx including contents恢复
分类目录归档:Oracle备份恢复
分布式存储故障导致数据库无法启动故障处理
国内xx医院使用了国外医疗行业龙头的pacs系统,由于是一个历史库,存放在分布式存储中,由于存储同时多个节点故障,导致数据库多个文件异常,数据库无法启动,三方维护人员尝试通通过rman归档进行应用日志,结果发现日志有损坏报ORA-00354 ORA-00353,无法记录恢复,希望我们给予支持
Mon Apr 29 13:28:40 2024 Media Recovery failed with error 354 Mon Apr 29 13:28:40 2024 Errors in file F:\XXXXXX_DB\ORACLE\ADMIN\diag\rdbms\xxx\msxxx1\trace\xxx_pr00_4568.trc: ORA-00283: recovery session canceled due to errors ORA-00354: corrupt redo log block header ORA-00353: log corruption near block 487424 change 8737273868 time 04/01/2024 01:38:25 ORA-00334: archived log: 'F:\XXXXXX_DB\ORADATA\XXX\LOGARC0000052184_0922116268.0001' ORA-283 signalled during: alter database recover logfile 'F:\XXXXXX_DB\ORADATA\XXX\LOGARC0000052184_0922116268.0001'...
接手故障之后,通过尝试恢复发现除该错误之外,还有ORA-600 4552之类错误
跳过这些异常文件恢复,最终确认异常文件有如下部分
这些文件由于日志无法正常应用(有日志损坏无法应用,有日志和数据文件block不匹配导致无法应用),这样的情况直接通过自研的Oracle Recovery Tools小工具直接修改文件头信息
然后尝试OPEN数据库结果报ORA-1207
对于这个故障可以通过rectl或者using backup ctl方式处理,然后open数据库成功
由于该系统是历史库,不会有新业务写入,通过对异常表和索引进行处理之后,客户测试业务可以正常访问,完成本次恢复
存储故障后oracle报—ORA-01122/ORA-01207故障处理
客户存储异常,通过硬件恢复解决存储故障之后,oracle数据库无法正常启动(存储cache丢失),尝试recover数据库报ORA-00283 ORA-01122 ORA-01110 ORA-01207错误
以前处理过比较类似的存储故障case:
又一起存储故障导致ORA-00333 ORA-00312恢复
存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理
SQL> recover database until cancel; ORA-00283: 恢复会话因错误而取消 ORA-01122: 数据库文件 536 验证失败 ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora' ORA-01207: 文件比控制文件更新 - 旧的控制文件
Sun May 05 00:09:03 2024 ALTER DATABASE RECOVER database until cancel Media Recovery Start started logmerger process Sun May 05 00:09:10 2024 SUCCESS: diskgroup FRA was mounted Sun May 05 00:09:10 2024 NOTE: dependency between database orcl and diskgroup resource ora.FRA.dg is established Sun May 05 00:09:14 2024 WARNING! Recovering data file 1 from a fuzzy backup. It might be an online backup taken without entering the begin backup command. Media Recovery failed with error 1122 Slave exiting with ORA-283 exception Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_pr00_8048.trc: ORA-00283: 恢复会话因错误而取消 ORA-01122: 数据库文件 536 验证失败 ORA-01110: 数据文件 536: '+DATA/orcl/dt_img_dat511.ora' ORA-01207: 文件比控制文件更新 - 旧的控制文件 Sun May 05 00:09:16 2024 Recovery Slave PR00 previously exited with exception 283 ORA-283 signalled during: ALTER DATABASE RECOVER database until cancel ...
using backup controlfile进行恢复
SQL> recover database using backup controlfile until cancel; ORA-00279: 更改 18646239951 (在 04/25/2024 17:14:50 生成) 对于线程 1 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505 ORA-00280: 更改 18646239951 (用于线程 1) 在序列 #1003886 中 指定日志: {<RET>=suggested | filename | AUTO | CANCEL} auto ORA-00279: 更改 18646239951 (在 04/25/2024 17:11:40 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807 ORA-00280: 更改 18646239951 (用于线程 2) 在序列 #677876 中 ORA-00279: 更改 18646255791 (在 04/25/2024 17:16:46 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099 ORA-00280: 更改 18646255791 (用于线程 2) 在序列 #677877 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677876.199531.1167239807' ORA-00279: 更改 18646295647 (在 04/25/2024 17:21:38 生成) 对于线程 2 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_2_seq_677878.199379.1167240623 ORA-00280: 更改 18646295647 (用于线程 2) 在序列 #677878 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_2_seq_677877.199472.1167240099' ORA-00279: 更改 18646331784 (在 04/25/2024 17:28:25 生成) 对于线程 1 是必需的 ORA-00289: 建议: +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507 ORA-00280: 更改 18646331784 (用于线程 1) 在序列 #1003887 中 ORA-00278: 此恢复不再需要日志文件 '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003886.199435.1167240505' ORA-00308: 无法打开归档日志 '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507' ORA-17503: ksfdopn: 2 未能打开文件 +FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507 ORA-15012: ASM file '+FRA/orcl/archivelog/2024_04_25/thread_1_seq_1003887.199320.1167241507' does not exist ORA-10879: error signaled in parallel recovery slave ORA-01547: 警告: RECOVER 成功但 OPEN RESETLOGS 将出现如下错误 ORA-01194: 文件 1 需要更多的恢复来保持一致性 ORA-01110: 数据文件 1: '+DATA/orcl/system01.dbf'
通过分析,确认由于cache丢失导致thread_1_seq_1003887这个日志丢失(而且redo已经被覆盖)
数据库无法通过正常recover的思路解决.通过屏蔽一致性,强制打开数据库,alert日志报ORA-600 2662错误
Sat May 04 17:23:00 2024 Redo thread 2 internally disabled at seq 1 (CKPT) ARC1: Archiving disabled thread 2 sequence 1 Archived Log entry 2 added for thread 2 sequence 1 ID 0x0 dest 1: ARC3: Archival started ARC0: STARTING ARCH PROCESSES COMPLETE Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc (incident=47066): ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc: ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_ora_3684.trc: ORA-00600: ??????, ??: [2662], [4], [1466533588], [4], [1466584862], [12583040], [], [], [], [], [], [] Error 600 happened during db open, shutting down database USER (ospid: 3684): terminating the instance due to error 600 Instance terminated by USER, pid = 3684 ORA-1092 signalled during: alter database open resetlogs...
通过修改数据库scn,open数据库报ORA-600 4137
Sun May 05 00:12:41 2024 replication_dependency_tracking turned off (no async multimaster replication found) LOGSTDBY: Validating controlfile with logical metadata LOGSTDBY: Validation complete Completed: alter database open resetlogs Sun May 05 00:12:56 2024 Trace dumping is performing id=[cdmp_20240505001256] Sun May 05 00:12:56 2024 ORACLE Instance orcl1 (pid = 22) - Error 600 encountered while recovering transaction (28, 21). Errors in file d:\app\administrator\diag\rdbms\orcl\orcl1\trace\orcl1_smon_5896.trc: ORA-00600: ??????, ??: [4137], [28.21.42965783], [0], [0], [], [], [], [], [], [], [], []
这个错误比较明显,由于28号回滚段异常导致,对异常回滚段进行处理,重建undo,数据库恢复主要工作完成
Oracle 23ai rm redo*.log恢复
在oracle 23ai的pdb中创建用户和表,并且插入数据(不提交),在另外一个会话中abort库,并从os层面rm删除掉redo文件,模拟数据库当前redo丢失,数据库恢复
创建用户和表并插入数据
[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:40:55 2024 Version 23.4.0.24.05 Copyright (c) 1982, 2024, Oracle. All rights reserved. Connected to: Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free Version 23.4.0.24.05 SQL> SQL> SQL> show pdbs; CON_ID CON_NAME OPEN MODE RESTRICTED ---------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 FREEPDB1 READ WRITE NO SQL> alter session set container=FREEPDB1; Session altered. SQL> create user xff identified by oracle; User created. SQL> grant dba to xff ; Grant succeeded. SQL> conn xff/oracle@FREEPDB1 Connected. SQL> create table t1 as select * from dba_objects; Table created. SQL> insert into t1 select *from t1; 75877 rows created. SQL> / 151754 rows created.
另外一个会话中abort库
[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:43:30 2024 Version 23.4.0.24.05 Copyright (c) 1982, 2024, Oracle. All rights reserved. Connected to: Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free Version 23.4.0.24.05 SQL> shutdown abort; ORACLE instance shut down. SQL> SQL> SQL> SQL> exit Disconnected from Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free Version 23.4.0.24.05
操作系统层面rm -rf 删除redo
[oracle@xifenfei ~]$ cd $ORACLE_BASE/oradata [oracle@xifenfei oradata]$ ls FREE [oracle@xifenfei oradata]$ cd FREE/ [oracle@192 FREE]$ ls control01.ctl FREEPDB1 redo01.log redo03.log system01.dbf undotbs2.dbf control02.ctl pdbseed redo02.log sysaux01.dbf temp01.dbf users01.dbf [oracle@xifenfei FREE]$ ls -ltr total 2441036 drwxr-x---. 2 oracle 1000 85 May 1 16:49 pdbseed -rw-r-----. 1 oracle oinstall 20979712 May 1 16:51 temp01.dbf drwxr-x---. 2 oracle 1000 104 May 1 16:55 FREEPDB1 -rw-r-----. 1 oracle oinstall 209715712 May 3 15:23 redo01.log -rw-r-----. 1 oracle oinstall 209715712 May 3 15:23 redo02.log -rw-r-----. 1 oracle oinstall 7348224 May 3 15:23 users01.dbf -rw-r-----. 1 oracle oinstall 1080041472 May 3 15:43 system01.dbf -rw-r-----. 1 oracle oinstall 692068352 May 3 15:43 sysaux01.dbf -rw-rw----. 1 oracle oinstall 52436992 May 3 15:43 undotbs2.dbf -rw-r-----. 1 oracle oinstall 209715712 May 3 15:43 redo03.log -rw-r-----. 1 oracle oinstall 18759680 May 3 15:43 control01.ctl -rw-r-----. 1 oracle oinstall 18759680 May 3 15:43 control02.ctl [oracle@xifenfei FREE]$ rm -rf redo0* [oracle@192 FREE]$ ls -l redo* ls: cannot access 'redo*': No such file or directory [oracle@xifenfei FREE]$
尝试启动数据库,报ora-00313,ora-00312,ora-27037等错误
[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:44:17 2024 Version 23.4.0.24.05 Copyright (c) 1982, 2024, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount; ORACLE instance started. Total System Global Area 1603726344 bytes Fixed Size 5360648 bytes Variable Size 671088640 bytes Database Buffers 922746880 bytes Redo Buffers 4530176 bytes Database mounted. SQL> recover database; ORA-00283: recovery session canceled due to errors ORA-00313: open failed for members of log group 3 of thread 1 ORA-00312: online log 3 thread 1: '/opt/oracle/oradata/FREE/redo03.log' ORA-27037: unable to obtain file status Linux-x86_64 Error: 2: No such file or directory Additional information: 7
由于所有redo均被删除(包含当前redo),因此只能强制resetlogs方式打开库,尝试打开数据库
SQL> shutdown immediate; ORA-01109: database not open Database dismounted. ORACLE instance shut down. SQL> startup mount pfile='/tmp/pfile' ORACLE instance started. Total System Global Area 1603726344 bytes Fixed Size 5360648 bytes Variable Size 671088640 bytes Database Buffers 922746880 bytes Redo Buffers 4530176 bytes Database mounted. SQL> recover database until cancel; ORA-00279: change 4244588 generated at 05/03/2024 15:23:22 needed for thread 1 ORA-00289: suggestion : /opt/oracle/product/23ai/dbhomeFree/dbs/arch1_6_1167842962.dbf ORA-00280: change 4244588 for thread 1 is in sequence #6 Specify log: {<RET>=suggested | filename | AUTO | CANCEL} cancel ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below ORA-01194: file 1 needs more recovery to be consistent ORA-01110: data file 1: '/opt/oracle/oradata/FREE/system01.dbf' ORA-01112: media recovery not started SQL> alter database open resetlogs; alter database open resetlogs * ERROR at line 1: ORA-00603: ORACLE server session terminated by irrecoverable error ORA-01092: ORACLE instance terminated. Disconnection forced ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [], [], [], [], [], [], [], [] Process ID: 5596 Session ID: 29 Serial number: 63204
数据库遇到常见的ORA-600 kcbzib_kcrsds_1错误,这类错误类似oracle在12c之前版本中的ORA-600 2662错误
ORA-600 kcbzib_kcrsds_1报错
存储故障,强制拉库报ORA-600 kcbzib_kcrsds_1处理
redo异常强制拉库报ORA-600 kcbzib_kcrsds_1修复
ORA-00603 ORA-01092 ORA-600 kcbzib_kcrsds_1
这种一般就是scn问题,通过修改数据库scn,数据库启动报ORA-16433错误
[oracle@xifenfei ~]$ sqlplus sys/oracle@free as sysdba SQL*Plus: Release 23.0.0.0.0 - Production on Fri May 3 15:49:00 2024 Version 23.4.0.24.05 Copyright (c) 1982, 2024, Oracle. All rights reserved. Connected to an idle instance. SQL> startup nomount pfile='/tmp/pfile'; ORACLE instance started. Total System Global Area 1603726344 bytes Fixed Size 5360648 bytes Variable Size 671088640 bytes Database Buffers 922746880 bytes Redo Buffers 4530176 bytes SQL> alter database mount; Database altered. SQL> recover database; ORA-00283: recovery session canceled due to errors ORA-16433: The database or pluggable database must be opened in read/write mode. SQL> shutdown abort; ORACLE instance shut down.
处理ctl,open数据库成功
SQL> startup nomount pfile='/tmp/pfile' ORACLE instance started. Total System Global Area 1603726344 bytes Fixed Size 5360648 bytes Variable Size 671088640 bytes Database Buffers 922746880 bytes Redo Buffers 4530176 bytes SQL> CREATE CONTROLFILE REUSE DATABASE "FREE" NORESETLOGS NOARCHIVELOG 2 MAXLOGFILES 16 3 MAXLOGMEMBERS 3 4 MAXDATAFILES 1024 5 MAXINSTANCES 8 6 MAXLOGHISTORY 292 7 LOGFILE 8 GROUP 1 '/opt/oracle/oradata/FREE/redo01.log' SIZE 200M BLOCKSIZE 512, 9 GROUP 2 '/opt/oracle/oradata/FREE/redo02.log' SIZE 200M BLOCKSIZE 512, 10 GROUP 3 '/opt/oracle/oradata/FREE/redo03.log' SIZE 200M BLOCKSIZE 512 11 -- STANDBY LOGFILE 12 DATAFILE 13 '/opt/oracle/oradata/FREE/system01.dbf', 14 '/opt/oracle/oradata/FREE/pdbseed/system01.dbf', 15 '/opt/oracle/oradata/FREE/sysaux01.dbf', 16 '/opt/oracle/oradata/FREE/pdbseed/sysaux01.dbf', 17 '/opt/oracle/oradata/FREE/users01.dbf', 18 '/opt/oracle/oradata/FREE/pdbseed/undotbs01.dbf', 19 '/opt/oracle/oradata/FREE/FREEPDB1/system01.dbf', 20 '/opt/oracle/oradata/FREE/FREEPDB1/sysaux01.dbf', 21 '/opt/oracle/oradata/FREE/FREEPDB1/undotbs01.dbf', 22 '/opt/oracle/oradata/FREE/FREEPDB1/users01.dbf', 23 '/opt/oracle/oradata/FREE/undotbs2.dbf' 24 CHARACTER SET AL32UTF8 25 ; Control file created. SQL> recover database; Media recovery complete. SQL> alter database open; Database altered. SQL> show pdbs; CON_ID CON_NAME OPEN MODE RESTRICTED ---------------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 FREEPDB1 READ WRITE NO
至此当前数据库redo故障模拟和恢复基本完成