-
加我微信
加我QQ
标签云
asm恢复 bbed bootstrap$ dul In Memory kcbzib_kcrsds_1 kccpb_sanity_check_2 kfed MySQL恢复 ORA-00312 ORA-00607 ORA-00704 ORA-01110 ORA-01555 ORA-01578 ORA-08103 ORA-600 2131 ORA-600 2662 ORA-600 2663 ORA-600 3020 ORA-600 4000 ORA-600 4137 ORA-600 4193 ORA-600 4194 ORA-600 16703 ORA-600 kcbzib_kcrsds_1 ORA-600 KCLCHKBLK_4 ORA-15042 ORA-15196 ORACLE 12C oracle dul ORACLE PATCH Oracle Recovery Tools oracle加密恢复 oracle勒索 oracle勒索恢复 oracle异常恢复 Oracle 恢复 ORACLE恢复 ORACLE数据库恢复 oracle 比特币 OSD-04016 YOUR FILES ARE ENCRYPTED 勒索恢复 比特币加密文章分类
- Others (2)
- 中间件 (2)
- WebLogic (2)
- 操作系统 (102)
- 数据库 (1,682)
- DB2 (22)
- MySQL (73)
- Oracle (1,544)
- Data Guard (52)
- EXADATA (8)
- GoldenGate (24)
- ORA-xxxxx (159)
- ORACLE 12C (72)
- ORACLE 18C (6)
- ORACLE 19C (15)
- ORACLE 21C (3)
- Oracle 23ai (7)
- Oracle ASM (67)
- Oracle Bug (8)
- Oracle RAC (53)
- Oracle 安全 (6)
- Oracle 开发 (28)
- Oracle 监听 (28)
- Oracle备份恢复 (565)
- Oracle安装升级 (92)
- Oracle性能优化 (62)
- 专题索引 (5)
- 勒索恢复 (79)
- PostgreSQL (18)
- PostgreSQL恢复 (6)
- SQL Server (27)
- SQL Server恢复 (8)
- TimesTen (7)
- 达梦数据库 (2)
- 生活娱乐 (2)
- 至理名言 (11)
- 虚拟化 (2)
- VMware (2)
- 软件开发 (37)
- Asp.Net (9)
- JavaScript (12)
- PHP (2)
- 小工具 (20)
-
最近发表
- 断电引起的ORA-08102: 未找到索引关键字, 对象号 39故障处理
- ORA-00227: corrupt block detected in control file
- 手工删除19c rac
- 解决oracle数据文件路径有回车故障
- .wstop扩展名勒索数据库恢复
- Oracle Recovery Tools工具一键解决ORA-00376 ORA-01110故障（文件offline）
- OGG-02771 Input trail file format RELEASE 19.1 is different from previous trail file form at RELEASE 11.2.
- OGG-02246 Source redo compatibility level 19.0.0 requires trail FORMAT 12.2 or higher
- GoldenGate 19安装和打patch
- dd破坏asm磁盘头恢复
- 删除asmlib磁盘导致磁盘组故障恢复
- Kylin Linux 安装19c
- ORA-600 krse_arc_complete.4
- Oracle 19c 202410补丁（RUs+OJVM）
- ntfs MFT损坏(ntfs文件系统故障)导致oracle异常恢复
- .mkp扩展名oracle数据文件加密恢复
- 清空redo,导致ORA-27048: skgfifi: file header information is invalid
- A_H_README_TO_RECOVER勒索恢复
- 通过alert日志分析客户自行对一个数据库恢复的来龙去脉和点评
- ORA-12514: TNS: 监听进程不能解析在连接描述符中给出的SERVICE_NAME
标签归档：ora.storage无法启动
ora.storage无法启动报ORA-12514故障处理
19.11集群，节点2人工重启之后，crs启动异常
[grid@xff2 ~]$ crsctl status res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE xff2 STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE xff2 STABLE
ora.crf
1 ONLINE ONLINE xff2 STABLE
ora.crsd
1 ONLINE OFFLINE STABLE
ora.cssd
1 ONLINE ONLINE xff2 STABLE
ora.cssdmonitor
1 ONLINE ONLINE xff2 STABLE
ora.ctssd
1 ONLINE ONLINE xff2 OBSERVER,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.drivers.acfs
1 ONLINE ONLINE xff2 STABLE
ora.evmd
1 ONLINE ONLINE xff2 STABLE
ora.gipcd
1 ONLINE ONLINE xff2 STABLE
ora.gpnpd
1 ONLINE ONLINE xff2 STABLE
ora.mdnsd
1 ONLINE ONLINE xff2 STABLE
ora.storage
1 ONLINE OFFLINE STABLE
--------------------------------------------------------------------------------
crs的alert日志显示
2024-03-05 12:46:26.021 [CLSECHO(3653)]ACFS-9327: Verifying ADVM/ACFS devices.
2024-03-05 12:46:26.040 [CLSECHO(3661)]ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
2024-03-05 12:46:26.065 [CLSECHO(3673)]ACFS-9156: Detecting control device '/dev/ofsctl'.
2024-03-05 12:46:26.357 [CLSECHO(3703)]ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
2024-03-05 12:46:26.376 [CLSECHO(3711)]ACFS-9322: completed
2024-03-05 12:46:27.764 [CSSDMONITOR(3855)]CRS-8500: Oracle Clusterware CSSDMONITOR process is starting with operating system process ID 3855
2024-03-05 12:46:27.839 [OSYSMOND(3857)]CRS-8500: Oracle Clusterware OSYSMOND process is starting with operating system process ID 3857
2024-03-05 12:46:28.129 [CSSDAGENT(3890)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 3890
2024-03-05 12:46:29.125 [OCSSD(3910)]CRS-8500: Oracle Clusterware OCSSD process is starting with operating system process ID 3910
2024-03-05 12:46:30.187 [OCSSD(3910)]CRS-1713: CSSD daemon is started in hub mode
2024-03-05 12:46:31.428 [OCSSD(3910)]CRS-1707: Lease acquisition for node xff2 number 2 completed
2024-03-05 12:46:32.630 [OCSSD(3910)]CRS-1621: The IPMI configuration data for this node stored in the Oracle registry is incomplete; details at (:CSSNK00002:) in /u01/app/grid/diag/crs/xff2/crs/trace/ocssd.trc
2024-03-05 12:46:32.630 [OCSSD(3910)]CRS-1617: The information required to do node kill for node xff2 is incomplete; details at (:CSSNM00004:) in /u01/app/grid/diag/crs/xff2/crs/trace/ocssd.trc
2024-03-05 12:46:32.638 [OCSSD(3910)]CRS-1605: CSSD voting file is online: /dev/sda1; details in /u01/app/grid/diag/crs/xff2/crs/trace/ocssd.trc.
2024-03-05 12:46:33.546 [OCSSD(3910)]CRS-1601: CSSD Reconfiguration complete. Active nodes are xff1 xff2 .
2024-03-05 12:46:35.405 [OCSSD(3910)]CRS-1720: Cluster Synchronization Services daemon (CSSD) is ready for operation.
2024-03-05 12:46:35.533 [OCTSSD(4138)]CRS-8500: Oracle Clusterware OCTSSD process is starting with operating system process ID 4138
2024-03-05 12:46:36.339 [OCTSSD(4138)]CRS-2403: The Cluster Time Synchronization Service on host xff2 is in observer mode.
2024-03-05 12:46:37.601 [OCTSSD(4138)]CRS-2407: The new Cluster Time Synchronization Service reference node is host xff1.
2024-03-05 12:46:37.601 [OCTSSD(4138)]CRS-2401: The Cluster Time Synchronization Service started on host xff2.
2024-03-05 12:46:54.181 [ORAROOTAGENT(2427)]CRS-5019: All OCR locations are on ASM disk groups [SYSTEMDG], and none of these disk groups are mounted. Details are at "(:CLSN00140:)" in "/u01/app/grid/diag/crs/xff2/crs/trace/ohasd_orarootagent_root.trc".
2024-03-05 12:47:15.209 [OLOGGERD(4553)]CRS-8500: Oracle Clusterware OLOGGERD process is starting with operating system process ID 4553
2024-03-05 12:52:04.581 [CRSCTL(8313)]CRS-1013: The OCR location in an ASM disk group is inaccessible. Details in /u01/app/grid/diag/crs/xff2/crs/trace/crsctl_8313.trc.
2024-03-05 12:56:44.519 [ORAROOTAGENT(2427)]CRS-5818: Aborted command 'start' for resource 'ora.storage'. Details at (:CRSAGF00113:) {0:5:3} in /u01/app/grid/diag/crs/xff2/crs/trace/ohasd_orarootagent_root.trc.
2024-03-05 12:56:44.608 [OHASD(2217)]CRS-2757: Command 'Start' timed out waiting for response from the resource 'ora.storage'. Details at (:CRSPE00221:) {0:5:3} in /u01/app/grid/diag/crs/xff2/crs/trace/ohasd.trc.
2024-03-05 12:56:44.606 [ORAROOTAGENT(2427)]CRS-5017: The resource action "ora.storage start" encountered the following error:
2024-03-05 12:56:44.606+agent's abort action pending. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/xff2/crs/trace/ohasd_orarootagent_root.trc".
2024-03-05 12:57:58.464 [CRSD(11801)]CRS-8500: Oracle Clusterware CRSD process is starting with operating system process ID 11801
2024-03-05 12:58:12.059 [CRSD(11801)]CRS-1013: The OCR location in an ASM disk group is inaccessible. Details in /u01/app/grid/diag/crs/xff2/crs/trace/crsd.trc.
ohasd_orarootagent_root 日志
2024-03-05 12:52:00.769 : OCRRAW:4255452928: kgfnConnect3: Got a Connection Error when connecting to ASM. 2024-03-05 12:52:00.771 : OCRRAW:4255452928: kgfnConnect2: failed to connect 2024-03-05 12:52:00.771 : OCRRAW:4255452928: kgfnConnect2Retry: failed to connect connect after 1 attempts, 124s elapsed 2024-03-05 12:52:00.771 : OCRRAW:4255452928: kgfo_kge2slos error stack at kgfoAl06: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor ORA-12514: TNS:listener does not currently know of service requested in connect descriptor 2024-03-05 12:52:00.771 : OCRRAW:4255452928: -- trace dump on error exit -- 2024-03-05 12:52:00.771 : OCRRAW:4255452928: Error [kgfoAl06] in [kgfokge] at kgfo.c:2176 2024-03-05 12:52:00.771 : OCRRAW:4255452928: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor ORA-12514: TNS:listener does not currently know of service requested 2024-03-05 12:52:00.771 : OCRRAW:4255452928: Category: 7 "/u01/app/grid/diag/crs/xff2/crs/trace/crsctl_8313.trc" 208L, 11809C 2024-03-05 12:52:03.543 : OCRRAW:4255452928: 9379 Error 4 opening dom root in 0xf9afdb79c0 2024-03-05 12:52:03.551 : OCRRAW:4255452928: kgfnConnect2: kgfnGetBeqData failed 2024-03-05 12:52:03.577 : OCRRAW:4255452928: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=节点1私网IP)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM))) 2024-03-05 12:52:03.578 : OCRRAW:4255452928: kgfnConnect2Int: ServerAttach 2024-03-05 12:52:04.579 : OCRRAW:4255452928: kgfnServerAttachConnErrors: Encountered service based error 12514 2024-03-05 12:52:04.579 : OCRRAW:4255452928: kgfnRecordErr 12514 OCI error: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor 2024-03-05 12:52:04.579 : OCRRAW:4255452928: kgfnConnect3: Got a Connection Error when connecting to ASM. 2024-03-05 12:52:04.581 : OCRRAW:4255452928: kgfnConnect2: failed to connect 2024-03-05 12:52:04.581 : OCRRAW:4255452928: kgfnConnect2Retry: failed to connect connect after 1 attempts, 122s elapsed 2024-03-05 12:52:04.581 : OCRRAW:4255452928: kgfo_kge2slos error stack at kgfoAl06: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor ORA-12514: TNS:listener does not currently know of service requested in connect descriptor 2024-03-05 12:52:04.581 : OCRRAW:4255452928: -- trace dump on error exit -- 2024-03-05 12:52:04.581 : OCRRAW:4255452928: Error [kgfoAl06] in [kgfokge] at kgfo.c:3180 2024-03-05 12:52:04.581 : OCRRAW:4255452928: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor ORA-12514: TNS:listener does not currently know of service requested 2024-03-05 12:52:04.581 : OCRRAW:4255452928: Category: 7 2024-03-05 12:52:04.581 : OCRRAW:4255452928: DepInfo: 12514 2024-03-05 12:52:04.581 : OCRRAW:4255452928: ADR is not properly configured 2024-03-05 12:52:04.581 : OCRRAW:4255452928: -- trace dump end -- OCRASM:4255452928: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=12514, loc=kgfokge 2024-03-05 12:52:04.581 : OCRASM:4255452928: ASM Error Stack : ORA-12514: TNS:listener does not currently know of service requested in connect descriptor ORA-12514: TNS:listener does not currently know of service requested in connect descriptor 2024-03-05 12:52:04.581 : OCRASM:4255452928: proprasmo: kgfoCheckMount returned [7] 2024-03-05 12:52:04.581 : OCRASM:4255452928: proprasmo: The ASM instance is down 2024-03-05 12:52:04.635 : OCRRAW:4255452928: proprioo: Failed to open [+SYSTEMDG/xff-cluster/OCRFILE/registry.255.1072903025]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE. 2024-03-05 12:52:04.635 : OCRRAW:4255452928: proprioo: No OCR/OLR devices are usable OCRUTL:4255452928: u_fill_errorbuf: Error Info : [Insufficient quorum to open OCR devices] default:4255452928: u_set_gbl_comp_error: comptype '107' : error '0' 2024-03-05 12:52:04.635 : OCRRAW:4255452928: proprinit: Could not open raw device 2024-03-05 12:52:04.635 : default:4255452928: a_init:7!: Backend init unsuccessful : [26] 2024-03-05 12:52:04.637 : default:4255452928: clsvactversion:4: Retrieving Active Version from local storage.
通过这里,初步判断是由于节点2访问(DESCRIPTION=(TCP_USER_TIMEOUT=1)(CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=节点1私网IP)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))异常导致,查看节点1的该监听状态
[grid@xff1 ~]$ lsnrctl status ASMNET1LSNR_ASM LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 05-MAR-2024 13:04:51 Copyright (c) 1991, 2021, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))) STATUS of the LISTENER ------------------------ Alias ASMNET1LSNR_ASM Version TNSLSNR for Linux: Version 19.0.0.0.0 - Production Start Date 20-MAY-2021 23:53:50 Uptime 25 days 8 hr. 15 min. 15 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/19c/grid/network/admin/listener.ora Listener Log File /u01/app/grid/diag/tnslsnr/xff1/asmnet1lsnr_asm/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=ASMNET1LSNR_ASM))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=节点1私网IP)(PORT=1525))) The listener supports no services The command completed successfully
发现该监听没有注册服务进去,检查相关listener参数配置
[grid@xff1 ~]$ sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Tue Mar 5 13:26:29 2024 Version 19.11.0.0.0 Copyright (c) 1982, 2020, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.11.0.0.0 SQL> show parameter listener; NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ forward_listener string listener_networks string local_listener string remote_listener string
初步判断是由于节点1的ASMNET1LSNR_ASM监听状态异常,很可能是由于asm实例的listener参数异常导致,比较稳妥的解决方案是重启节点1,让其重新生成listener相关参数，实现动态注册,临时解决方法,
[grid@xff1 ~]$ sqlplus / as sysasm SQL*Plus: Release 19.0.0.0.0 - Production on Tue Mar 5 13:05:11 2024 Version 19.11.0.0.0 Copyright (c) 1982, 2020, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.11.0.0.0 SQL> ALTER SYSTEM SET local_listener ='(ADDRESS=(PROTOCOL=TCP)(HOST=节点1私网IP)(PORT=1525))' sid='+ASM1' SCOPE=MEMORY; System altered. [grid@xff1 ~]$ lsnrctl status ASMNET1LSNR_ASM LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 05-MAR-2024 13:05:21 Copyright (c) 1991, 2021, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))) STATUS of the LISTENER ------------------------ Alias ASMNET1LSNR_ASM Version TNSLSNR for Linux: Version 19.0.0.0.0 - Production Start Date 20-MAY-2021 23:53:50 Uptime 25 days 8 hr. 15 min. 45 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/19c/grid/network/admin/listener.ora Listener Log File /u01/app/grid/diag/tnslsnr/xff1/asmnet1lsnr_asm/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=ASMNET1LSNR_ASM))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=节点1私网IP)(PORT=1525))) Services Summary... Service "+ASM" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service... Service "+ASM_DATA" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service... Service "+ASM_FRA" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service... Service "+ASM_SYSTEMDG" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service... The command completed successfully [grid@xff1 ~]$
设置节点1的asm实例的local_listener 参数之后,集群启动成功
[grid@xff2 ~]$ crsctl status res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE xff2 STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE xff2 STABLE
ora.crf
1 ONLINE ONLINE xff2 STABLE
ora.crsd
1 ONLINE ONLINE xff2 STABLE
ora.cssd
1 ONLINE ONLINE xff2 STABLE
ora.cssdmonitor
1 ONLINE ONLINE xff2 STABLE
ora.ctssd
1 ONLINE ONLINE xff2 OBSERVER,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.drivers.acfs
1 ONLINE ONLINE xff2 STABLE
ora.evmd
1 ONLINE ONLINE xff2 STABLE
ora.gipcd
1 ONLINE ONLINE xff2 STABLE
ora.gpnpd
1 ONLINE ONLINE xff2 STABLE
ora.mdnsd
1 ONLINE ONLINE xff2 STABLE
ora.storage
1 ONLINE ONLINE xff2 STABLE
--------------------------------------------------------------------------------