Instance terminated by PMON

jl8323

QMNC started with pid=18, OS id=10455
Wed Nov  4 13:08:51 2009
Completed: ALTER DATABASE OPEN
Wed Nov  4 13:08:51 2009
Errors in file /usr/oracle/admin/ultraeos/bdump/ultraeos_mmon_10439.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [4193] [37] [34]
Wed Nov  4 13:18:53 2009
Process 0x0xa75c5a48 appears to be hung while dumping
Current time = 1725304580, process death time = 1725244347 interval = 60000
Attempting to kill process 0x0xa75c5a48 with OS pid = 10439
OSD kill succeeded for process 0xa75c5a48
DEBUG: Replaying xcb 0xa6edb2e8, pmd 0xa54f0eb4 for failed op 8
Doing block recovery for file 2 block 4427
No block recovery was needed
Wed Nov  4 13:22:38 2009
Errors in file /usr/oracle/admin/ultraeos/bdump/ultraeos_pmon_10421.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [4193] [37] [34]
Wed Nov  4 13:22:39 2009
DEBUG: Replaying xcb 0xa6edb2e8, pmd 0xa54f0eb4 for failed op 8
Doing block recovery for file 2 block 4427
No block recovery was needed
Wed Nov  4 13:22:39 2009
Errors in file /usr/oracle/admin/ultraeos/bdump/ultraeos_pmon_10421.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [4193] [37] [34]
Wed Nov  4 13:22:39 2009
PMON: terminating instance due to error 472
Instance terminated by PMON, pid = 10421

启实例后，跑点SQL，实例就自动关闭。


以下是ultraeos_pmon_10421.trc
/usr/oracle/admin/ultraeos/bdump/ultraeos_pmon_10421.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORACLE_HOME = /usr/oracle/product/10g
System name:    Linux
Node name:      linux5domain
Release:        2.6.18-8.el5xen
Version:        #1 SMP Thu Mar 15 21:02:53 EDT 2007
Machine:        i686
Instance name: ultraeos
Redo thread mounted by this instance: 1
Oracle process number: 2
Unix process pid: 10421, image: oracle@linux5domain (PMON)

*** 2009-11-04 13:18:53.880
*** SERVICE NAMESYS$BACKGROUND) 2009-11-04 13:18:53.880
*** SESSION ID555.1) 2009-11-04 13:18:53.880
*** 2009-11-04 13:18:53.880
Process 0x0xa75c5a48 appears to be hung while dumping
Current time = 1725304580, process death time = 1725244347 interval = 60000
Attempting to kill process 0x0xa75c5a48 with OS pid = 10439
OSD kill succeeded for process 0xa75c5a48
DEBUG: Replaying xcb 0xa6edb2e8, pmd 0xa54f0eb4 for failed op 8
DEBUG: Reconstructing undo block 0x80114b for xcb 0xa6edb2e8
Doing block recovery for file 2 block 4427
Block header before block recovery:
buffer tsn: 1 rdba: 0x0080114b (2/4427)
scn: 0x0000.02d7d1e9 seq: 0x01 flg: 0x04 tail: 0xd1e90201
frmt: 0x02 chkval: 0x2bac type: 0x02=KTU UNDO BLOCK
No block recovery was needed
  lrda: 0.0 hscn: 48740923.0 hsub-scn: 65535
Block image after block recovery (no redo applied):
buffer tsn: 1 rdba: 0x0080114b (2/4427)
scn: 0x0000.02d7d1e9 seq: 0x01 flg: 0x04 tail: 0xd1e90201
frmt: 0x02 chkval: 0x2bac type: 0x02=KTU UNDO BLOCK
Hex dump of block: st=0, typ_found=1


另外，每次实例启动后马上就就会有个报错：
Completed: ALTER DATABASE OPEN
Wed Nov  4 13:08:51 2009
Errors in file /usr/oracle/admin/ultraeos/bdump/ultraeos_mmon_10439.trc:
文件内容：


/usr/oracle/admin/ultraeos/bdump/ultraeos_mmon_10439.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORACLE_HOME = /usr/oracle/product/10g
System name:    Linux
Node name:      linux5domain
Release:        2.6.18-8.el5xen
Version:        #1 SMP Thu Mar 15 21:02:53 EDT 2007
Machine:        i686
Instance name: ultraeos
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 10439, image: oracle@linux5domain (MMON)

*** SERVICE NAMESYS$BACKGROUND) 2009-11-04 13:08:51.403
*** SESSION ID546.1) 2009-11-04 13:08:51.403
*** ktuc_diag_dmp: dump of current change vector
ktudb big redo: cnt: 7 um: 0xa54f1f2c uba: 0x0080114b.0022.2e
*** ktuc_diag_dmp: dump of block after image in hex
Dump of memory from 0x760DE000 to 0x760E0000

howard_zhang

错误很明显是UNDO出了问题，之前做过什么事情？

howard_zhang

传一个完整的DUMP文件
/usr/oracle/admin/ultraeos/bdump/ultraeos_pmon_10421.trc

jl8323

-rw-r----- 1 oracle oinstall 597072744 Nov  4 13:22 ultraeos_pmon_10421.trc


太大了。。。

ol 34: [ 2]  c1 1e

*-----------------------------
* Rec #0x4  slt: 0x24  objn: 97081(0x00017b39)  objd: 97081  tblspc: 0(0x00000000)
*       Layer:  11 (Row)   opc: 1   rci 0x03   
Undo type:  Regular undo   Last buffer split:  No
Temp Object:  No
Tablespace Undo:  No
rdba: 0x00000000
*-----------------------------
KDO undo record:
KTB Redo
op: 0x02  ver: 0x01  
op: C  uba: 0x0080114b.0025.03
KDO Op code: URP row dependencies Disabled
  xtype: XA flags: 0x00000000  bdba: 0x0040fce1  hdba: 0x0040fc99
itli: 1  ispac: 0  maxfr: 4863
tabn: 0 slot: 12(0xc) flag: 0x2c lock: 1 ckix: 191
ncol: 35 nnew: 32 size: -7
col  1: [32]
34 30 32 38 38 30 39 39 32 34 62 33 66 36 37 66 30 31 32 34 62 34 33 36 30
34 32 31 30 31 66 33
col  2: [32]


还有就是一堆0DF8B0 B7E90001 00000001 0040FCE1 0040FC99  [..........@...@.]
760DF8C0 811512FF B7E90001 00BF012C 0125000C  [........,.....%.]
760DF8D0 01260000 00000017 00000013 85BEE512  [..&.............]
760DF8E0 E58D9CE6 8FE5A18A 84A4E5B0 008690E7  [................]
760DF8F0 0018004A 00200010 00200040 00200020  [J..... .@. . . .]
760DF900 00000000 00200020 00160002 000B000B  [.... . .........]
760DF910 00000000 00080000 00200000 00020000  [.......... .....]
760DF920 00120002 00090002 00010001 00190016  [................]
760DF930 00160000 00010001 00000002 00017B39  [............9{..]
760DF940 00017B39 00000000 00000000 0324010B  [9{............$.]
760DF950 00100000 00000102 00000000 0080114B  [............K...]
760DF960 00030025 0040FCE1 0040FC99 010512FF  [%.....@...@.....]
760DF970 B7E90001 00BF012C 2023000C B818FFF9  [....,.....# ....]
760DF980 00000802 00020001 00040003 00060005  [................]
760DF990 00080007 000A0009 000E000C 0010000F  [................]
760DF9A0 00120011 00140013 00160015 00180017  [................]
760DF9B0 001A0019 001C001B 001E001D 0020001F  [.............. .]
760DF9C0 00220021 38323034 39393038 33623432  [!.".4028809924b3]
760DF9D0 66373666 34323130 36333462 31323430  [f67f0124b4360421]
760DF9E0 33663130 38323034 39393038 33623432  [01f34028809924b3]
760DF9F0 66373666 34323130 39333462 38626534  [f67f0124b4394eb8]
760DFA00 31643530 38323034 39393038 33623432  [05d14028809924b3]
760DFA10 66373666 34323130 39333462 38626534  [f67f0124b4394eb8]
760DFA20 31643530 38323034 39393038 33623432  [05d14028809924b3]
760DFA30 66373666 34323130 36333462 31323430  [f67f0124b4360421]
760DFA40 33663130 38323034 30303138 62366431  [01f3402881001d6b]
760DFA50 31636166 64313130 66666236 66376139  [fac1011d6bff9a7f]
760DFA60 31303030 000002C1 696D6461 908FE66E  [0001....admin...]
760DFA70 28A4BAE4 E58D9CE6 8FE5A18A 000029B0  [...(.........)..]

howard_zhang

._corrupted_rollback_segments
考虑使用这个隐含参数吧。

mengxb001

老大一眼就看出是undo了

jl8323

SQL> create undo tablespace tbs_undo datafile '/usr/oracle/oradata/ultraeos/undotbs02.dbf' size 300M ;

Tablespace created.

SQL> alter system set undo_tablespace='tbs_undo' scope=both;

System altered.

SQL> show parameter undo

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
undo_management                      string      AUTO
undo_retention                       integer     900
undo_tablespace                      string      tbs_undo

jl8323

一会试下结果怎么样，如果还不行。。。。我就。。。

jl8323

OK了，重建了UNDO。
这下记住这个了，ORA-00600:  [4193]
多谢！！！！！

seagull76

原帖由 jl8323 于 2009-11-4 14:44 发表
OK了，重建了UNDO。
这下记住这个了，ORA-00600:  [4193]
多谢！！！！！

这个问题看来很常见。