- UID
- 12521
- 阅读权限
- 100
- 帖子
- 14310
- 精华贴数
- 57
- 技术排名
- 23
- 技术积分
- 51675
- 社区排名
- 579
- 社区积分
- 3819
- 注册时间
- 2002-5-26
- 精华贴数
- 57
- 技术积分
- 51675
- 社区积分
- 3819
- 注册时间
- 2002-5-26
- 论坛徽章:
- 70
|
发表于 2007-12-18 22:06:27
|显示全部楼层
这次在客户出处理ORA-4031的故障,但是发现在系统正常期间却没有什么值得怀疑的痕迹。跟大家讨论一下。
操作系统:IBM AIX 5.3
数据库:Oracle 9.2.0.6
Shared Pool:2G
问题现象:
Weblogic中间件收到ORA-4031报错,然后SQLPLUS无法登陆,重新启动数据库,系统恢复正常。同样的错误在最近2个月内发生了3次。在第一次发生问题前的几天内刚实施了Change Data Capture,通过ODI实施的。
告警日志内容:
- Tue Dec 11 17:14:49 2007
- Errors in file /oracle/app/admin/ctais2/udump/ctais2_ora_778732.trc:
- ORA-04031: unable to allocate 4216 bytes of shared memory ("shared pool","IDX_DJ_NSRXX_P_NSRMCCTAIS2","sga heap(2,0)","library cache")
- ORA-04031: unable to allocate 4216 bytes of shared memory ("shared pool","IDX_DJ_NSRXX_P_NSRMCCTAIS2","sga heap(2,0)","library cache")
- Tue Dec 11 17:14:51 2007
- Errors in file /oracle/app/admin/ctais2/bdump/ctais2_pmon_393248.trc:
- ORA-04031: unable to allocate 4216 bytes of shared memory ("shared pool","unknown object","sga heap(2,0)","library cache")
- 。。。。。。。。。
- Tue Dec 11 17:16:40 2007
- Completed checkpoint up to RBA [0x2225.2.10], SCN: 0x07ba.62a2620f
- Tue Dec 11 17:20:56 2007
- Beginning log switch checkpoint up to RBA [0x2226.2.10], SCN: 0x07ba.62a2b207
- Thread 1 advanced to log sequence 8742
- Current log# 1 seq# 8742 mem# 0: /oracle/app/oradata/ctais2/redo01a.log
- Current log# 1 seq# 8742 mem# 1: /oracle/app/oradata/ctais2/redo01b.log
- Tue Dec 11 17:20:56 2007
- ARC0: Evaluating archive log 3 thread 1 sequence 8741
- ARC0: Beginning to archive log 3 thread 1 sequence 8741
- Creating archive destination LOG_ARCHIVE_DEST_1: '/oralog/1_8741.dbf'
- ARC0: Completed archiving log 3 thread 1 sequence 8741
- 。。。。。。。。。
- Tue Dec 11 17:46:13 2007
- Errors in file /oracle/app/admin/ctais2/udump/ctais2_ora_1413316.trc:
- ORA-00600: internal error code, arguments: [729], [864], [space leak], [], [], [], [], []
- Tue Dec 11 17:46:14 2007
- Errors in file /oracle/app/admin/ctais2/udump/ctais2_ora_1921394.trc:
- ORA-00600: internal error code, arguments: [729], [240], [space leak], [], [], [], [], []
复制代码
因为报错是在sga heap(2,0),所以在ctais2_ora_778732.trc中查看Subpool 2的dump,但是free memory还有 148200480,报错的library cache区域也只有173949024而已。
- ===============================
- Memory Utilization of Subpool 2
- ===============================
- Allocation Name Size
- _________________________ __________
- "free memory " 148200480
- "miscellaneous " 8271560
- "trace buf hdr xtend " 182160
- "errors " 0
- "partitioning d " 0
- "PL/SQL SOURCE " 0
- "transaction " 641040
- "parameters " 0
- "sim memory hea " 2656472
- "table definiti " 0
- "trigger inform " 0
- "MTTR advisory " 1015088
- "PL/SQL PPCODE " 0
- "transaction co " 0
- "PL/SQL DIANA " 0
- "joxlod: in ehe " 70808
- "db_block_hash_buckets " 1572768
- "UNDO INFO HASH " 77600
- "kglsim hash table bkts " 700416
- "Temporary Tables State Ob" 517080
- "session param values " 3020160
- "trace buffer " 3325952
- "dictionary cache " 1065472
- "Checkpoint queue " 1283200
- "kglsim heap " 8424000
- "temporary tabl " 287456
- "PL/SQL MPCODE " 37304
- "library cache " 173949024
- "kglsim object batch " 7608384
- "sql area " 163320
- "sessions " 995072
- "PLS non-lib hp " 2088
- "event statistics per sess" 4151504
- "fixed allocation callback" 288
- "KGLS heap " 61320
- "trigger defini " 0
- "obj htab chunk " 655872
- "KQR M PO " 162864
复制代码
统计部分的命中率也没什么大问题。
- LIBRARY CACHE STATISTICS:
- namespace gets hit ratio pins hit ratio reloads invalids
- -------------- --------- --------- --------- --------- ---------- ----------
- CRSR 498593069 0.986 1235442503 0.842 38042363 575091
- TABL/PRCD/TYPE 787258490 0.998 1413907949 0.987 6769881 0
- BODY/TYBD 465948 0.993 467940 0.986 3152 0
- TRGR 27415575 0.999 27417188 0.993 172702 0
- INDX 141777026 0.999 99729644 0.998 690 0
- CLST 524459 0.989 706762 0.983 1 0
复制代码
后面那个ORA-600报UGA内存泄露的错误,估计应该是4031错误的后遗症,而不是导致4031的原因,因为:
1. 泄露的内存并不大,最大的10480而已。
2. 查看Call Stack Trace特征,有opilof,表示是session logout的时候出现的内存泄露,而因为是dedicated server模式,所以即使是UGA内存泄露也不会影响到SGA
关于ORA-600 [729] 的问题可以参考Metalink Note:403584.1
今天在系统正常期间,检查共享池sharedpool各chunck的情况,很正常,大小size的free chunck都不少。
- SQL> select KSMCHIDX "SubPool", 'sga heap('||KSMCHIDX||',0)'sga_heap,ksmchcom ChunkComment,
- 2 decode(round(ksmchsiz/1000),0,'0-1K', 1,'1-2K', 2,'2-3K',3,'3-4K',
- 3 4,'4-5K',5,'5-6k',6,'6-7k',7,'7-8k',8,
- 4 '8-9k', 9,'9-10k','> 10K') "size",
- 5 count(*),ksmchcls Status, sum(ksmchsiz) Bytes
- 6 from x$ksmsp
- 7 where KSMCHCOM = 'free memory'
- 8 group by ksmchidx, ksmchcls,
- 9 'sga heap('||KSMCHIDX||',0)',ksmchcom, ksmchcls,decode(round(ksmchsiz/1000),0,'0-1K',
- 10 1,'1-2K', 2,'2-3K', 3,'3-4K',4,'4-5K',5,'5-6k',6,
- 11 '6-7k',7,'7-8k',8,'8-9k', 9,'9-10k','> 10K');
- SubPool SGA_HEAP CHUNKCOMMENT size COUNT(*) STATUS BYTES
- -------------------- --------------- ---------------- ---------- -------------------- -------- --------------------
- 1 sga heap(1,0) free memory 0-1K 4842 free 909400
- 1 sga heap(1,0) free memory 1-2K 297 free 369896
- 1 sga heap(1,0) free memory 2-3K 822 free 1903264
- 1 sga heap(1,0) free memory 3-4K 4616 free 13839336
- 1 sga heap(1,0) free memory 4-5K 3634 free 13827256
- 1 sga heap(1,0) free memory 8-9k 3 free 24808
- 1 sga heap(1,0) free memory 1-2K 1 R-free 1416
- 1 sga heap(1,0) free memory 6-7k 1 R-free 5696
- 1 sga heap(1,0) free memory 9-10k 1 R-free 8768
- 1 sga heap(1,0) free memory > 10K 77 R-free 75113896
- 2 sga heap(2,0) free memory 0-1K 5948 free 729904
- 2 sga heap(2,0) free memory 1-2K 55 free 52056
- 2 sga heap(2,0) free memory 2-3K 225 free 453936
- 2 sga heap(2,0) free memory 3-4K 6799 free 21146640
- 2 sga heap(2,0) free memory 4-5K 4900 free 18657216
- 2 sga heap(2,0) free memory 5-6k 1 free 4624
- 2 sga heap(2,0) free memory 7-8k 1 free 7168
- 2 sga heap(2,0) free memory 8-9k 4 free 32000
- 2 sga heap(2,0) free memory 1-2K 1 R-free 648
- 2 sga heap(2,0) free memory 8-9k 2 R-free 16304
- 2 sga heap(2,0) free memory > 10K 80 R-free 78728920
- 3 sga heap(3,0) free memory 0-1K 5572 free 914096
- 3 sga heap(3,0) free memory 1-2K 204 free 153120
- 3 sga heap(3,0) free memory 2-3K 2818 free 6129920
- 3 sga heap(3,0) free memory 3-4K 5323 free 15764400
- 3 sga heap(3,0) free memory 4-5K 3710 free 14163560
- 3 sga heap(3,0) free memory 2-3K 1 R-free 1712
- 3 sga heap(3,0) free memory 4-5K 1 R-free 4496
- 3 sga heap(3,0) free memory 8-9k 3 R-free 24456
- 3 sga heap(3,0) free memory 9-10k 2 R-free 17656
- 3 sga heap(3,0) free memory > 10K 64 R-free 75655456
- 4 sga heap(4,0) free memory 0-1K 6566 free 1192232
- 4 sga heap(4,0) free memory 1-2K 405 free 289456
- 4 sga heap(4,0) free memory 2-3K 1300 free 2728536
- 4 sga heap(4,0) free memory 3-4K 5989 free 17931464
- 4 sga heap(4,0) free memory 4-5K 3683 free 14001176
- 4 sga heap(4,0) free memory 9-10k 1 free 8736
- 4 sga heap(4,0) free memory 1-2K 1 R-free 616
- 4 sga heap(4,0) free memory 3-4K 1 R-free 2624
- 4 sga heap(4,0) free memory 5-6k 1 R-free 5424
- 4 sga heap(4,0) free memory 6-7k 1 R-free 5504
- 4 sga heap(4,0) free memory 8-9k 1 R-free 8152
- 4 sga heap(4,0) free memory > 10K 58 R-free 75088136
- 5 sga heap(5,0) free memory 0-1K 5695 free 689448
- 5 sga heap(5,0) free memory 1-2K 771 free 847032
- 5 sga heap(5,0) free memory 2-3K 1160 free 2422184
- 5 sga heap(5,0) free memory 3-4K 4939 free 15128240
- 5 sga heap(5,0) free memory 4-5K 3140 free 12045192
- 5 sga heap(5,0) free memory 4-5K 1 R-free 3592
- 5 sga heap(5,0) free memory 8-9k 1 R-free 8152
- 5 sga heap(5,0) free memory > 10K 73 R-free 79633288
- 6 sga heap(6,0) free memory 0-1K 5840 free 1088488
- 6 sga heap(6,0) free memory 1-2K 309 free 412944
- 6 sga heap(6,0) free memory 2-3K 2362 free 5076752
- 6 sga heap(6,0) free memory 3-4K 5250 free 15702056
- 6 sga heap(6,0) free memory 4-5K 3878 free 14726920
- 6 sga heap(6,0) free memory 5-6k 1 free 4648
- 6 sga heap(6,0) free memory 4-5K 1 R-free 3912
- 6 sga heap(6,0) free memory 6-7k 3 R-free 18840
- 6 sga heap(6,0) free memory 7-8k 1 R-free 7080
- 6 sga heap(6,0) free memory > 10K 94 R-free 73405856
- 61 rows selected.
复制代码
[ 本帖最后由 Kamus 于 2007-12-18 22:44 编辑 ] |
附件: 你需要登录才可以下载或查看附件。没有帐号?注册
|