诡异的分布式事务错误

jieyancai

源端：windows2008x64+oracle11.2.0.4
目标端：centos6.6 64位+oracle12.2.0.1
目标端的监听和sqlnet.ora有添加如下参数：sqlnet.ora：
DIAG_ADR_ENABLED = OFF
SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8

listener.ora：
DIAG_ADR_ENABLED_LISTENER = OFF

源端创建了dblink到目标端，手动查询正常。但是后续总报错，问题可能在哪？分布式事务总被挂起，不得不在源端手动删除。但问题没有根本解决：
select 'execute DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY(''' || LOCAL_TRAN_ID || ''');' || chr(10) || 'commit;' from DBA_2PC_PENDING;

源端alert还是会不断报错：
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾
Tue Jan 16 10:34:38 2018
Errors in file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾
Tue Jan 16 10:37:16 2018
Errors in file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾
Tue Jan 16 10:39:57 2018
DDE: Problem Key 'ORA 603' was completely flood controlled (0x6)
Further messages for this problem key will be suppressed for up to 10 minutes
Errors in file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾
Tue Jan 16 10:42:24 2018
Errors in file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾
Tue Jan 16 10:42:57 2018
Thread 1 cannot allocate new log, sequence 60179
Private strand flush not complete
  Current log# 2 seq# 60178 mem# 0: D:\ORACLE\APP\ORACLE\ORADATA\GDDB\REDO02.LOG
Thread 1 advanced to log sequence 60179 (LGWR switch)
  Current log# 3 seq# 60179 mem# 0: D:\ORACLE\APP\ORACLE\ORADATA\GDDB\REDO03.LOG
Tue Jan 16 10:43:45 2018
Errors in file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾


trc文件如下：
Trace file D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\trace\gddb_reco_2088.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
Windows NT Version V6.1 Service Pack 1
CPU                 : 4 - type 8664, 4 Physical Cores
Process Affinity    : 0x0x0000000000000000
Memory (Avail/Total): Ph:10434M/16338M, Ph+PgF:21954M/32675M
Instance name: gddb
Redo thread mounted by this instance: 1
Oracle process number: 14
Windows thread id: 2088, image: ORACLE.EXE (RECO)


*** 2018-01-15 21:02:34.880
*** SESSION ID390.1) 2018-01-15 21:02:34.880
*** CLIENT ID) 2018-01-15 21:02:34.880
*** SERVICE NAMESYS$BACKGROUND) 2018-01-15 21:02:34.880
*** MODULE NAME) 2018-01-15 21:02:34.880
*** ACTION NAME) 2018-01-15 21:02:34.880

Incident 76652 created, dump file: D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\incident\incdir_76652\gddb_reco_2088_i76652.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)


*** 2018-01-15 21:02:36.487
ERROR, tran=16.8.531229, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:37.392
Incident 76653 created, dump file: D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\incident\incdir_76653\gddb_reco_2088_i76653.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)


*** 2018-01-15 21:02:38.702
ERROR, tran=9.10.1012460, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:40.496
Incident 76654 created, dump file: D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\incident\incdir_76654\gddb_reco_2088_i76654.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)


*** 2018-01-15 21:02:41.775
ERROR, tran=4.29.947695, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:43.522
Incident 76655 created, dump file: D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\incident\incdir_76655\gddb_reco_2088_i76655.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)


*** 2018-01-15 21:02:44.770
ERROR, tran=18.13.579833, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:46.564
Incident 76656 created, dump file: D:\ORACLE\APP\ORACLE\diag\rdbms\gddb\gddb\incident\incdir_76656\gddb_reco_2088_i76656.trc
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)


*** 2018-01-15 21:02:47.719
ERROR, tran=5.24.944217, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:49.606
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 76657)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:02:49.622
ERROR, tran=17.1.536041, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:52.680
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 76658)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:02:52.680
ERROR, tran=20.10.550534, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:55.753
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 76659)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:02:55.753
ERROR, tran=19.33.607290, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:02:58.810
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80540)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:02:58.826
ERROR, tran=2.2.881376, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:01.868
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80541)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:01.868
ERROR, tran=13.9.602344, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:04.926
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80542)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:05.207
ERROR, tran=14.24.623155, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:07.968
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80543)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:07.968
ERROR, tran=5.30.944991, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:11.025
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80544)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:11.041
ERROR, tran=3.33.971164, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:14.083
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80545)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:14.099
ERROR, tran=7.12.912012, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:17.156
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80546)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:17.156
ERROR, tran=9.4.1012023, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:20.183
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80547)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:20.183
ERROR, tran=10.19.986427, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:23.303
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80548)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:23.303
ERROR, tran=9.30.1012832, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾

*** 2018-01-15 21:03:26.360
DDE: Problem Key 'ORA 603' was flood controlled (0x2) (incident: 80549)
ORA-00603: ORACLE 服务器会话因致命错误而终止
ORA-02063: 紧接着 line (起自 ZYFX_180)

*** 2018-01-15 21:03:26.360
ERROR, tran=14.22.624429, session#=1, ose=0:
ORA-03150: 数据库链接的通信通道的文件结尾
ORA-03113: 通信通道的文件结尾


目标端alert日志中不断报错如下：
2018-01-16T10:33:52.168218+08:00
Session (103,56956): RECO logon successful: Inbound connection from client
Session (103,56956): RECO logon successful: DB Logon User: RECO, Remote Machine: GDZYTJ, Program: ORACLE.EXE, OS User: SYSTEM
Session (103,56956): RECO logon successful: Client IP Address: 172.1.1.18
opiodr aborting process unknown ospid (32446) as a result of ORA-603
2018-01-16T10:36:33.411939+08:00
Session (544,17684): RECO logon successful: Inbound connection from client
Session (544,17684): RECO logon successful: DB Logon User: RECO, Remote Machine: GDZYTJ, Program: ORACLE.EXE, OS User: SYSTEM
Session (544,17684): RECO logon successful: Client IP Address: 172.1.1.18
DDE: Problem Key 'ORA 603' was completely flood controlled (0x4)
Further messages for this problem key will be suppressed for up to 10 minutes
opiodr aborting process unknown ospid (32749) as a result of ORA-603
2018-01-16T10:38:59.711447+08:00
Session (154,16589): RECO logon successful: Inbound connection from client
Session (154,16589): RECO logon successful: DB Logon User: RECO, Remote Machine: GDZYTJ, Program: ORACLE.EXE, OS User: SYSTEM
Session (154,16589): RECO logon successful: Client IP Address: 172.1.1.18
opiodr aborting process unknown ospid (647) as a result of ORA-603
2018-01-16T10:40:21.139623+08:00
Session (495,50899): RECO logon successful: Inbound connection from client
Session (495,50899): RECO logon successful: DB Logon User: RECO, Remote Machine: GDZYTJ, Program: ORACLE.EXE, OS User: SYSTEM
Session (495,50899): RECO logon successful: Client IP Address: 172.1.1.18
opiodr aborting process unknown ospid (864) as a result of ORA-603