昨天跑批量数据的程序时遇到oracle错误：
$tail -f INDB_ERROR_8.LOG
[Time]2014-12-01 04:10:31: activeAccountDeposit in oracle error: =ORA-01591: lock held by in-doubt distributed transaction 20.21.65527021
[Time]2014-12-01 04:10:31: 文件:/billing/drecv4/drecvlog/201411/DRECV/tmp/8-15-accountdeposit.dat改名到/billing/drecv4/drecvlog/201411/DRECV/tmp/8-15-accountdeposit.dat.err失败!ORA-01591: lock held by in-doubt distributed transaction 20.21.65527021
DwriteOffCore.cpp:3152
[Time]2014-12-01 04:10:31: 8-15-accountdeposit.dat更新表发生错误!ORA-01591: lock held by in-doubt distributed transaction 20.21.65527021
DwriteOffCore.cpp:4431
[Time]2014-12-01 04:10:31: 入库发生错误!
重复跑程序错误依旧。
打开数据库，查看这个tran_id确实有，其实是11月19号的时候就留下了这个事务。这是分布式事务的问题，当执行这个事务的时候遇到一些问题，比如网络问题，oracle就会卡住在这里，一直留着，当后面你更新到这条数据的时候就会报这个错。
然后找dba，他执行一下：
rollback force ‘20.21.65527021’
commit
搞定。
后续防止这个错误的其中一个方法就是，在跑批量程序前，统一先检查并杀掉这些阻塞。
附录：
ORA-01591: "lock held by in-doubt distributed transaction %s"
Cause: Trying to access resource that is locked by a dead
two-phase commit transaction that is in prepared state.
Action: DBA should query the pending_trans$ and related tables,
and attempt to repair network connection(s) to
coordinator and commit point. If timely repair is not
possible, DBA should contact DBA at commit point if
known or end user for correct outcome, or use heuristic
default if given to issue a heuristic commit or abort
command to finalize the local portion of the
distributed transaction.