集群版本:doris-3.1.4-rc02
报错的主要表象是，高频stream load写入时就出现大量versionincomplete的tablet，直接影响后续数据写入和查询。我已经尝试排查了主机的负载、网络、磁盘写入，这些方面都正常。后来也把rpc超时时间改大过。
故障链路大致如下：
① BE 间 brpc RPC 大面积超时
文件: belog/be.WARNING · 出现 42,011 次
5个BE互访都有超时
W20260701 14:24:02.437556 3054563 ref_count_closure.h:123] RPC meet failed: [E1008]Reached timeout=60000ms @30.3.2.22:8060
W20260701 14:24:02.548307 3054729 ref_count_closure.h:123] RPC meet failed: [E1008]Reached timeout=60000ms @30.3.2.20:8060
W20260701 14:24:02.685165 3054660 ref_count_closure.h:123] RPC meet failed: [E1008]Reached timeout=60000ms @30.3.2.21:8060
W20260701 14:24:02.825752 3054586 ref_count_closure.h:123] RPC meet failed: [E1008]Reached timeout=60000ms @30.3.2.19:8060
W20260701 14:24:02.961490 3054660 ref_count_closure.h:123] RPC meet failed: [E1008]Reached timeout=60000ms @30.3.2.18:8060
② BE 侧 Stream Load Pipeline 任务失败，取消所有 VNodeChannel
文件: belog/be.WARNING
W20260701 14:24:03.997830 1201111 vtablet_writer.cpp:831] cancel node channel VNodeChannel[...-1780712731998],
load_id=ee4678588dea5225-7568aa7feb84f9b2, txn_id=1687279, node=30.3.2.22:8060,
error message: [INTERNAL_ERROR]VNodeChannel[...-1780712731996], ... node=30.3.2.21:8060,
open failed, err: [INTERNAL_ERROR]failed to open tablet writer, error=RPC call is timed out,
error_text=[E1008]Reached timeout=60000ms @30.3.2.21:8060, ... host: 30.3.2.21
W20260701 14:24:03.998138 3055410 task_scheduler.cpp:166] Pipeline task failed.
query_id: ee4678588dea5225-7568aa7feb84f9b2
reason: [INTERNAL_ERROR]VNodeChannel[...-1780712731996], ... node=30.3.2.21:8060, open failed,
err: [INTERNAL_ERROR]failed to open tablet writer, error=RPC call is timed out, ...
W20260701 14:24:30.972052 1200024 stream_load_executor.cpp:105] fragment execute failed,
err_msg=[INTERNAL_ERROR]VNodeChannel[...-1780712731996], ... node=30.3.2.21:8060, open failed,
err: [INTERNAL_ERROR]failed to open tablet writer, error=RPC call is timed out, ...
label=group_commit_a540bdeb54ab65ac_160341e16bb1d384, elapse(s)=64
③ FE 回放 ABORT 事务 (1,044 个因 open_tablet_writer 超时)
文件: felog/fe.log
2026-07-02 00:00:18,849 INFO (replayer|139) [DatabaseTransactionMgr.replayUpsertTransactionState():2468]
replay a ABORTED transaction TransactionState. transaction id: 1730115,
label: group_commit_264d3ff29de1fcd2_0eb530f4f7fbada6, db id: 1781492882762,
transaction status: ABORTED, error replicas num: 0,
reason: [INTERNAL_ERROR]VNodeChannel[...-1780712731999], ... node=30.3.2.20:8060, open failed,
err: [INTERNAL_ERROR]failed to open tablet writer, error=RPC call is timed out,
error_text=[E1008]Reached timeout=60000ms @30.3.2.20:8060, ... host: 30.3.2.20
④ BE 侧 Publish Version 失败 — tablet version not exists（2,766 次）
文件: belog/be.INFO
超时节点的版本数据未写入，其他副本收到 publish 时本地找不到该版本
W20260702 13:39:13.930881 2514761 engine_publish_version_task.cpp:354]
publish version failed on transaction, tablet version not exists.
transaction_id=1746341, tablet_id=1782821512658, tablet_state=TABLET_RUNNING, version=220
W20260702 13:39:13.930984 2514761 engine_publish_version_task.cpp:354]
publish version failed on transaction, tablet version not exists.
transaction_id=1746341, tablet_id=1782821512662, tablet_state=TABLET_RUNNING, version=220
...
(同一个 txn 下连续几十个 tablet 全部报 version not exists)
⑤ MoW 表 Publish 批量失败 — version not continuous（135 次）
文件: belog/be.INFO
版本链断裂，tablet 本地 max_version=315，txn 要求 publish 到 317（跳过了 316）
W20260702 15:02:32.001420 2514762 task_worker_pool.cpp:2047]
failed to publish version|signature=1747896|transaction_id=1747896|error_tablets_num=38
|error=[E-3115]version not continuous for mow,
tablet_id=1782821512902, tablet_max_version=315, txn_version=317
W20260702 15:04:39.000556 2514762 task_worker_pool.cpp:2047]
failed to publish version|signature=1747930|transaction_id=1747930|error_tablets_num=14
|error=[E-3115]version not continuous for mow,
tablet_id=1782821512902, tablet_max_version=315, txn_version=318
W20260702 15:05:11.000625 2514758 task_worker_pool.cpp:2047]
failed to publish version|signature=1747938|transaction_id=1747938|error_tablets_num=14
|error=[E-3115]version not continuous for mow,
tablet_id=1782821512902, tablet_max_version=315, txn_version=319
⑥ 版本修复(Clone/Snapshot)也失败（1,891 次）
文件: belog/be.WARNING
源端副本也找不到 miss version（因为原先所有副本的写入都因 RPC 超时而未完成）
W20260701 14:24:00.091352 3879666 status.h:427]
meet error status: [INTERNAL_ERROR]failed to find missed version when snapshot.
tablet=1781495694173, schema_hash=1835180137, version=[280-280]
W20260701 14:24:00.141409 1060286 status.h:427]
meet error status: [INTERNAL_ERROR]failed to find missed version when snapshot.
tablet=1781495694153, schema_hash=1835180137, version=[280-280]
⑦ FE 回放 VISIBLE 事务时发现 error_replicas 非 0（236 个）
文件: felog/fe.log
事务虽被标记 VISIBLE，但部分副本实际上写入失败（数据不一致）
2026-07-02 00:32:21,569 INFO (replayer|139) [DatabaseTransactionMgr.replayUpsertTransactionState():2468]
replay a VISIBLE transaction TransactionState. transaction id: 1731104,
label: group_commit_dc47058550e245a0_6c69c08e945a8181,
transaction status: VISIBLE, error replicas num: 20,
replica ids: 1782821382224,1782821382257,1782821382131,1782821382164,1782821382197,
commit time: 1782923541550, finish time: 1782923541565
2026-07-02 01:00:37,103 INFO (replayer|139) [DatabaseTransactionMgr.replayUpsertTransactionState():2468]
replay a VISIBLE transaction TransactionState. transaction id: 1731863,
label: group_commit_5c442412c4114a68_7db767bcb5f785ae,
transaction status: VISIBLE, error replicas num: 77,
replica ids: 1781500803622,1781500032178,1781500032305,1781500803616,1781500803489,
commit time: 1782925226114, finish time: 1782925237096
