版本：2.1.9
在使用http streamload 导入数据时，经常性出现 报错：[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backends
具体的streamload 返回的信息如下：
{"Label":"330f1eb7-8575-4598-9b14-3538b72ada6d","LoadBytes":"0","LoadTimeMs":"0","Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backends","NumberFilteredRows":"0","NumberLoadedRows":"0","NumberTotalRows":"0","NumberUnselectedRows":"0","Status":"Fail","TxnId":"174951242","success":false}
当前已经排查，服务器内存、cpu、磁盘IO、 网络都没有达到资源的瓶颈
be的日志信息中也能看到先关的No available 错误如下：
I20250925 13:30:14.677070 10024 query_context.cpp:189] Query f2cd099efff14689-ae90ab7496915643 deconstructed, mem_tracker: , deregister query/load memory tracker, queryId=f2cd099efff14689-ae90ab7496915643, Limit=2.00 GB, CurrUsed=22.31 KB, PeakUsed=230.31 MB
I20250925 13:30:14.681389 9085 internal_service.cpp:660] Cancel query a27c29337aba4085-84087b2d4a7b5228, reason: INTERNAL_ERROR
I20250925 13:30:14.681416 9083 internal_service.cpp:660] Cancel query a27c29337aba4085-84087b2d4a7b5228, reason: INTERNAL_ERROR
W20250925 13:30:14.683947 9085 fragment_mgr.cpp:1305] Could not find the query id:a27c29337aba4085-84087b2d4a7b5228 fragment id:1 to cancel
W20250925 13:30:14.683960 9083 fragment_mgr.cpp:1305] Could not find the query id:a27c29337aba4085-84087b2d4a7b5228 fragment id:0 to cancel
I20250925 13:30:14.693490 8717 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 127808358 group data size: 0 row num: 53 consume bytes: 2598 way cnt: 6 batch size: 4064
I20250925 13:30:14.831149 8719 compaction.cpp:369] start cumulative compaction. tablet=123809401, output_version=[1436342-1436657], permits: 31
I20250925 13:30:14.831248 8719 merger.cpp:405] Start to do vertical compaction, tablet_id: 123809401
I20250925 13:30:14.831256 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 1394 way cnt: 9 batch size: 4064
I20250925 13:30:14.833181 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 2583 way cnt: 9 batch size: 4064
I20250925 13:30:14.834271 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 2432 way cnt: 9 batch size: 4064
I20250925 13:30:14.835366 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 1903 way cnt: 9 batch size: 4064
I20250925 13:30:14.836469 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 1715 way cnt: 9 batch size: 4064
I20250925 13:30:14.837512 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809401 group data size: 0 row num: 41 consume bytes: 1435 way cnt: 9 batch size: 4064
I20250925 13:30:14.848263 8719 tablet_meta_manager.cpp:293] remove old version delete bitmap, tablet_id: 123809401 version: 1436657, removed keys size: 0
I20250925 13:30:14.848367 8719 compaction.cpp:786] succeed to do cumulative compaction is_vertical=1. tablet=123809401, output_version=[1436342-1436657], current_max_version=1436657, disk=/msun/data/disk1/doris, segments=9, input_rowset_size=68678, output_rowset_size=11358, input_row_num=47, output_row_num=37, filtered_row_num=0, merged_row_num=10. elapsed time=0.017237s. cumulative_compaction_policy=size_based, compact_row_per_second=2725
I20250925 13:30:14.853601 9991 daemon.cpp:221] os physical memory 60.45 GB. process memory used 20.16 GB(= 28.41 GB[vm/rss] - 8.25 GB[tc/jemalloc_cache] + 0[reserved] + 0B[waiting_refresh]), limit 54.41 GB, soft limit 48.97 GB. sys available memory 32.45 GB(= 32.45 GB[proc/available] - 0[reserved] - 0B[waiting_refresh]), low water mark 3.02 GB, warning water mark 6.05 GB.
I20250925 13:30:14.965724 8719 compaction.cpp:369] start cumulative compaction. tablet=123809403, output_version=[1436188-1436657], permits: 31
I20250925 13:30:14.965821 8719 merger.cpp:405] Start to do vertical compaction, tablet_id: 123809403
I20250925 13:30:14.965832 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 1768 way cnt: 9 batch size: 4064
I20250925 13:30:14.967693 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 3276 way cnt: 9 batch size: 4064
I20250925 13:30:14.968807 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 3207 way cnt: 9 batch size: 4064
I20250925 13:30:14.969885 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 2402 way cnt: 9 batch size: 4064
I20250925 13:30:14.970922 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 2182 way cnt: 9 batch size: 4064
I20250925 13:30:14.972051 8719 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 123809403 group data size: 0 row num: 52 consume bytes: 1820 way cnt: 9 batch size: 4064
I20250925 13:30:14.979074 8719 tablet_meta_manager.cpp:293] remove old version delete bitmap, tablet_id: 123809403 version: 1436657, removed keys size: 0
I20250925 13:30:14.979190 8719 compaction.cpp:786] succeed to do cumulative compaction is_vertical=1. tablet=123809403, output_version=[1436188-1436657], current_max_version=1436657, disk=/msun/data/disk1/doris, segments=9, input_rowset_size=68843, output_rowset_size=11921, input_row_num=57, output_row_num=48, filtered_row_num=0, merged_row_num=9. elapsed time=0.013483s. cumulative_compaction_policy=size_based, compact_row_per_second=4225
I20250925 13:30:15.005508 9942 stream_load.cpp:208] new income streaming load request.id=5b4f90363f4c199e-0e1b8b98f2c9e6a2, job_id=-1, txn_id=-1, label=c726670d-347f-41a3-894e-c94e8be4d041, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
I20250925 13:30:15.005513 9944 stream_load.cpp:208] new income streaming load request.id=3740131af37e2b0b-36fcda8c5dd903bd, job_id=-1, txn_id=-1, label=1cd21da5-37c8-4325-9be5-177802db0ca6, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
W20250925 13:30:15.009006 9942 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=5b4f90363f4c199e-0e1b8b98f2c9e6a2, job_id=-1, txn_id=174913525, label=c726670d-347f-41a3-894e-c94e8be4d041, elapse(s)=0
I20250925 13:30:15.009048 9942 stream_load.cpp:214] finished to handle HTTP header, id=5b4f90363f4c199e-0e1b8b98f2c9e6a2, job_id=-1, txn_id=174913525, label=c726670d-347f-41a3-894e-c94e8be4d041, elapse(s)=0
W20250925 13:30:15.009382 9944 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=3740131af37e2b0b-36fcda8c5dd903bd, job_id=-1, txn_id=174913524, label=1cd21da5-37c8-4325-9be5-177802db0ca6, elapse(s)=0
I20250925 13:30:15.009416 9944 stream_load.cpp:214] finished to handle HTTP header, id=3740131af37e2b0b-36fcda8c5dd903bd, job_id=-1, txn_id=174913524, label=1cd21da5-37c8-4325-9be5-177802db0ca6, elapse(s)=0
I20250925 13:30:15.013211 9949 stream_load.cpp:208] new income streaming load request.id=d24cb9a6194cf2cc-b7c90fcc28b3e2a4, job_id=-1, txn_id=-1, label=fe9b4a48-58bf-4c3f-98e9-927c58316cc5, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
I20250925 13:30:15.014042 9951 stream_load.cpp:208] new income streaming load request.id=c94acb98b6fde29a-74aecf4905eaca8f, job_id=-1, txn_id=-1, label=51d440e6-055f-4d5a-a324-047a44b7ecb5, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
W20250925 13:30:15.015340 9949 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=d24cb9a6194cf2cc-b7c90fcc28b3e2a4, job_id=-1, txn_id=174913526, label=fe9b4a48-58bf-4c3f-98e9-927c58316cc5, elapse(s)=0
I20250925 13:30:15.015370 9949 stream_load.cpp:214] finished to handle HTTP header, id=d24cb9a6194cf2cc-b7c90fcc28b3e2a4, job_id=-1, txn_id=174913526, label=fe9b4a48-58bf-4c3f-98e9-927c58316cc5, elapse(s)=0
W20250925 13:30:15.016170 9951 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=c94acb98b6fde29a-74aecf4905eaca8f, job_id=-1, txn_id=174913527, label=51d440e6-055f-4d5a-a324-047a44b7ecb5, elapse(s)=0
I20250925 13:30:15.016186 9951 stream_load.cpp:214] finished to handle HTTP header, id=c94acb98b6fde29a-74aecf4905eaca8f, job_id=-1, txn_id=174913527, label=51d440e6-055f-4d5a-a324-047a44b7ecb5, elapse(s)=0
I20250925 13:30:15.017040 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913525
I20250925 13:30:15.017047 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913525, transaction_id=174913525, partition_id_size=0
I20250925 13:30:15.017063 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913524
I20250925 13:30:15.017077 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913525
I20250925 13:30:15.017079 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913526
I20250925 13:30:15.017082 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913525
I20250925 13:30:15.017086 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913525, transaction_id=174913525
I20250925 13:30:15.017295 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913524, transaction_id=174913524, partition_id_size=0
I20250925 13:30:15.017302 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913524
I20250925 13:30:15.017304 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913524
I20250925 13:30:15.017306 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913524, transaction_id=174913524
I20250925 13:30:15.017481 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913526, transaction_id=174913526, partition_id_size=0
I20250925 13:30:15.017486 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913526
I20250925 13:30:15.017489 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913526
I20250925 13:30:15.017493 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913526, transaction_id=174913526
I20250925 13:30:15.018839 9951 stream_load.cpp:208] new income streaming load request.id=bb450a0153fd960c-ba9bcd376508949f, job_id=-1, txn_id=-1, label=de2dd488-f77f-4ac0-b5b1-88589be7c32a, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
I20250925 13:30:15.018978 9949 stream_load.cpp:208] new income streaming load request.id=c1409c236e7de51e-cb849b32381e60b6, job_id=-1, txn_id=-1, label=e0e98561-5706-4460-a707-92a24cc45a51, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
W20250925 13:30:15.020848 9951 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=bb450a0153fd960c-ba9bcd376508949f, job_id=-1, txn_id=174913528, label=de2dd488-f77f-4ac0-b5b1-88589be7c32a, elapse(s)=0
I20250925 13:30:15.020870 9951 stream_load.cpp:214] finished to handle HTTP header, id=bb450a0153fd960c-ba9bcd376508949f, job_id=-1, txn_id=174913528, label=de2dd488-f77f-4ac0-b5b1-88589be7c32a, elapse(s)=0
W20250925 13:30:15.021673 9949 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=c1409c236e7de51e-cb849b32381e60b6, job_id=-1, txn_id=174913529, label=e0e98561-5706-4460-a707-92a24cc45a51, elapse(s)=0
I20250925 13:30:15.021693 9949 stream_load.cpp:214] finished to handle HTTP header, id=c1409c236e7de51e-cb849b32381e60b6, job_id=-1, txn_id=174913529, label=e0e98561-5706-4460-a707-92a24cc45a51, elapse(s)=0
I20250925 13:30:15.021919 8717 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 127808358 group data size: 0 row num: 53 consume bytes: 3678 way cnt: 6 batch size: 4064
I20250925 13:30:15.023236 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913527
I20250925 13:30:15.023242 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913527, transaction_id=174913527, partition_id_size=0
I20250925 13:30:15.023248 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913527
I20250925 13:30:15.023255 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913527
I20250925 13:30:15.023249 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913528
I20250925 13:30:15.023262 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913527, transaction_id=174913527
I20250925 13:30:15.023267 3805033 task_worker_pool.cpp:337] successfully submit task|type=CLEAR_TRANSACTION_TASK|signature=174913529
I20250925 13:30:15.023437 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913528, transaction_id=174913528, partition_id_size=0
I20250925 13:30:15.023442 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913528
I20250925 13:30:15.023444 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913528
I20250925 13:30:15.023447 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913528, transaction_id=174913528
I20250925 13:30:15.023628 8826 task_worker_pool.cpp:1638] get clear transaction task. signature=174913529, transaction_id=174913529, partition_id_size=0
I20250925 13:30:15.023633 8826 storage_engine.cpp:715] begin to clear transaction task. transaction_id=174913529
I20250925 13:30:15.023636 8826 storage_engine.cpp:743] finish to clear transaction task. transaction_id=174913529
I20250925 13:30:15.023648 8826 task_worker_pool.cpp:1654] finish to clear transaction task. signature=174913529, transaction_id=174913529
I20250925 13:30:15.024268 9949 stream_load.cpp:208] new income streaming load request.id=0c48aca787780b0c-b0dd5e4ea01d5fb9, job_id=-1, txn_id=-1, label=4143f6c3-b56b-444a-ab09-e73be300ef45, elapse(s)=0, db=cdrapp_cxrmyy, tbl=dw_dwb_sv_os_out_seedoc_info, group_commit=0
W20250925 13:30:15.026239 9949 stream_load.cpp:678] plan streaming load failed. errmsg=[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = No available backendsid=0c48aca787780b0c-b0dd5e4ea01d5fb9, job_id=-1, txn_id=174913530, label=4143f6c3-b56b-444a-ab09-e73be300ef45, elapse(s)=0
I20250925 13:30:15.026258 9949 stream_load.cpp:214] finished to handle HTTP header, id=0c48aca787780b0c-b0dd5e4ea01d5fb9, job_id=-1, txn_id=174913530, label=4143f6c3-b56b-444a-ab09-e73be300ef45, elapse(s)=0
I20250925 13:30:15.102994 8720 compaction.cpp:369] start cumulative compaction. tablet=272954749, output_version=[1207810-1207959], permits: 31
I20250925 13:30:15.103188 8720 merger.cpp:405] Start to do vertical compaction, tablet_id: 272954749
I20250925 13:30:15.103202 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1054 way cnt: 3 batch size: 4064
I20250925 13:30:15.105324 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 2356 way cnt: 3 batch size: 4064
I20250925 13:30:15.106732 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1968 way cnt: 3 batch size: 4064
I20250925 13:30:15.107877 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 2319 way cnt: 3 batch size: 4064
I20250925 13:30:15.108994 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1884 way cnt: 3 batch size: 4064
I20250925 13:30:15.110139 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1147 way cnt: 3 batch size: 4064
I20250925 13:30:15.111236 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1546 way cnt: 3 batch size: 4064
I20250925 13:30:15.112372 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1608 way cnt: 3 batch size: 4064
I20250925 13:30:15.113458 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1718 way cnt: 3 batch size: 4064
I20250925 13:30:15.114539 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 2002 way cnt: 3 batch size: 4064
I20250925 13:30:15.115630 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1547 way cnt: 3 batch size: 4064
I20250925 13:30:15.116683 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1240 way cnt: 3 batch size: 4064
I20250925 13:30:15.117770 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1333 way cnt: 3 batch size: 4064
I20250925 13:30:15.118845 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 2101 way cnt: 3 batch size: 4064
I20250925 13:30:15.119998 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1231 way cnt: 3 batch size: 4064
I20250925 13:30:15.121088 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1023 way cnt: 3 batch size: 4064
I20250925 13:30:15.122166 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1375 way cnt: 3 batch size: 4064
I20250925 13:30:15.123241 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 868 way cnt: 3 batch size: 4064
I20250925 13:30:15.124351 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1178 way cnt: 3 batch size: 4064
I20250925 13:30:15.125392 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 899 way cnt: 3 batch size: 4064
I20250925 13:30:15.126515 8720 merger.cpp:388] estimate batch size for vertical compaction, tablet id: 272954749 group data size: 0 row num: 31 consume bytes: 1256 way cnt: 3 batch size: 4064
I20250925 13:30:15.136511 8720 tablet_meta_manager.cpp:293] remove old version delete bitmap, tablet_id: 272954749 version: 1207959, removed keys size: 0
这种的错误可能的原因是什么，怎么进一步排查和解决
