fail to offer request to the work pool

Viewed 16

Doris 版本

  • 2.1.8

fe.log 中报错日志(是BE节点: fail to offer request to the work pool)

2025-09-24 14:28:41,463 WARN (backend-rpc-callback-26|311) [Coordinator$PipelineExecContext$1.onSuccess():3502] Failed to cancel query 81260a6ee50543e7-8745ad09ee0184a3 instance initiated=true done=false backend: 13026,fragment id=F15, reason: Status [errorCode=CANCELLED, errorMsg=(10.11.59.163)[CANCELLED]fail to offer request to the work pool, pool=PriorityThreadPool(name=brpc_light, queue_size=10240/10240, active_thread=128/128, total_get_wait_time=1456392891832966, total_put_wait_time=1526466951)]
2025-09-24 14:28:41,463 INFO (mysql-nio-pool-156|4878) [Coordinator$PipelineExecContext.cancelFragment():3485] Frangment has already been cancelled. Query 81260a6ee50543e7-8745ad09ee0184a3 backend: 13029, fragment id=F07
2025-09-24 14:28:41,463 WARN (mysql-nio-pool-156|4878) [ConnectProcessor.handleQueryException():471] Process one query failed because.
org.apache.doris.common.UserException: errCode = 2, detailMessage = (10.11.59.163)[CANCELLED]fail to offer request to the work pool, pool=PriorityThreadPool(name=brpc_light, queue_size=10240/10240, active_thread=128/128, total_get_wait_time=1456392891832966, total_put_wait_time=1526466951)
        at org.apache.doris.qe.Coordinator.waitPipelineRpc(Coordinator.java:1287) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.Coordinator.sendPipelineCtx(Coordinator.java:1081) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.Coordinator.execInternal(Coordinator.java:778) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.Coordinator.exec(Coordinator.java:694) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeAndSendResult(StmtExecutor.java:1872) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleCacheStmt(StmtExecutor.java:1738) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1812) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryWithRetry(StmtExecutor.java:851) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:805) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:561) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:537) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:344) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:225) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:285) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:313) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:483) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]

1 Answers

这个看着是线程超了,可以先调整 be conf brpc_light_work_pool_max_queue_size 默认是10240。
在复现的时候可以收集个 pstack : pstack be_pid > be_ps.txt 发我主页微信