首先是be:
- 在warning中出现大量的:
W20250508 00:04:41.645372 21113 fragment_mgr.cpp:1305] Could not find the query id:879b1437cda64070-ac3ae0649bd6dcc0 fragment id:2 to cancel
W20250508 00:05:41.647545 21067 fragment_mgr.cpp:1305] Could not find the query id:760845c7142b4791-a5a9341ab8309046 fragment id:11 to cancel
W20250508 00:05:41.647718 21137 fragment_mgr.cpp:1305] Could not find the query id:760845c7142b4791-a5a9341ab8309046 fragment id:6 to cancel
- be中出现大量的INTERNAL_ERROR,如在10.0.3.30上出现如下日志
W20250508 00:05:55.331712 20350 status.h:415] meet error status: [INTERNAL_ERROR]Failed to connect to backend 11888328: [E1008]Reached timeout=60000ms @10.0.3.31:8060
而在10.0.3.31上出现的又是类似这种
W20250508 00:05:55.331712 20350 status.h:415] meet error status: [INTERNAL_ERROR]Failed to connect to backend 11888328: [E1008]Reached timeout=60000ms @10.0.3.30:8060
再就是fe:
- 在fe当中出现大量的如下日志
2025-05-09 23:33:26,768 WARN (Auto Analysis Job Executor-0|529) [BaseAnalysisTask.runQuery():313] Failed to execute sql SELECT CONCAT(11590, '-', -1, '-', 'properties') AS `id`, 0 AS `catalog_id`, 11228 AS `db_id`, 11590 AS `tbl_id`, -1 AS `idx_id`, 'properties' AS `col_id`, NULL AS `part_id`, 704434 AS `row_count`, ROUND(NDV(`properties`) * 1) as `ndv`, ROUND(SUM(CASE WHEN `properties` IS NULL THEN 1 ELSE 0 END) * 1) AS `null_count`, SUBSTRING(CAST('null' AS STRING), 1, 1024) AS `min`, SUBSTRING(CAST('null' AS STRING), 1, 1024) AS `max`, COUNT(1) * 4 * 1 AS `data_size`, NOW() FROM ( SELECT * FROM `internal`.`mid_analyze`.`dm_b_iot_report` ) as t
2025-05-09 23:33:26,768 WARN (Expired Analysis Task Killer|55) [AnalysisTaskWrapper.cancel():85] Job id [12512666], Task id [12512680], catalog [internal], db [mid_analyze], table [dm_b_iot_report], column [properties] cancelled, cost time:58
2025-05-09 23:33:26,768 WARN (Auto Analysis Job Executor-0|529) [AnalysisTaskWrapper.run():76] Analyze Job id [12512666], Task id [12512680], catalog [internal], db [mid_analyze], table [dm_b_iot_report], column [properties] failed.
java.util.concurrent.ExecutionException: java.lang.RuntimeException: ColStatsData is invalid, skip analyzing. ('11590--1-properties',0,11228,11590,-1,'properties',null,704434,0,58159,'null','null',232636,'2025-05-09 23:33:26')
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_43]
at org.apache.doris.statistics.AnalysisTaskWrapper.run(AnalysisTaskWrapper.java:67) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]
Caused by: java.lang.RuntimeException: ColStatsData is invalid, skip analyzing. ('11590--1-properties',0,11228,11590,-1,'properties',null,704434,0,58159,'null','null',232636,'2025-05-09 23:33:26')
at org.apache.doris.statistics.BaseAnalysisTask.runQuery(BaseAnalysisTask.java:299) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.statistics.OlapAnalysisTask.doSample(OlapAnalysisTask.java:136) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.statistics.OlapAnalysisTask.doExecute(OlapAnalysisTask.java:96) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.statistics.BaseAnalysisTask.execute(BaseAnalysisTask.java:175) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.statistics.AnalysisTaskWrapper.lambda$new$0(AnalysisTaskWrapper.java:43) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.statistics.AnalysisTaskWrapper$$Lambda$2861/1752430766.call(Unknown Source) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_43]
at org.apache.doris.statistics.AnalysisTaskWrapper.run(AnalysisTaskWrapper.java:66) ~[doris-fe.jar:1.2-SNAPSHOT]
... 5 more
2025-05-09 23:34:34,946 WARN (mysql-nio-pool-6837|223028) [ResultReceiver.cancel():231] ResultReceiver of query e6a09fc92d484809-a97c5714a4e28c3a cancel failed, typically means the future is finished
2025-05-09 23:34:35,055 WARN (mysql-nio-pool-6837|223028) [ResultReceiver.cancel():231] ResultReceiver of query 4e6fd32a7a0a46f5-a2bcb150db131a65 cancel failed, typically means the future is finished
- timeout日志
2025-05-09 23:33:09,545 WARN (agent-task-pool-1999|222994) [AgentBatchTask.run():191] task exec error. backend[10036]
org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:179) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:464) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:362) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:245) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[libthrift-0.16.0.jar:0.16.0]
at org.apache.doris.thrift.BackendService$Client.recvSubmitTasks(BackendService.java:214) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
at org.apache.doris.thrift.BackendService$Client.submitTasks(BackendService.java:201) ~[fe-common-1.2-SNAPSHOT.jar:1.2-SNAPSHOT]
at org.apache.doris.task.AgentBatchTask.submitTasks(AgentBatchTask.java:218) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.task.AgentBatchTask.run(AgentBatchTask.java:188) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_43]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_43]
at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[?:1.8.0_43]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_43]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_43]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_43]
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_43]
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:177) ~[libthrift-0.16.0.jar:0.16.0]
... 14 more
2025-05-09 23:33:09,557 WARN (thrift-server-pool-10|20859) [MasterImpl.finishTask():122] cannot find task. type: PUBLISH_VERSION, backendId: 11888328, signature: 25334837
- io异常日志
2025-05-09 23:33:16,887 WARN (mysql-nio-pool-6836|222999) [MysqlConnectProcessor.processOnce():466] Null packet received from network. remote: 10.0.3.30:36174
2025-05-09 23:33:16,887 WARN (mysql-nio-pool-6836|222999) [ReadListener.lambda$handleEvent$0():60] Exception happened in one session(org.apache.doris.qe.ConnectContext@59ee7a1).
java.io.IOException: Error happened when receiving packet.
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:467) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener$$Lambda$2183/1729626238.run(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]
1. 请问这些日志有什么影响,是什么原因造成的.
2. 昨天doris挂了一次,不知道什么原因,领导说是查询把doris搞蹦了,但是doris没有自动恢复,需要手动接入,不清楚为什么doris不会自动恢复
