doris 外表查询 paimon 的聚合表,be 报错 IOException: java.lang.OutOfMemoryError: Java heap space

Viewed 23

查询的paimon表结构:
image.png

现象描述:
这个表是一个日志数据(每天30几亿条)的聚合表,src_contents 数组嵌套表里的数据比较多,查询语句:
select * from paimon_catalog.oss_paimon_db.pile_log_agg_list
where ymd='2025-03-03'
and terminal_id='3201060022867601'
and flow_id='32010600228676012503022015490605';
报错:image.png
有的flow_id 能查出来,有的flow_id查就报内存溢出;
我想问是否有哪个查数可以调这个使用内存的大小

be 节点报错信息:
image.png
2025-02-27 19:31:56,457 WARN (mysql-nio-pool-85717|22420597) [Coordinator.getNext():1376] query failed: (172.16.201.242)[CANCELLED]cur path: hdfs://dummyPath. IOException: java.io.IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: OutOfMemoryError: Java heap space
2025-02-27 19:31:56,457 WARN (mysql-nio-pool-85717|22420597) [StmtExecutor.executeAndSendResult():1868] cancel fragment query_id:9f80bba8f9ef41cd-9e9b3c7eb496391e cause errCode = 2, detailMessage = (172.16.201.242)[CANCELLED]cur path: hdfs://dummyPath. IOException: java.io.IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: OutOfMemoryError: Java heap space
2025-02-27 19:31:56,458 WARN (mysql-nio-pool-85717|22420597) [Coordinator.cancel():1496] Query 9f80bba8f9ef41cd-9e9b3c7eb496391e already in abnormal status Status [errorCode=CANCELLED, errorMsg=(172.16.201.242)[CANCELLED]cur path: hdfs://dummyPath. IOException: java.io.IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: IOException: java.lang.OutOfMemoryError: Java heap space
CAUSED BY: OutOfMemoryError: Java heap space], but received cancel again,so that send cancel to BE again
java.lang.Exception: cancel failed
at org.apache.doris.qe.Coordinator.cancel(Coordinator.java:1498) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeAndSendResult(StmtExecutor.java:1869) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1736) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleQueryWithRetry(StmtExecutor.java:804) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:758) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:538) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:517) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:328) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:260) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:288) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:341) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]

1 Answers

这个你调整下be里面的jvm配置,增大jvm内存
-Xmx2048m 这是默认的,你可以改更大