4.0 不是新增 Spill Disk 功能,当离线计算任务的内存占用超出阈值时,自动将部分中间数据写入磁盘,为什么还会报错

Viewed 27

已经做了设置
set enable_spill=true;
set exec_mem_limit = 10g;
set query_timeout = 3600;
还是会报错
1105 - errCode = 2, detailMessage = (t247)[MEM_LIMIT_EXCEEDED]Cancel Top Memory task: TaskId=2970cafa057142c0-9fe2cdbb51f6fcde, Memory(Used=14.50 GB, Limit=28.12 GB, Peak=14.62 GB). because sys available memory less than low water mark. in backend t247, os physical memory 31.24 GB. process memory used 17.12 GB, limit 28.12 GB. sys available memory 1.51 GB, low water mark 1.56 GB. execute again after enough memory, details see be.INFO.

1 Answers

测试了很多次。显示有落盘,但是还是报内存不足,我的参数是set exec_mem_limit = 10g;
set enable_spill=true;
set query_timeout = 3600;服务器配置2台节点,一台32G内存,一台48G内存image.pngimage.png