version:2.1.10
场景:使用Stream Load导入数据做实时分析,每秒会调用10次左右的。每次请求的文件远低于数值
这个limit具体是什么产生的,如何调高配置
version:2.1.10
场景:使用Stream Load导入数据做实时分析,每秒会调用10次左右的。每次请求的文件远低于数值
这个limit具体是什么产生的,如何调高配置
我指的是TmpFileMgr里的MAX_TOTAL_FILE_SIZE_BYTES
public synchronized TmpFile upload(UploadFile uploadFile) throws TmpFileException {
if (uploadFile.file.getSize() > MAX_SINGLE_FILE_SIZE) {
throw new TmpFileException("File size " + uploadFile.file.getSize()
+ " exceed limit " + MAX_SINGLE_FILE_SIZE);
}
if (totalFileSize + uploadFile.file.getSize() > MAX_TOTAL_FILE_SIZE_BYTES) {
throw new TmpFileException("Total file size will exceed limit " + MAX_TOTAL_FILE_SIZE_BYTES);
}
if (fileMap.size() > MAX_TOTAL_FILE_NUM) {
throw new TmpFileException("Number of temp file " + fileMap.size() + " exceed limit " + MAX_TOTAL_FILE_NUM);
}
long fileId = fileIdGenerator.incrementAndGet();
String fileUUID = UUID.randomUUID().toString();
TmpFile tmpFile = new TmpFile(fileId, fileUUID, uploadFile.file.getOriginalFilename(),
uploadFile.file.getSize(), uploadFile.columnSeparator);
try {
tmpFile.save(uploadFile.file);
} catch (IOException e) {
throw new TmpFileException("Failed to upload file. Reason: " + e.getMessage());
}
fileMap.put(tmpFile.id, tmpFile);
totalFileSize += uploadFile.file.getSize();
return tmpFile;
}
这里的totalFileSize在实例化后一直在增加,我没看到哪里有清空的方式,比如:UploadAction方法中创建的
private static TmpFileMgr fileMgr = new TmpFileMgr(Config.tmp_dir);
导致这个totalFileSize始终在上升,最终导致Total file size will exceed limit 1073741824