Doris Streamloader在workers参数不同时，成功导入的数据量不一致。

Viewed 45
ingestion

使用Doris Streamloader导入一个csv文件，目的表是明细模型表（Duplicate Key)，命令参数workers数量1和'>1'，导入表行数量有差异。都提示成功导入。
workes=1的结果
image.png
workers=2或者workers>1的结果
image.png
image.png

huwei20551
2 Answers

方便来个具体的案例不，需要提供一下表结构、数据文件以及你的操作。doris是什么版本呢？

黄海军@SelectDB4828

doris版本3.0.2
数据库脚本
drop table IF EXISTS database1.f_ods_hq1;
CREATE TABLE IF NOT EXISTS database1.f_ods_hq1
(
dm VARCHAR(20) NOT NULL COMMENT "dm",
jys VARCHAR(20) NOT NULL COMMENT "jys",
jyr INT NOT NULL COMMENT "jyr",
jysj INT NOT NULL COMMENT "jysj",
cjj INT COMMENT "cjj",
cjl BIGINT COMMENT "cjl",
cje BIGINT COMMENT "cje",
cjbs BIGINT COMMENT "cjbs",
iopv INT COMMENT "IOPV",
cjbz CHAR COMMENT "cjbz",
bsbz CHAR COMMENT "BSbz",
drljcjl BIGINT COMMENT "drljcjl",
drcje BIGINT COMMENT "drcje",
zgj INT COMMENT "zgj",
zdj INT COMMENT "zdj",
kpj INT COMMENT "kpj",
qsp INT COMMENT "qsp",
ssj1 INT COMMENT "ssj1",
ssj2 INT COMMENT "ssj2",
ssj3 INT COMMENT "ssj3",
ssj4 INT COMMENT "ssj4",
ssj5 INT COMMENT "ssj5",
ssj6 INT COMMENT "ssj6",
ssj7 INT COMMENT "ssj7",
ssj8 INT COMMENT "ssj8",
ssj9 INT COMMENT "ssj9",
ssj10 INT COMMENT "ssj10",
ssl1 INT COMMENT "ssl1",
ssl2 INT COMMENT "ssl2",
ssl3 INT COMMENT "ssl3",
ssl4 INT COMMENT "ssl4",
ssl5 INT COMMENT "ssl5",
ssl6 INT COMMENT "ssl6",
ssl7 INT COMMENT "ssl7",
ssl8 INT COMMENT "ssl8",
ssl9 INT COMMENT "ssl9",
ssl10 INT COMMENT "ssl10",
sbj1 INT COMMENT "sbj1",
sbj2 INT COMMENT "sbj2",
sbj3 INT COMMENT "sbj3",
sbj4 INT COMMENT "sbj4",
sbj5 INT COMMENT "sbj5",
sbj6 INT COMMENT "sbj6",
sbj7 INT COMMENT "sbj7",
sbj8 INT COMMENT "sbj8",
sbj9 INT COMMENT "sbj9",
sbj10 INT COMMENT "sbj10",
sbl1 INT COMMENT "sbl1",
sbl2 INT COMMENT "sbl2",
sbl3 INT COMMENT "sbl3",
sbl4 INT COMMENT "sbl4",
sbl5 INT COMMENT "sbl5",
sbl6 INT COMMENT "sbl6",
sbl7 INT COMMENT "sbl7",
sbl8 INT COMMENT "sbl8",
sbl9 INT COMMENT "sbl9",
sbl10 INT COMMENT "sbl10",
jqpjjsj INT COMMENT "jqpjjsj",
jqpjjbj INT COMMENT "jqpjjbj",
jszl BIGINT COMMENT "jszl",
jbzl BIGINT COMMENT "jbzl",
bjqzs INT COMMENT "bjqzs",
pzzs INT COMMENT "pzzs",
szpzs INT COMMENT "szpzs",
xdpzs INT COMMENT "xdpzs",
sppzs INT COMMENT "sppzs",
cs1 INT COMMENT "cs1"
)
DUPLICATE KEY(dm, jys, jyr,jysj)
DISTRIBUTED BY HASH(dm, jyr) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 3"
);
命令行：doris-streamloader --u "bgusr" --source_file="hq2.csv" --url="http://127.0.0.1:8030" --header="column_separator:,?skip_lines:1" --db="database1" --table="f_ods_hq1" --debug=true --workers=1

源数据文件：hq2.csv (提取码：1134)

huwei20551