doris-streamloader 与 curl 提交 Stream Load 导入作业 遇到的问题

使用 doris-streamloader 运行不成功

• 参考: https://doris.apache.org/zh-CN/docs/ecosystem/doris-streamloader

运行命令

运行报错分析

检查了好几遍, 确认csv文件中的字段分隔符,字段个数

Reason: actual column number in csv file is  less than  schema column number.actual number: 1, schema column number: 66; line delimiter: [
], column separator: [  ], result values:[G1080140010001000102025080100000151,151,2025080100,441,441,441,0,1,2,1,晋KV6505,NULL,1,NULL,16,NULL,27,NULL,4,v1,G10801401002535,30,223B22BC,C2B7CDF899010001,9901000281900792,45,C2B7CDF899010001,23,9901,99010032232480254627,2,3,NULL,NULL,NULL,NULL,NULL,NULL,2,1,"2025-08-01 00:55:59",0,1,"2025-08-01 00:56:20",0,0,NULL,NULL,G1080,G1080140010,G108014001000100010,0,2,NULL,2,NULL,0,NULL,NULL,NULL,"2025-08-01 00:56:30",NULL,NULL,NULL,NULL,NULL, ]. src line [G1080140010001000102025080100000151,151,2025080100,441,441,441,0,1,2,1,晋KV6505,NULL,1,NULL,16,NULL,27,NULL,4,v1,G10801401002535,30,223B22BC,C2B7CDF899010001,9901000281900792,45,C2B7CDF899010001,23,9901,99010032232480254627,2,3,NULL,NULL,NULL,NULL,NULL,NULL,2,1,"2025-08-01 00:55:59",0,1,"2025-08-01 00:56:20",0,0,NULL,NULL,G1080,G1080140010,G108014001000100010,0,2,NULL,2,NULL,0,NULL,NULL,NULL,"2025-08-01 00:56:30",NULL,NULL,NULL,NULL,NULL];
Reason: actual column number in csv file is  less than  schema column number.actual number: 1, schema column number: 66; line delimiter: [
], column separator: [  ], result values:[G1080140010001000102025080100000156,156,2025080100,441,441,441,0,1,2,1,晋L01209D,NULL,5,NULL,16,NULL,27,NULL,1,v1,G10801401002535,32,6DC19F34,C2B7CDF899010001,9901000281658455,45,C2B7CDF899010001,23,9901,99011032232480288335,2,3,NULL,NULL,NULL,NULL,NULL,NULL,2,1,"2025-08-01 00:57:46",0,1,"2025-08-01 00:58:10",0,0,NULL,NULL,G1080,G1080140010,G108014001000100010,0,2,NULL,2,NULL,0,NULL,NULL,NULL,"2025-08-01 00:58:20",NULL,NULL,NULL,NULL,NULL, ]. src line [G1080140010001000102025080100000156,156,2025080100,441,441,441,0,1,2,1,晋L01209D,NULL,5,NULL,16,NULL,27,NULL,1,v1,G10801401002535,32,6DC19F34,C2B7CDF899010001,9901000281658455,45,C2B7CDF899010001,23,9901,99011032232480288335,2,3,NULL,NULL,NULL,NULL,NULL,NULL,2,1,"2025-08-01 00:57:46",0,1,"2025-08-01 00:58:10",0,0,NULL,NULL,G1080,G1080140010,G108014001000100010,0,2,NULL,2,NULL,0,NULL,NULL,NULL,"2025-08-01 00:58:20",NULL,NULL,NULL,NULL,NULL];

使用 curl 提交 Stream Load 导入作业 运行成功

• 使用原生的命令, 就可以运行成功
  https://doris.apache.org/zh-CN/docs/data-operate/import/import-way/stream-load-manual?_highlight=stream&_highlight=load

运行命令

所有的运行命令

-- 运行不成功的命令
/opt/apps/doris-streamloader \
--source_file="etc_pass_rec_08.csv" \
--url="http://10.11.59.162:8030" \
--header="column_separator:," \
--header="columns:list_no,record_no,hour_batch_no,pay_fee,pay_fee_real,fee,discount_fee,direction,computer_order,obu_sign,vehicle_plate,vehicle_plate_original,plate_color,plate_color_original,vehicle_type,vehicle_type_original,vehicle_class,fee_vehicle_class,antenna_id,rate_module_version,rate_params_version,consume_time,obu_mac,obu_issue,obu_sn,obu_version,card_issue,card_type,card_net_no,card_no,is_local,type,image_serial_no,plate_image_serial_no,side_image_serial_no,image_link,plate_image_link,side_image_link,image_status,trans_status,trans_time,statistic,charging_status,charging_time,flow_status,bill_status,bill_time,bill_id,road_code,section_code,node_code,confidence,send_kf_flag,send_kf_time,send_zj_flag,send_zj_time,bill_process,settle_id,node_mileage,discount_type,create_time,update_time,spare1,spare2,spare3,spare4" \
--header="skip_header:1" \
--db="tmp" \
--table="t_etc_pass_record_202508_test" \
--u="用户名" \
--p="密码"

-- 成功命令
curl --location-trusted -u 用户名:密码 \
    -H "Expect:100-continue" \
    -H "column_separator:," \
    -H "columns:list_no,record_no,hour_batch_no,pay_fee,pay_fee_real,fee,discount_fee,direction,computer_order,obu_sign,vehicle_plate,vehicle_plate_original,plate_color,plate_color_original,vehicle_type,vehicle_type_original,vehicle_class,fee_vehicle_class,antenna_id,rate_module_version,rate_params_version,consume_time,obu_mac,obu_issue,obu_sn,obu_version,card_issue,card_type,card_net_no,card_no,is_local,type,image_serial_no,plate_image_serial_no,side_image_serial_no,image_link,plate_image_link,side_image_link,image_status,trans_status,trans_time,statistic,charging_status,charging_time,flow_status,bill_status,bill_time,bill_id,road_code,section_code,node_code,confidence,send_kf_flag,send_kf_time,send_zj_flag,send_zj_time,bill_process,settle_id,node_mileage,discount_type,create_time,update_time,spare1,spare2,spare3,spare4" \
    -T etc_pass_rec_08.csv \
    -XPUT http://10.11.59.162:8030/api/tmp/t_etc_pass_record_202508_test/_stream_load