spark-doris-connector 写入异常,sql却提示成功

Viewed 26

image.png
spark-doris-connector版本 25.1.0
spark版本 aws-emr的spark 3.5.5
执行 insert into doris视图,spark-executor 有错误日志。但是却提示该insert 语句执行成功

1 Answers

CREATE TEMPORARY VIEW dwd_shopify_pub_shop_nf_d
USING doris
OPTIONS(
'table.identifier' = 'shopify_bussi.dwd_shopify_pub_shop_nf'
,'doris.fenodes' = ${doris_fe2}
,'user' = ${doris_user}
,'password' = ${doris_pwd}
,'doris.sink.properties.format' = 'json'
,'doris.sink.max-retries'='3'
,'doris.sink.properties.max_filter_ratio'='0'
);

 insert into dwd_shopify_pub_shop_nf_d(shop_name)
    select 
           t1.name                       as shop_name
     
    from cw_middle_platform.ods_shopify_shop_nf t1
    inner join cw_middle_platform.temp_shop_id_domain_mapping t2
    on t1.myshopify_domain = t2.myshopify_domain limit 10;