通过Spark Doris Connector的方式访问Doris表,提示报错fragment_executor execute failed

Viewed 24

错误提示
image.png
image.png

问题描述:
doris2.1.8中两张表
CREATE TABLE orders ( order_id bigint NULL, user_id bigint NULL, product_id bigint NULL, order_time datetime NULL, ship_time datetime NULL, amount double NULL, quantity int NULL, discount float NULL, total decimal(10,2) NULL, order_status text NULL, payment_method text NULL, payment_time datetime NULL, shipping_address text NULL, shipping_zip text NULL, tracking_number text NULL, is_gift boolean NULL, gift_note text NULL, coupon_code text NULL, currency varchar(5) NULL, exchange_rate double NULL, device_id text NULL, app_channel text NULL, order_type tinyint NULL, invoice_required boolean NULL, invoice_title text NULL, loyalty_points int NULL, feedback text NULL, delivery_type text NULL, delivery_eta date NULL, metadata json NULL ) ENGINE=OLAP DUPLICATE KEY(order_id, user_id, product_id) PARTITION BY RANGE(order_time) (PARTITION p202405 VALUES [('0000-01-01 00:00:00'), ('2024-06-01 00:00:00')), PARTITION p202406 VALUES [('2024-06-01 00:00:00'), ('2024-07-01 00:00:00')), PARTITION pMAX VALUES [('2024-07-01 00:00:00'), ('9999-12-31 00:00:00'))) DISTRIBUTED BY HASH(order_id) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "min_load_replica_num" = "-1", "is_being_synced" = "false", "storage_medium" = "hdd", "storage_format" = "V2", "inverted_index_storage_format" = "V1", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false", "group_commit_interval_ms" = "10000", "group_commit_data_bytes" = "134217728" );

CREATE TABLE orders_new ( order_id bigint NULL, user_id bigint NULL, product_id bigint NULL, order_time datetime NULL, ship_time datetime NULL, amount double NULL, quantity int NULL, discount float NULL, total decimal(10,2) NULL, order_status text NULL, payment_method text NULL, payment_time datetime NULL, shipping_address text NULL, shipping_zip text NULL, tracking_number text NULL, is_gift boolean NULL, gift_note text NULL, coupon_code text NULL, currency varchar(5) NULL, exchange_rate double NULL, device_id text NULL, app_channel text NULL, order_type tinyint NULL, invoice_required boolean NULL, invoice_title text NULL, loyalty_points int NULL, feedback text NULL, delivery_type text NULL, delivery_eta date NULL, metadata json NULL ) ENGINE=OLAP DUPLICATE KEY(order_id, user_id, product_id) PARTITION BY RANGE(order_time) (PARTITION p202405 VALUES [('0000-01-01 00:00:00'), ('2024-06-01 00:00:00')), PARTITION p202406 VALUES [('2024-06-01 00:00:00'), ('2024-07-01 00:00:00')), PARTITION pMAX VALUES [('2024-07-01 00:00:00'), ('9999-12-31 00:00:00'))) DISTRIBUTED BY HASH(order_id) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "min_load_replica_num" = "-1", "is_being_synced" = "false", "storage_medium" = "hdd", "storage_format" = "V2", "inverted_index_storage_format" = "V1", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false", "group_commit_interval_ms" = "10000", "group_commit_data_bytes" = "134217728" );

sparksql (spark3.3.2, spark-doris-connector-25.1.0)
CREATE OR REPLACE TEMPORARY VIEW orders8 USING doris OPTIONS('table.identifier'='test1.orders','fenodes'='xx','user'='root','password'='xx','doris.batch.size'='1024');

CREATE OR REPLACE TEMPORARY VIEW orders8_new USING doris OPTIONS('table.identifier'='test1.orders_new','fenodes'='xx','user'='root','password'='xx','doris.batch.size'='1024');

insert into orders8_new select * from orders8 limit 1; 这条sql报错
insert into orders8_new select * from orders8; 这条不报错

doris218 be的错误日志如下:
image.png
image.png

2 Answers

复现数据文件如下
order_id user_id product_id order_time ship_time amount quantity discount total order_status payment_method payment_time shipping_address shipping_zip tracking_number is_gift gift_note coupon_code currency exchange_rate device_id app_channel order_type invoice_required invoice_title loyalty_points feedback delivery_type delivery_eta metadata
4467662 43 8240309 2025-04-20 16:44:42 2025-04-21 16:44:42 1704.21 5 0.05 1999.89 delivered paypal 2025-04-22 01:44:42 621 King Lakes North Eric AL 86541 20121 42b2ea9d-05ac-49a9-8ff3-02492451ba67 0 Rule will herself wait investment. crime LAK 1.08 b93204b7-53cb-4089-a0a0-1b773bd577c9 website 2 1 Haley Inc 611 Site end teach serious crime simply. Middle newspaper get pass front your. standard 2025-04-22 NULL
0 3937093 584041 2023-05-20 15:18:04 2023-05-24 15:18:04 775.85 9 0.16 315.26 shipped credit_card 2023-05-21 23:18:04 0034 Morgan Summit Port Rickey KY 42798 81317 a8c5d923-3032-488c-9bcf-eaf6d2f1b9c2 1 Find we wide film name black threat. activity MAD 0.92 7306760a-5562-4fad-a884-c93aa5c76d74 appstore 1 0 Jenkins-Fry 586 Land quite threat like often account. Other early lawyer serious. Money happy education stay. express 2023-05-29 NULL
4999999 2283815 407452 2024-10-18 20:38:46 2024-10-22 20:38:46 1001.35 10 0.19 205.51 pending paypal 2024-10-19 09:38:46 19321 Jones Roads Crystalfurt OR 54401 73986 17005bac-fac3-47be-9aa4-cf2a8c61e9f6 0 Too region run alone sell growth remember. second COP 1.19 d0bf96f7-6ce9-46e4-a9c1-d8a2d6ef133d appstore 3 0 Hill Group 577 Style board federal decision from thing approach scene. Watch spend first American hit forward. standard 2024-10-25 NULL
4999999 2283815 407452 2024-10-18 20:38:46 2024-10-22 20:38:46 1001.35 10 0.19 205.51 pending paypal 2024-10-19 09:38:46 19321 Jones Roads Crystalfurt OR 54401 73986 17005bac-fac3-47be-9aa4-cf2a8c61e9f6 0 Too region run alone sell growth remember. second COP 1.19 d0bf96f7-6ce9-46e4-a9c1-d8a2d6ef133d appstore 3 0 Hill Group 577 Style board federal decision from thing approach scene. Watch spend first American hit forward. standard 2024-10-25 NULL

doris2.1.8一定报错,但是doris换成2.1.11版本,会有错误提示,但是能跑过,怀疑是doris的问题