升级doris2.1.8版本后使用24.0.0的spark-connector出现的问题？

Question

spark3.1版本，用的是spark的spark-doris-connector-spark-3.1的24.0.0版本，读取doris时报如下错误，这是什么原因呢？

github上看到报错的源码位置

gnehil · Answer

麻烦发一下 connector 的配置参数以及读取的表结构吧，我尝试复现一下看看

慎始敬终 · Answer

就这些参数

表结构
CREATE TABLE ads_ip_host (
host varchar(1024) NULL COMMENT "host",
server_ip varchar(150) NULL COMMENT "服务器ip",
start_time datetime MIN NULL DEFAULT "1970-01-01 00:00:00" COMMENT "开始时间",
end_time datetime MAX NULL DEFAULT "1970-01-01 00:00:00" COMMENT "结束时间",
cnt bigint SUM NULL DEFAULT "0" COMMENT "关联次数",
ip_geo_tag varchar(20) REPLACE NULL DEFAULT "" COMMENT "Ip归属地国家编码",
ip_geo_code_tag varchar(20) REPLACE NULL DEFAULT "" COMMENT "Ip归属地",
large_small_flag varchar(20) REPLACE NULL DEFAULT "0" COMMENT "大小流量标识",
ipv4_ipv6_flag varchar(20) REPLACE NULL DEFAULT "0" COMMENT "Ipv4_ipv6标识",
http_cnt bigint SUM NULL DEFAULT "0" COMMENT "http协议次数",
ssl_cnt bigint SUM NULL DEFAULT "0" COMMENT "ssl协议次数",
dns_cnt bigint SUM NULL DEFAULT "0" COMMENT "dns协议次数",
flow_cnt bigint SUM NULL DEFAULT "0" COMMENT "flow协议次数",
INDEX ip_idx (server_ip) USING INVERTED COMMENT 'ip索引'
) ENGINE=OLAP
AGGREGATE KEY(host, server_ip)
DISTRIBUTED BY HASH(host) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"min_load_replica_num" = "-1",
"bloom_filter_columns" = "server_ip",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
);

慎始敬终 · Answer

今天在我们一个spark3.4.4的集群上，把doris连接器的依赖也升级为了spark-doris-connector-spark-3.4 24.0.0版本，完整报错如下
25/02/12 17:45:25 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local21:38779 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:25 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local21:34042 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:25 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local55:39824 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 WARN TaskSetManager: Lost task 8.0 in stage 0.0 (TID 8) (local51 executor 11): java.lang.ArrayIndexOutOfBoundsException: 0
at org.apache.doris.spark.read.DorisPartitionReader.$anonfun$get$2(DorisPartitionReader.scala:57)
at org.apache.doris.spark.read.DorisPartitionReader.$anonfun$get$2$adapted(DorisPartitionReader.scala:54)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at org.apache.doris.spark.read.DorisPartitionReader.get(DorisPartitionReader.scala:54)
at org.apache.doris.spark.read.DorisPartitionReader.get(DorisPartitionReader.scala:31)
at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.next(DataSourceRDD.scala:135)
at org.apache.spark.sql.execution.datasources.v2.MetricsRowIterator.next(DataSourceRDD.scala:172)
at org.apache.spark.sql.execution.datasources.v2.MetricsRowIterator.next(DataSourceRDD.scala:169)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.next(DataSourceRDD.scala:67)
at org.apache.spark.InterruptibleIterator.next(InterruptibleIterator.scala:40)
at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

25/02/12 17:45:28 INFO TaskSetManager: Starting task 8.1 in stage 0.0 (TID 10) (local55, executor 20, partition 8, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 6.0 in stage 0.0 (TID 6) on local51, executor 1: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 1]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 6.1 in stage 0.0 (TID 11) (local55, executor 20, partition 6, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1) on local21, executor 5: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 2]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 1.1 in stage 0.0 (TID 12) (local55, executor 17, partition 1, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 4.0 in stage 0.0 (TID 4) on local21, executor 9: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 3]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 4.1 in stage 0.0 (TID 13) (local55, executor 3, partition 4, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 5.0 in stage 0.0 (TID 5) on local55, executor 10: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 4]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 5.1 in stage 0.0 (TID 14) (local21, executor 16, partition 5, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 WARN TaskSetManager: Lost task 8.1 in stage 0.0 (TID 10) (local55 executor 20): java.lang.RuntimeException: backend get next failed
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:170)
at org.apache.doris.spark.client.read.AbstractThriftReader.lockClient(AbstractThriftReader.java:231)
at org.apache.doris.spark.client.read.AbstractThriftReader.hasNext(AbstractThriftReader.java:166)
at org.apache.doris.spark.read.DorisPartitionReader.next(DorisPartitionReader.scala:48)
at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:120)
at org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:158)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:63)
at scala.Option.exists(Option.scala:376)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.advanceToNextIter(DataSourceRDD.scala:97)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.doris.spark.exception.DorisInternalException: Doris server Backend{host='192.168.1.174', httpPort=-1, rpcPort=9060} internal failed, status code [ABORTED] error message is [(192.168.1.174)[ABORTED]fragment_executor execute failed]
at org.apache.doris.spark.client.DorisBackendThriftClient.getNext(DorisBackendThriftClient.java:195)
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:168)
... 30 more

25/02/12 17:45:28 INFO TaskSetManager: Starting task 8.2 in stage 0.0 (TID 15) (local51, executor 4, partition 8, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 2.0 in stage 0.0 (TID 2) on local21, executor 8: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 5]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 2.1 in stage 0.0 (TID 16) (local21, executor 5, partition 2, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0) on local55, executor 21: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 6]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 0.1 in stage 0.0 (TID 17) (local55, executor 10, partition 0, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 2.1 in stage 0.0 (TID 16) on local21, executor 5: java.lang.RuntimeException (backend get next failed) [duplicate 1]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 2.2 in stage 0.0 (TID 18) (local51, executor 15, partition 2, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 9.0 in stage 0.0 (TID 9) on local55, executor 14: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 7]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 9.1 in stage 0.0 (TID 19) (local21, executor 16, partition 9, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 0.1 in stage 0.0 (TID 17) on local55, executor 10: java.lang.RuntimeException (backend get next failed) [duplicate 2]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 0.2 in stage 0.0 (TID 20) (local21, executor 16, partition 0, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local55:45772 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local55:36887 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local21:35913 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local51:39776 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 3.0 in stage 0.0 (TID 3) on local55, executor 20: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 8]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 3.1 in stage 0.0 (TID 21) (local51, executor 18, partition 3, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 6.1 in stage 0.0 (TID 11) on local55, executor 20: java.lang.RuntimeException (backend get next failed) [duplicate 3]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 6.2 in stage 0.0 (TID 22) (local55, executor 10, partition 6, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:28 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local51:46845 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:28 INFO TaskSetManager: Lost task 7.0 in stage 0.0 (TID 7) on local21, executor 19: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 9]
25/02/12 17:45:28 INFO TaskSetManager: Starting task 7.1 in stage 0.0 (TID 23) (local51, executor 1, partition 7, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:29 INFO TaskSetManager: Lost task 7.1 in stage 0.0 (TID 23) on local51, executor 1: java.lang.RuntimeException (backend get next failed) [duplicate 4]
25/02/12 17:45:29 INFO TaskSetManager: Starting task 7.2 in stage 0.0 (TID 24) (local55, executor 17, partition 7, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:29 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on local51:44001 (size: 6.7 KiB, free: 5.2 GiB)
25/02/12 17:45:29 INFO TaskSetManager: Lost task 6.2 in stage 0.0 (TID 22) on local55, executor 10: java.lang.ArrayIndexOutOfBoundsException (0) [duplicate 10]
25/02/12 17:45:29 INFO TaskSetManager: Starting task 6.3 in stage 0.0 (TID 25) (local55, executor 20, partition 6, PROCESS_LOCAL, 20095 bytes)
25/02/12 17:45:29 INFO TaskSetManager: Lost task 6.3 in stage 0.0 (TID 25) on local55, executor 20: java.lang.RuntimeException (backend get next failed) [duplicate 5]
25/02/12 17:45:29 ERROR TaskSetManager: Task 6 in stage 0.0 failed 4 times; aborting job
25/02/12 17:45:29 INFO YarnScheduler: Cancelling stage 0
25/02/12 17:45:29 INFO YarnScheduler: Killing all running tasks in stage 0: Stage cancelled
25/02/12 17:45:29 INFO YarnScheduler: Stage 0 was cancelled
25/02/12 17:45:29 INFO DAGScheduler: ShuffleMapStage 0 (count at IpIntService.scala:87) failed in 4.563 s due to Job aborted due to stage failure: Task 6 in stage 0.0 failed 4 times, most recent failure: Lost task 6.3 in stage 0.0 (TID 25) (local55 executor 20): java.lang.RuntimeException: backend get next failed
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:170)
at org.apache.doris.spark.client.read.AbstractThriftReader.lockClient(AbstractThriftReader.java:231)
at org.apache.doris.spark.client.read.AbstractThriftReader.hasNext(AbstractThriftReader.java:166)
at org.apache.doris.spark.read.DorisPartitionReader.next(DorisPartitionReader.scala:48)
at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:120)
at org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:158)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:63)
at scala.Option.exists(Option.scala:376)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.advanceToNextIter(DataSourceRDD.scala:97)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.doris.spark.exception.DorisInternalException: Doris server Backend{host='192.168.1.101', httpPort=-1, rpcPort=9060} internal failed, status code [ABORTED] error message is [(192.168.1.101)[ABORTED]fragment_executor execute failed]
at org.apache.doris.spark.client.DorisBackendThriftClient.getNext(DorisBackendThriftClient.java:195)
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:168)
... 30 more

Driver stacktrace:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 6 in stage 0.0 failed 4 times, most recent failure: Lost task 6.3 in stage 0.0 (TID 25) (local55 executor 20): java.lang.RuntimeException: backend get next failed
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:170)
at org.apache.doris.spark.client.read.AbstractThriftReader.lockClient(AbstractThriftReader.java:231)
at org.apache.doris.spark.client.read.AbstractThriftReader.hasNext(AbstractThriftReader.java:166)
at org.apache.doris.spark.read.DorisPartitionReader.next(DorisPartitionReader.scala:48)
at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:120)
at org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:158)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:63)
at scala.Option.exists(Option.scala:376)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.advanceToNextIter(DataSourceRDD.scala:97)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.doris.spark.exception.DorisInternalException: Doris server Backend{host='192.168.1.101', httpPort=-1, rpcPort=9060} internal failed, status code [ABORTED] error message is [(192.168.1.101)[ABORTED]fragment_executor execute failed]
at org.apache.doris.spark.client.DorisBackendThriftClient.getNext(DorisBackendThriftClient.java:195)
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:168)
... 30 more

Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2791)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2727)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2726)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2726)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1211)
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1211)
at scala.Option.foreach(Option.scala:407)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1211)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2990)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2929)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2918)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.RuntimeException: backend get next failed
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:170)
at org.apache.doris.spark.client.read.AbstractThriftReader.lockClient(AbstractThriftReader.java:231)
at org.apache.doris.spark.client.read.AbstractThriftReader.hasNext(AbstractThriftReader.java:166)
at org.apache.doris.spark.read.DorisPartitionReader.next(DorisPartitionReader.scala:48)
at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:120)
at org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:158)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:63)
at scala.Option.exists(Option.scala:376)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.advanceToNextIter(DataSourceRDD.scala:97)
at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:101)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
at org.apache.spark.scheduler.Task.run(Task.scala:139)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:554)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1529)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:557)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.doris.spark.exception.DorisInternalException: Doris server Backend{host='192.168.1.101', httpPort=-1, rpcPort=9060} internal failed, status code [ABORTED] error message is [(192.168.1.101)[ABORTED]fragment_executor execute failed]
at org.apache.doris.spark.client.DorisBackendThriftClient.getNext(DorisBackendThriftClient.java:195)
at org.apache.doris.spark.client.read.AbstractThriftReader.lambda$hasNext$3(AbstractThriftReader.java:168)
... 30 more
25/02/12 17:45:29 INFO SparkContext: Invoking stop() from shutdown hook
25/02/12 17:45:29 INFO SparkContext: SparkContext is stopping with exitCode 0.
25/02/12 17:45:29 INFO SparkUI: Stopped Spark web UI at http://local21:4063
25/02/12 17:45:29 INFO YarnClientSchedulerBackend: Interrupting monitor thread
25/02/12 17:45:29 INFO YarnClientSchedulerBackend: Shutting down all executors
25/02/12 17:45:29 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
25/02/12 17:45:29 INFO YarnClientSchedulerBackend: YARN client scheduler backend Stopped
25/02/12 17:45:29 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
25/02/12 17:45:29 INFO MemoryStore: MemoryStore cleared
25/02/12 17:45:29 INFO BlockManager: BlockManager stopped

慎始敬终 · Answer

你好，我是用dorisManager升级集群的，从2.1.6升级为了2.1.8. 从dorisManager上看到doris集群状态正常。这是这一段详细报错，感谢大佬指导！
W20250213 11:00:12.644881 181238 status.h:415] meet error status: [ABORTED]

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::FragmentMgr::cancel_instance(doris::TUniqueId const&, doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
2#  doris::ExternalScanContextMgr::clear_scan_context(std::__cxx11::basic_string, std::allocator > const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
3#  doris::BackendService::close_scanner(doris::TScanCloseResult&, doris::TScanCloseParams const&) at /home/zcp/repo_center/doris_release/doris/be/src/service/backend_service.cpp:0
4#  doris::BackendServiceProcessor::process_close_scanner(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) at /home/zcp/repo_center/doris_release/doris/gensrc/build/gen_cpp/BackendService.cpp:6308
5#  doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string, std::allocator > const&, int, void*) at /home/zcp/repo_center/doris_release/doris/gensrc/build/gen_cpp/BackendService.cpp:0
6#  apache::thrift::TDispatchProcessor::process(std::shared_ptr, std::shared_ptr, void*) at /home/zcp/repo_center/doris_release/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:0
7#  apache::thrift::server::TConnectedClient::run()
8#  apache::thrift::server::TThreadedServer::TConnectedClientRunner::run()
9#  apache::thrift::concurrency::Thread::threadMain(std::shared_ptr)
10# void std::__invoke_impl), std::shared_ptr >(std::__invoke_other, void (*&&)(std::shared_ptr), std::shared_ptr&&)
11# std::__invoke_result), std::shared_ptr >::type std::__invoke), std::shared_ptr >(void (*&&)(std::shared_ptr), std::shared_ptr&&)
12# void std::thread::_Invoker), std::shared_ptr > >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>)
13# std::thread::_Invoker), std::shared_ptr > >::operator()()
14# std::thread::_State_impl), std::shared_ptr > > >::_M_run()
15# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85
16# start_thread
17# __clone

W20250213 11:00:12.645220 106158 doris_main.cpp:123] thrift internal message: TSocket::open() getaddrinfo() 未知的名称或服务
W20250213 11:00:12.645430 106158 status.h:415] meet error status: [THRIFT_RPC_ERROR]Couldn't open transport for :0 (Could not resolve host for client socket.)

0#  doris::ThriftClientImpl::open() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1#  doris::ThriftClientImpl::open_with_retry(int, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
2#  doris::ClientCacheHelper::_create_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3#  doris::ClientCacheHelper::get_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4#  doris::ClientConnection::ClientConnection(doris::ClientCache*, doris::TNetworkAddress const&, int, doris::Status*, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5#  doris::FragmentMgr::coordinator_callback(doris::ReportStatusRequest const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
6#  doris::PlanFragmentExecutor::send_report(bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:0
7#  doris::PlanFragmentExecutor::open() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:304
8#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:180
9#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
10# std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone

W20250213 11:00:12.738864 106154 runtime_state.h:214] Task is cancelled, instance: 9c8d974b9b5441c0-8c0cbcad62d4f633|2041b66c345e929f-592ecaee299990af, st = [CANCELLED]fragment_executor execute failed
W20250213 11:00:12.739215 106154 status.h:415] meet error status: [ABORTED]fragment_executor execute failed

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
2#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
3#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
4#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
5#  start_thread
6#  __clone

W20250213 11:00:12.739332 106154 status.h:415] meet error status: [INTERNAL_ERROR]open failed

0#  doris::PlanFragmentExecutor::close() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:615
1#  doris::PlanFragmentExecutor::~PlanFragmentExecutor() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/move.h:206
2#  std::_Function_handler const&)::$_0>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/atomicity.h:98
3#  doris::FunctionRunnable::~FunctionRunnable() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:44
4#  doris::ThreadPool::dispatch_thread() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/atomicity.h:98
5#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
6#  start_thread
7#  __clone

W20250213 11:00:12.752460 181260 backend_service.cpp:868] fragment_instance_id [2041b66c345e929f-592ecaee299990af] fetch result status [[ABORTED]fragment_executor execute failed

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
2#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
3#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
4#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
5#  start_thread
6#  __clone

]
W20250213 11:00:12.753784 181260 backend_service.cpp:868] fragment_instance_id [2041b66c345e929f-592ecaee299990af] fetch result status [[ABORTED]fragment_executor execute failed

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
2#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
3#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
4#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
5#  start_thread
6#  __clone

]
W20250213 11:00:12.754197 181260 backend_service.cpp:868] fragment_instance_id [2041b66c345e929f-592ecaee299990af] fetch result status [[ABORTED]fragment_executor execute failed

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
2#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
3#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
4#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
5#  start_thread
6#  __clone

]
W20250213 11:00:12.769817 181260 fragment_mgr.cpp:1293] Could not find the fragment instance id:2041b66c345e929f-592ecaee299990af to cancel
W20250213 11:00:13.745874 106158 doris_main.cpp:123] thrift internal message: TSocket::open() getaddrinfo() 未知的名称或服务
W20250213 11:00:13.745954 106158 status.h:415] meet error status: [THRIFT_RPC_ERROR]Couldn't open transport for :0 (Could not resolve host for client socket.)

0#  doris::ThriftClientImpl::open() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1#  doris::ThriftClientImpl::open_with_retry(int, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
2#  doris::ClientCacheHelper::_create_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3#  doris::ClientCacheHelper::get_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4#  doris::ClientConnection::ClientConnection(doris::ClientCache*, doris::TNetworkAddress const&, int, doris::Status*, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5#  doris::FragmentMgr::coordinator_callback(doris::ReportStatusRequest const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
6#  doris::PlanFragmentExecutor::send_report(bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:0
7#  doris::PlanFragmentExecutor::open() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:304
8#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:180
9#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
10# std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone

W20250213 11:00:15.846427 106158 doris_main.cpp:123] thrift internal message: TSocket::open() getaddrinfo() 未知的名称或服务
W20250213 11:00:15.846508 106158 status.h:415] meet error status: [THRIFT_RPC_ERROR]Couldn't open transport for :0 (Could not resolve host for client socket.)

0#  doris::ThriftClientImpl::open() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1#  doris::ThriftClientImpl::open_with_retry(int, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
2#  doris::ClientCacheHelper::_create_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3#  doris::ClientCacheHelper::get_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4#  doris::ClientConnection::ClientConnection(doris::ClientCache*, doris::TNetworkAddress const&, int, doris::Status*, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5#  doris::FragmentMgr::coordinator_callback(doris::ReportStatusRequest const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
6#  doris::PlanFragmentExecutor::send_report(bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:0
7#  doris::PlanFragmentExecutor::open() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:304
8#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:180
9#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
10# std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone

W20250213 11:00:16.408576 108085 stream_load_executor.cpp:204] begin transaction failed, errmsg=[NOT_FOUND]TStatus: errCode = 7, detailMessage = unknown database, database=xdn_testid=c949aa4ac45bfe60-a4218dd1088339be, job_id=-1, txn_id=-1, label=ads_msisdn_tacfbda5675-451b-403a-918b-9a1c1e256e931739415616398, elapse(s)=0
W20250213 11:00:19.946866 106158 status.h:415] meet error status: [INTERNAL_ERROR]query_id: 9c8d974b9b5441c0-8c0cbcad62d4f633, couldn't get a client for TNetworkAddress(hostname=, port=0), reason is [THRIFT_RPC_ERROR]Couldn't open transport for :0 (Could not resolve host for client socket.)

0#  doris::ThriftClientImpl::open() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1#  doris::ThriftClientImpl::open_with_retry(int, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
2#  doris::ClientCacheHelper::_create_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3#  doris::ClientCacheHelper::get_client(doris::TNetworkAddress const&, std::function&, void**, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4#  doris::ClientConnection::ClientConnection(doris::ClientCache*, doris::TNetworkAddress const&, int, doris::Status*, int) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:380
5#  doris::FragmentMgr::coordinator_callback(doris::ReportStatusRequest const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
6#  doris::PlanFragmentExecutor::send_report(bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:0
7#  doris::PlanFragmentExecutor::open() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:304
8#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:180
9#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
10# std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
11# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
12# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
13# start_thread
14# __clone


0#  doris::FragmentMgr::coordinator_callback(doris::ReportStatusRequest const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::PlanFragmentExecutor::send_report(bool) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:0
2#  doris::PlanFragmentExecutor::open() at /home/zcp/repo_center/doris_release/doris/be/src/runtime/plan_fragment_executor.cpp:304
3#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/tuple:180
4#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
5#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
6#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
7#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
8#  start_thread
9#  __clone

W20250213 11:00:19.946938 106158 plan_fragment_executor.cpp:406] Got error while opening fragment a4468b0a96b83026-c419d410de2d1695, query id: 9c8d974b9b5441c0-8c0cbcad62d4f633: [CANCELLED]cancelled
W20250213 11:00:19.946990 106158 runtime_state.h:218] Task is already cancelled, instance: 9c8d974b9b5441c0-8c0cbcad62d4f633|a4468b0a96b83026-c419d410de2d1695, original cancel msg: , new cancel msg: [CANCELLED]PlanFragmentExecutor open failed, reason: [CANCELLED]cancelled
W20250213 11:00:19.947027 106158 status.h:415] meet error status: [ABORTED]PlanFragmentExecutor open failed, reason: [CANCELLED]cancelled

0#  doris::PlanFragmentExecutor::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:0
1#  doris::PlanFragmentExecutor::execute() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
2#  doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function const&) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3#  std::_Function_handler const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701
4#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_release/doris/be/src/util/threadpool.cpp:0
5#  doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
6#  start_thread
7#  __clone

JNSimba · Answer

可以升级到最新的25.0.1试试

升级doris2.1.8版本后使用24.0.0的spark-connector出现的问题？

5 Answers