CREATE ROUTINE LOAD遇到的问题

Viewed 10

CREATE ROUTINE LOAD

FROM KAFKA
(
    "kafka_broker_list" = "10.15.8.72:9092",
    "kafka_topic" = "test",
    -- "property.group.id" = "console-consumer-38930",
    "property.kafka_default_offsets" = "OFFSET_BEGINNING"
    -- 分区和offsets
    -- "kafka_partitions"="0",
    -- "kafka_offsets" = "OFFSET_BEGINNING",
    -- 安全认证
    -- "property.security.protocol" = "ssl",
    -- "property.ssl.ca.location" = "FILE:ca.pem",
    -- "property.ssl.certificate.location" = "FILE:client.pem",
    -- "property.ssl.key.location" = "FILE:client.key",
    -- "property.ssl.key.password" = "ssl_passwd"
);

按照上面的配置进行创建,一直RUNNING,但数据导入不成功,执行SHOW ROUTINE LOAD,Progress和Lag都是0:0,问题是

failed to get latest partition offset. {}errCode = 2, detailMessage = Failed to get latest offsets of kafka topic: test. error: errCode = 2, detailMessage = Failed to get info

后面修改 ROUTINE LOAD,指定消费组,数据导入成功,但出现新的问题:

2025-10-27 18:15:04:[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = txn 3016007 could not be COMMITTED while task 87a572f2041e4146-bac551221f63d0e6 has been aborted.

于是就暂停、再恢复,又出现问题:

failed to get latest partition offset. {}errCode = 2, detailMessage = Failed to get latest offsets of kafka topic: test. error: errCode = 2, detailMessage = Failed to get info

企业微信截图_17615609199350.png
现在卡在kafka导入doris这一步,有时直接创建就能成功,有时暂停后恢复才成功。
跪求大佬解决。(我玩虚拟机没遇到过这些问题,数据同步配置都很顺畅)

1 Answers

得看下BE的日志中的详细错误,拿着你的job_id 在be.INFO 中搜下