Doris Streaming job同步Mysql库表,无法同步增量数据

Viewed 7
mysql> SELECT * FROM jobs("type"="insert") WHERE Name = "mysql_lpg_sync"\G
*************************** 1. row ***************************
                 Id: 1789028746438
               Name: mysql_lpg_sync
            Definer: root
        ExecuteType: STREAMING
  RecurringStrategy: 
             Status: RUNNING
         ExecuteSql: FROM MYSQL('include_tables'='***脱敏***','database'='db','skip_snapshot_backfill'='true','driver_class'='com.mysql.cj.jdbc.Driver','offset'='latest','driver_url'='https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.0.33/mysql-connector-j-8.0.33.jar','jdbc_url'='jdbc:mysql://172.16.1.162:3306/db?connectionTimeZone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true','user'='doris_sync' ) TO DATABASE ods_lpg ('table.create.properties.replication_num'='3')
         CreateTime: 2026-09-11 10:01:25
   SucceedTaskCount: 1
    FailedTaskCount: 0
  CanceledTaskCount: 0
            Comment: 
         Properties: 
      CurrentOffset: {"ts_sec":"0","file":"mysql-bin.000005","pos":"33419424","kind":"SPECIFIC","gtids":"7aea4b4c-b886-11ef-910f-00163e276791:1-23164480,d771e384-50ae-11ee-9723-00163e276791:1-11600041,ecb822e7-d2ab-11ec-897c-00163e1d61fc:1-5757492","splitId":"binlog-split","row":"0","event":"0","server_id":"1"}
          EndOffset: {"ts_sec":"0","file":"mysql-bin.000005","pos":"33433901","kind":"SPECIFIC","gtids":"7aea4b4c-b886-11ef-910f-00163e276791:1-23164480,\nd771e384-50ae-11ee-9723-00163e276791:1-11600041,\necb822e7-d2ab-11ec-897c-00163e1d61fc:1-5757492","row":"0","event":"0"}
      LoadStatistic: {"scannedRows":0,"loadBytes":0,"fileNumber":0,"fileSize":0,"filteredRows":0}
           ErrorMsg: 
      JobRuntimeMsg: No data available for consumption at the moment, will retry after 1789093281418
                Lag: 0
LastTaskSuccessTime: 2026-09-11 10:01:41
1 row in set (0.02 sec)

在进行Doris的POC验证,尝试使用Streaming job来实时同步Mysql库表到Doris。
源库: Mysql 8.0.46
Doris 为1FE+3BE集群,版本: 4.1.3

测试结果发现:

  1. 使用'offset'= initial:全量 + 增量同步 时,全量部分同步完成后,新增的增量部分并没有同步成功。
  2. 更换成 'offset'= latest时, 发现也没有同步到增量数据,CurrentOffset始终保持在创建任务的节点。

请问这是什么情况? 要怎么解决呢?

1 Answers

已解决,把mysql的gtid_mode打开就可以了