doris3读取mysql数据写入到hive报错UnsupportedFileSystemException: No FileSystem for scheme "s3"

Viewed 25

背景:

  1. doris(doris-3.1.2)中创建hive和mysql对应的外部catalog;
  2. hive使用的是oss对象存储作为底层数据存储
    CREATE CATALOG hive PROPERTIES (
    "uri" = "thrift://metastore_host:9083",
    "type" = "hms",
    "oss.secret_key" = "*XXX",
    "oss.endpoint" = "*XXX",
    "oss.access_key" = "*XXX",
    "ipc.client.fallback-to-simple-auth-allowed" = "true",
    "hive.metastore.uris" = "thrift://host:9083",
    "hadoop.username" = "hive",
    "fs.defaultFS" = "oss://bucket-name",
    "dfs.nameservices" = "outer-nameservice",
    "dfs.namenode.rpc-address.outer-nameservice.m2" = "hostname1:8020",
    "dfs.namenode.rpc-address.outer-nameservice.m1" = "hostname2:8020",
    "dfs.ha.namenodes.outer-nameservice" = "m1,m2",
    "dfs.client.failover.proxy.provider.outer-nameservice" = "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"

);

执行sql:
insert into hive.ods.pc_transaction_table(id,name,...)
select id,name,... from mysql.db.transaction_table where id>100;

报错信息:
doris-jdbc报错信息:
ERROR 1105 (HY000) at line 1: errCode = 2, detailMessage = org.apache.hadoop.hive.metastore.api.MetaException: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "s3"
doris-fe报错信息:
image.png

2 Answers

3.1.3 应该修复了:https://github.com/apache/doris/pull/57973
可以升级试一下