tpch对比测试doris内表、hive表和paimon表，paimon性能最差

Viewed 26
catalog

使用2.1.8.1版本基于tpch 100G测试集对比doris内表、hive表和paimon表性能。跑query2查询时，观察profile发现paimon表plan time明显高于其他两种表。相关信息如下：
paimon表：

Summary:
   - Profile ID: 8720fc3114a44a61-96aeb3243e8a91e3
   - Task Type: QUERY
   - Start Time: 2025-04-21 11:14:33
   - End Time: 2025-04-21 11:14:34
   - Total: 1s354ms
   - Task State: EOF
   - User: admin
   - Default Catalog: hdfs_paimon_test
   - Default Db: tpch100_paimon
   - Sql Statement: select
s_acctbal,
s_name,
n_name,
p_partkey,
p_mfgr,
s_address,
s_phone,
s_comment
from
part,
supplier,
partsupp,
nation,
region
where
p_partkey = ps_partkey
and s_suppkey = ps_suppkey
and p_size = 15
and p_type like '%BRASS'
and s_nationkey = n_nationkey
and n_regionkey = r_regionkey
and r_name = "EUROPE"
and ps_supplycost = (
select
min(ps_supplycost)
from
partsupp,
supplier,
nation,
region
where
p_partkey = ps_partkey
and s_suppkey = ps_suppkey
and s_nationkey = n_nationkey
and n_regionkey = r_regionkey
and r_name = "EUROPE"
)
order by
s_acctbal desc,
n_name,
s_name,
p_partkey
limit 100 LIMIT 1001;
Execution Summary:
   - Parse SQL Time: 0ms
   - Nereids Lock Table Time: 1ms
   - Nereids Analysis Time: 2ms
   - Nereids Rewrite Time: 4ms
   - Nereids Optimize Time: 1ms
   - Nereids Translate Time: 1ms
   - Workload Group: normal
   - Analysis Time: 3ms
   - Plan Time: 772ms
     - JoinReorder Time: N/A
     - CreateSingleNode Time: N/A
     - QueryDistributed Time: N/A
     - Init Scan Node Time: 26ms
     - Finalize Scan Node Time: 120ms
       - Get Splits Time: 117ms
         - Get Partitions Time: N/A
         - Get Partition Files Time: N/A
       - Create Scan Range Time: 3ms
   - Schedule Time: 29ms
     - Fragment Assign Time: 1ms
     - Fragment Serialize Time: 4ms
     - Fragment RPC Phase1 Time: 23ms
     - Fragment RPC Phase2 Time: 1ms
     - Fragment Compressed Size: 949.51 KB
     - Fragment RPC Count: 2
   - Schedule Time Of BE: {"phase1":{"158.2.100.39: 8060":{"RPC Work Time":"20ms","RPC Latency From FE To BE":"3ms","RPC Work Queue Time":"0ms","RPC Latency From BE To FE":"0ms"}},"phase2":{"158.2.100.39: 8060":{"RPC Work Time":"0ms","RPC Latency From FE To BE":"1ms","RPC Work Queue Time":"0ms","RPC Latency From BE To FE":"0ms"}}}
   - Wait and Fetch Result Time: 550ms
     - Fetch Result Time: 548ms
     - Write Result Time: 0ms
   - Doris Version: doris-2.1.8-1-834d802457
   - Is Nereids: Ye

hive表：

Summary:
   - Profile ID: f8764ce7b277424d-b2e76d9338bf5dce
   - Task Type: QUERY
   - Start Time: 2025-04-21 11:08:20
   - End Time: 2025-04-21 11:08:21
   - Total: 1s25ms
   - Task State: EOF
   - User: admin
   - Default Catalog: hive_test
   - Default Db: tpch100_hive
   - Sql Statement: select
s_acctbal,
s_name,
n_name,
p_partkey,
p_mfgr,
s_address,
s_phone,
s_comment
from
part,
supplier,
partsupp,
nation,
region
where
p_partkey = ps_partkey
and s_suppkey = ps_suppkey
and p_size = 15
and p_type like '%BRASS'
and s_nationkey = n_nationkey
and n_regionkey = r_regionkey
and r_name = "EUROPE"
and ps_supplycost = (
select
min(ps_supplycost)
from
partsupp,
supplier,
nation,
region
where
p_partkey = ps_partkey
and s_suppkey = ps_suppkey
and s_nationkey = n_nationkey
and n_regionkey = r_regionkey
and r_name = "EUROPE"
)
order by
s_acctbal desc,
n_name,
s_name,
p_partkey
limit 100 LIMIT 1001;
Execution Summary:
   - Parse SQL Time: 0ms
   - Nereids Lock Table Time: 2ms
   - Nereids Analysis Time: 1ms
   - Nereids Rewrite Time: 4ms
   - Nereids Optimize Time: 2ms
   - Nereids Translate Time: 0ms
   - Workload Group: normal
   - Analysis Time: 3ms
   - Plan Time: 9ms
     - JoinReorder Time: N/A
     - CreateSingleNode Time: N/A
     - QueryDistributed Time: N/A
     - Init Scan Node Time: 0ms
     - Finalize Scan Node Time: 1ms
       - Get Splits Time: 0ms
         - Get Partitions Time: 0ms
         - Get Partition Files Time: 0ms
       - Create Scan Range Time: 1ms
   - Schedule Time: 22ms
     - Fragment Assign Time: 1ms
     - Fragment Serialize Time: 1ms
     - Fragment RPC Phase1 Time: 19ms
     - Fragment RPC Phase2 Time: 1ms
     - Fragment Compressed Size: 50.43 KB
     - Fragment RPC Count: 2
   - Schedule Time Of BE: {"phase1":{"158.2.100.39: 8060":{"RPC Work Time":"18ms","RPC Latency From FE To BE":"0ms","RPC Work Queue Time":"0ms","RPC Latency From BE To FE":"1ms"}},"phase2":{"158.2.100.39: 8060":{"RPC Work Time":"1ms","RPC Latency From FE To BE":"0ms","RPC Work Queue Time":"0ms","RPC Latency From BE To FE":"0ms"}}}
   - Wait and Fetch Result Time: 991ms
     - Fetch Result Time: 991ms
     - Write Result Time: 0ms
   - Doris Version: doris-2.1.8-1-834d802457
   - Is Nereids: Yes
   - Is Pipel
Wen Zhenghu1
1 Answers