doris-grafana-dashboard监控面板异常,无法使用

Viewed 15
  • doris版本:4.1
  • 安装方式:k8s-operator

我参考官方文档的监控章节完成了集成,并导入了监控面板:doris-grafana-dashboard.json,与doris-grafana-dashboard-cloud.json 都试了

但是dashboard上所有下拉选项不是None,就是只有ALL选项,如图:

image.png

image.png

我直接查指标,发现里面是有数据的:

image.png

是官方的dashboard不对么?

2 Answers

dashboard 是对的,我本地刚测试,我觉得是你操作姿势可能有问题,比如你的data source源是对的不?确定这个 dashboard 绑定的Prometheus 源是对的?
image.png

我在本地重建了一套环境,发现还是不行,这一套环境就只有一个prometheus,不会选错的

那可能是上传的指标有问题,以下是我一个be的指标,帮忙看看正常不呢,帮忙跟你的be环境对比一下呢:

我执行: curl http://be1.doris.b.x/metrics  后返回:
# TYPE doris_be_spill_disk_capacity gauge
doris_be_spill_disk_capacity{path="/opt/apache-doris/be/storage/spill"} 3036857835520
# TYPE doris_be_spill_disk_limit gauge
doris_be_spill_disk_limit{path="/opt/apache-doris/be/storage/spill"} 546634410393
# TYPE doris_be_spill_disk_has_spill_data gauge
doris_be_spill_disk_has_spill_data{path="/opt/apache-doris/be/storage/spill"} 0
# TYPE doris_be_spill_disk_avail_capacity gauge
doris_be_spill_disk_avail_capacity{path="/opt/apache-doris/be/storage/spill"} 740702064640
# TYPE doris_be_spill_disk_data_size gauge
doris_be_spill_disk_data_size{path="/opt/apache-doris/be/storage/spill"} 0
# TYPE doris_be_spill_disk_has_spill_gc_data gauge
doris_be_spill_disk_has_spill_gc_data{path="/opt/apache-doris/be/storage/spill"} 0
# TYPE doris_be_workload_group_cpu_time_sec counter
doris_be_workload_group_cpu_time_sec{id="1",workload_group="normal"} 281057
# TYPE doris_be_workload_group_total_local_scan_bytes counter
doris_be_workload_group_total_local_scan_bytes{id="1",workload_group="normal"} 1083230277300
# TYPE doris_be_workload_group_mem_used_bytes gauge
doris_be_workload_group_mem_used_bytes{id="1",workload_group="normal"} 365544683
# TYPE doris_be_workload_group_remote_scan_bytes counter
doris_be_workload_group_remote_scan_bytes{id="1",workload_group="normal"} 18343854890
# TYPE doris_be_runtime_filter_consumer_wait_ready_ms counter
doris_be_runtime_filter_consumer_wait_ready_ms 17625394
# TYPE doris_be_runtime_filter_consumer_ready_num gauge
doris_be_runtime_filter_consumer_ready_num 53187
# TYPE doris_be_compaction_task_state_total counter
doris_be_compaction_task_state_total{type="base"} 0
doris_be_compaction_task_state_total{type="base"} 0
doris_be_compaction_task_state_total{type="cumulative"} 0
doris_be_compaction_task_state_total{type="cumulative"} 0
# TYPE doris_be_scanner_ctx_cnt counter
doris_be_scanner_ctx_cnt 1
# TYPE doris_be_runtime_filter_consumer_num gauge
doris_be_runtime_filter_consumer_num 57842
# TYPE doris_be_arrow_flight_work_pool_max_queue_size gauge
doris_be_arrow_flight_work_pool_max_queue_size 18446744073709551615
# TYPE doris_be_num_io_bytes_read_from_peer counter
doris_be_num_io_bytes_read_from_peer 0
# TYPE doris_be_hdfs_file_reader_total counter
doris_be_hdfs_file_reader_total 438628
# TYPE doris_be_s3_file_open_writing gauge
doris_be_s3_file_open_writing 0
# TYPE doris_be_s3_file_open_reading gauge
doris_be_s3_file_open_reading 0
# TYPE doris_be_s3_bytes_written_total counter
doris_be_s3_bytes_written_total 0
# TYPE doris_be_stream_load counter
doris_be_stream_load{type="load_rows"} 3069556
doris_be_stream_load{type="receive_bytes"} 7745741251
# TYPE doris_be_local_bytes_read_total counter
doris_be_local_bytes_read_total 2820356450598
# TYPE doris_be_segment_read counter
doris_be_segment_read{type="segment_read_total"} 11355868
doris_be_segment_read{type="segment_row_total"} 50989054534368
# TYPE doris_be_pipeline_task_queue_size counter
doris_be_pipeline_task_queue_size 0
# TYPE doris_be_ann_index_load_costs_ms counter
doris_be_ann_index_load_costs_ms 0
# TYPE doris_be_ann_index_search_costs_ms counter
doris_be_ann_index_search_costs_ms 0
# TYPE doris_be_cache_capacity gauge
doris_be_cache_capacity{name="AnnIndexIVFListCache"} 21646635184
doris_be_cache_capacity{name="CreateTabletRRIdxCache"} 10000
doris_be_cache_capacity{name="IndexPageCache"} 618475520
doris_be_cache_capacity{name="LoadStateChannelCache "} 1024
doris_be_cache_capacity{name="PointQueryRowCache"} 6184753024
doris_be_cache_capacity{name="TabletColumnObjectPool"} 102400
doris_be_cache_capacity{name="ConditionCache"} 536870912
doris_be_cache_capacity{name="QueryCache"} 536870912
doris_be_cache_capacity{name="PointQueryLookupConnectionCache"} 2048
doris_be_cache_capacity{name="InvertedIndexSearcherCache"} 3092376576
doris_be_cache_capacity{name="InvertedIndexQueryCache"} 3092376576
doris_be_cache_capacity{name="TabletSchemaCache"} 102400
doris_be_cache_capacity{name="SegmentCache"} 1546188288
doris_be_cache_capacity{name="MowTabletVersionCache"} 100000
doris_be_cache_capacity{name="MowDeleteBitmapAggCache"} 309237760
doris_be_cache_capacity{name="DataPageCache"} 5566277632
doris_be_cache_capacity{name="PKIndexPageCache"} 3092376576
doris_be_cache_capacity{name="CommonObjLRUCache"} 1024
# TYPE doris_be_ann_index_search_cnt counter
doris_be_ann_index_search_cnt 0
# TYPE doris_be_ann_ivf_on_disk_search_costs_ms counter
doris_be_ann_ivf_on_disk_search_costs_ms 0
# TYPE doris_be_ann_index_build_index_threads counter
doris_be_ann_index_build_index_threads 0
# TYPE doris_be_memory_jemalloc_allocated_bytes gauge
doris_be_memory_jemalloc_allocated_bytes 4181716112
# TYPE doris_be_memory_jemalloc_tcache_bytes gauge
doris_be_memory_jemalloc_tcache_bytes 449161832
# TYPE doris_be_memory_jemalloc_pactive_num gauge
doris_be_memory_jemalloc_pactive_num 1288678
# TYPE doris_be_light_work_active_threads gauge
doris_be_light_work_active_threads 0
# TYPE doris_be_thread_pool_max_threads gauge
doris_be_thread_pool_max_threads{id="ca466ac793f51dab-d16848274bfa8f81",workload_group="",thread_pool_name="AsyncFetchAutoIncIDExecutor"} 3
doris_be_thread_pool_max_threads{id="a44de8673d93c2f4-c86218196e073983",workload_group="",thread_pool_name="EvHttpServer"} 128
doris_be_thread_pool_max_threads{id="e74dbddb49dbe552-cec7d125c8fa3fb4",workload_group="",thread_pool_name="MigrationTaskThreadPool"} 1
doris_be_thread_pool_max_threads{id="c94a356e0777a0c7-21b509f9bd65b991",workload_group="",thread_pool_name="TaskWP_DELETE"} 3
doris_be_thread_pool_max_threads{id="e541481740247b42-d5804043e53b2eb0",workload_group="",thread_pool_name="TaskWP_CLEAR_TRANSACTION_TASK"} 1
doris_be_thread_pool_max_threads{id="724c0abf75cf90a9-3e3abeeab6d16d98",workload_group="",thread_pool_name="TaskWP_PUBLISH_VERSION"} 8
doris_be_thread_pool_max_threads{id="0c409b77075507b7-f92c764e773849b8",workload_group="",thread_pool_name="TaskWP_DROP_TABLE"} 3
doris_be_thread_pool_max_threads{id="df486dfa9966a110-05f9dab8d713b4b9",workload_group="",thread_pool_name="TaskWP_CREATE_TABLE"} 3
doris_be_thread_pool_max_threads{id="684e3b22745421a6-53750ba7baa2cbb8",workload_group="",thread_pool_name="TaskWP_SUBMIT_TABLE_COMPACTION"} 1
doris_be_thread_pool_max_threads{id="7149ee9859975eea-4af42c73f045668e",workload_group="",thread_pool_name="TaskWP_MOVE"} 1
doris_be_thread_pool_max_threads{id="9a451cd7fa961da0-18eb5a59fdb2f0b4",workload_group="",thread_pool_name="TaskWP_RELEASE_SNAPSHOT"} 5
doris_be_thread_pool_max_threads{id="b4449831f5c86e35-232a4f577ad1c2b1",workload_group="",thread_pool_name="TaskWP_MAKE_SNAPSHOT"} 5
doris_be_thread_pool_max_threads{id="8f429b7ec6727d5f-98d9283d12964abb",workload_group="",thread_pool_name="TaskWP_DOWNLOAD"} 1
doris_be_thread_pool_max_threads{id="2d4ea06989746114-dc6cad8fed5e708b",workload_group="",thread_pool_name="TaskWP_UPLOAD"} 1
doris_be_thread_pool_max_threads{id="e14a24282105b80d-7dc351b0847e628d",workload_group="",thread_pool_name="TaskWP_CHECK_CONSISTENCY"} 1
doris_be_thread_pool_max_threads{id="bf4ab940992f57ee-c1a7a6a77ac7548e",workload_group="",thread_pool_name="TaskWP_ALTER_INVERTED_INDEX"} 3
doris_be_thread_pool_max_threads{id="0e4273e2531ec0a7-25c937cfd8070a81",workload_group="",thread_pool_name="mf_normal"} 128
doris_be_thread_pool_max_threads{id="5343a5006eb4c059-d1a051493124a5a3",workload_group="",thread_pool_name="ColdDataCompactionTaskThreadPool"} 2
doris_be_thread_pool_max_threads{id="3a4baeca76f42c54-b49891e90010e6bc",workload_group="",thread_pool_name="SegCompactionTaskThreadPool"} 5
doris_be_thread_pool_max_threads{id="c34079f5fbe73cd3-e5d5c0b901a1b89e",workload_group="",thread_pool_name="CumuCompactionTaskThreadPool"} 5
doris_be_thread_pool_max_threads{id="ff43ee17b534ebcd-19e6c9f61c0274a1",workload_group="",thread_pool_name="BaseCompactionTaskThreadPool"} 4
doris_be_thread_pool_max_threads{id="b84b7308fcad257d-8f21992136b299ac",workload_group="",thread_pool_name="MemTableHighPriorityFlushThreadPool"} 128
doris_be_thread_pool_max_threads{id="c849b8445ad5dbc1-78fd5bcdc2f75798",workload_group="",thread_pool_name="MemTableFlushThreadPool"} 128
doris_be_thread_pool_max_threads{id="914761c28ea6c0e4-51884f5bd2ce309e",workload_group="",thread_pool_name="TaskWP_UPDATE_TABLET_META_INFO"} 1
doris_be_thread_pool_max_threads{id="5245378eef663bd4-35d27b98e414a591",workload_group="",thread_pool_name="TaskWP_CLEAN_TRASH"} 1
doris_be_thread_pool_max_threads{id="c94ae366cb2b5c71-9806439ffa9ff3bb",workload_group="",thread_pool_name="p_normal_simple_scheduler"} 32
doris_be_thread_pool_max_threads{id="c147d13bfaebebd5-4c97cc174010479c",workload_group="",thread_pool_name="DownloadThreadPool"} 64
doris_be_thread_pool_max_threads{id="61482d670aaedb3e-71d277e4dc89bbb9",workload_group="normal",thread_pool_name="rs_normal"} 512
doris_be_thread_pool_max_threads{id="c64e5533441a7acd-6dac81bdc607d09c",workload_group="",thread_pool_name="SegmentPrefetchThreadPool"} 2000
doris_be_thread_pool_max_threads{id="6348d593ebcf26e4-e6d97b81fec29194",workload_group="",thread_pool_name="NonBlockCloseThreadPool"} 64
doris_be_thread_pool_max_threads{id="324d594d3102e87b-6474a510cc810980",workload_group="",thread_pool_name="S3FileSystemThreadPool"} 64
doris_be_thread_pool_max_threads{id="af4ae0af82899e35-5ae0b68e45a8b7b4",workload_group="",thread_pool_name="S3FileUploadThreadPool"} 64
doris_be_thread_pool_max_threads{id="e9408edac6fbc85b-e894d205fce4a491",workload_group="",thread_pool_name="BufferedReaderPrefetchThreadPool"} 64
doris_be_thread_pool_max_threads{id="c84c0f5f75f5dbaa-60b8022ab5f36697",workload_group="",thread_pool_name="SendBatchThreadPool"} 64
doris_be_thread_pool_max_threads{id="514810825f75d9bc-5b98ecada2a82e8a",workload_group="",thread_pool_name="SegmentFileWriterThreadPool"} 6
doris_be_thread_pool_max_threads{id="7148711d669985d9-95d2456e85b036b9",workload_group="",thread_pool_name="LazyReleaseMemoryThreadPool"} 1
doris_be_thread_pool_max_threads{id="854b3ca3d5c3edcc-553e4074cd220196",workload_group="",thread_pool_name="routine_load"} 1024
doris_be_thread_pool_max_threads{id="694b373de3a58e55-1b6914e79ae2a7a8",workload_group="",thread_pool_name="UDFCloseWorkers"} 32
doris_be_thread_pool_max_threads{id="464fb817654fa0fd-fe8b81ace56008ba",workload_group="",thread_pool_name="GroupCommitThreadPool"} 10
doris_be_thread_pool_max_threads{id="df451a3774b69230-455cd12205925ab2",workload_group="",thread_pool_name="GroupCommitReplayWalThreadPool"} 10
doris_be_thread_pool_max_threads{id="8b49b9fb74d07988-2de021519038d9a2",workload_group="",thread_pool_name="FragmentMgrAsyncWorkThreadPool"} 2048
doris_be_thread_pool_max_threads{id="b541e11a1ffb0677-6f0c7937d9d46da1",workload_group="",thread_pool_name="TaskWP_UPDATE_VISIBLE_VERSION"} 1
doris_be_thread_pool_max_threads{id="fb49dd683ba09d51-5068a141c6e6468f",workload_group="",thread_pool_name="TaskWP_PUSH_INDEX_POLICY"} 1
doris_be_thread_pool_max_threads{id="1844717bd1b922b3-0bc3e21f0a38678d",workload_group="",thread_pool_name="TaskWP_STORAGE_MEDIUM_MIGRATE"} 1
doris_be_thread_pool_max_threads{id="7248a414c59bb8d1-7a59a88dd8a44993",workload_group="",thread_pool_name="SingleReplicaCompactionTaskThreadPool"} 1
doris_be_thread_pool_max_threads{id="06465cbebd6f4710-0225b156f5e698ad",workload_group="normal",thread_pool_name="ls_normal"} 64
doris_be_thread_pool_max_threads{id="b349ab8d5357a1a4-b0d11a555b348792",workload_group="",thread_pool_name="TaskWP_ALTER_TABLE"} 3
doris_be_thread_pool_max_threads{id="7c4600b288cddf72-bb4aa692c2c42f87",workload_group="",thread_pool_name="TaskWP_PUSH_STORAGE_POLICY"} 1
doris_be_thread_pool_max_threads{id="4145541e0df790ba-544d845bb33721b5",workload_group="",thread_pool_name="p_normal_blocking_scheduler"} 64
doris_be_thread_pool_max_threads{id="6646dfa5d6425a03-86ca7e7fa64eb4a6",workload_group="",thread_pool_name="TaskWP_CLEAN_UDF_CACHE"} 1
doris_be_thread_pool_max_threads{id="454b6c7a20ffbee8-1421edae6769b1be",workload_group="",thread_pool_name="TabletPublishTxnThreadPool"} 32
doris_be_thread_pool_max_threads{id="524498e6f574bd3a-f4f74af94289c8bc",workload_group="",thread_pool_name="TaskWP_GC_BINLOG"} 1
doris_be_thread_pool_max_threads{id="e44c7e94f40d0fff-8167ff996a3697b0",workload_group="",thread_pool_name="ReportProfileThreadPool"} 5
doris_be_thread_pool_max_threads{id="0a4de764390539bc-1bd209fdde4fcab4",workload_group="",thread_pool_name="TabletCalcDeleteBitmapThreadPool"} 16
doris_be_thread_pool_max_threads{id="4543c697b3e06e57-836059f814e0f689",workload_group="",thread_pool_name="TabletCalcDeleteBitmapThreadPool"} 32
doris_be_thread_pool_max_threads{id="e7497874a56e2077-809953fe87da388a",workload_group="",thread_pool_name="SendTableStatsThreadPool"} 32
doris_be_thread_pool_max_threads{id="c94405c4d0ba40ae-31a1c638c8c46f81",workload_group="",thread_pool_name="TabletMetaCheckpointTaskThreadPool"} 1
doris_be_thread_pool_max_threads{id="1441c2083caa45b9-f873ad1bcb1672a3",workload_group="",thread_pool_name="TaskWP_PUSH_COOLDOWN_CONF"} 1
# TYPE doris_be_fd_num_used gauge
doris_be_fd_num_used 46880
# TYPE doris_be_memory_pswpin gauge
doris_be_memory_pswpin 0
# TYPE doris_be_snmp_tcp_out_segs counter
doris_be_snmp_tcp_out_segs 5047239413
# TYPE doris_be_host_cpu_num counter
doris_be_host_cpu_num 32
# TYPE doris_be_ann_index_in_memory_cnt counter
doris_be_ann_index_in_memory_cnt 0
# TYPE doris_be_small_file_cache_count gauge
doris_be_small_file_cache_count 0
# TYPE doris_be_thrift_current_connections gauge
doris_be_thrift_current_connections{name="backend"} 10
doris_be_thrift_current_connections{name="heartbeat"} 1
# TYPE doris_be_load_average gauge
doris_be_load_average{mode="1_minutes"} 4.860000
doris_be_load_average{mode="5_minutes"} 6.970000
doris_be_load_average{mode="15_minutes"} 7.550000
# TYPE doris_be_proc counter
doris_be_proc{mode="interrupt"} 295181641812
doris_be_proc{mode="ctxt_switch"} 698615979028
doris_be_proc{mode="procs_running"} 8
doris_be_proc{mode="procs_blocked"} 0
# TYPE doris_be_max_network_receive_bytes_rate gauge
doris_be_max_network_receive_bytes_rate 5423735
# TYPE doris_be_result_buffer_block_count gauge
doris_be_result_buffer_block_count 698
# TYPE doris_be_load_channel_count gauge
doris_be_load_channel_count 0
# TYPE doris_be_ann_ivf_on_disk_fetch_page_costs_ms counter
doris_be_ann_ivf_on_disk_fetch_page_costs_ms 0
# TYPE doris_be_brpc_function_endpoint_stub_count gauge
doris_be_brpc_function_endpoint_stub_count 0
# TYPE doris_be_local_file_open_writing gauge
doris_be_local_file_open_writing 36
# TYPE doris_be_heavy_work_active_threads gauge
doris_be_heavy_work_active_threads 0
# TYPE doris_be_heavy_work_max_threads gauge
doris_be_heavy_work_max_threads 18446744073709551615
# TYPE doris_be_memory_jemalloc_active_bytes gauge
doris_be_memory_jemalloc_active_bytes 5278425088
# TYPE doris_be_memory_jemalloc_muzzy_purged_num gauge
doris_be_memory_jemalloc_muzzy_purged_num 34189228757
# TYPE doris_be_network_receive_packets counter
doris_be_network_receive_packets{device="eth0"} 1914080909
doris_be_network_receive_packets{device="lo"} 26890261
# TYPE doris_be_arrow_flight_work_pool_queue_size gauge
doris_be_arrow_flight_work_pool_queue_size 0
# TYPE doris_be_thread_pool_task_wait_worker_time_ns_total counter
doris_be_thread_pool_task_wait_worker_time_ns_total{id="ca466ac793f51dab-d16848274bfa8f81",workload_group="",thread_pool_name="AsyncFetchAutoIncIDExecutor"} 10958
doris_be_thread_pool_task_wait_worker_time_ns_total{id="a44de8673d93c2f4-c86218196e073983",workload_group="",thread_pool_name="EvHttpServer"} 6891867
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e74dbddb49dbe552-cec7d125c8fa3fb4",workload_group="",thread_pool_name="MigrationTaskThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c94a356e0777a0c7-21b509f9bd65b991",workload_group="",thread_pool_name="TaskWP_DELETE"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e541481740247b42-d5804043e53b2eb0",workload_group="",thread_pool_name="TaskWP_CLEAR_TRANSACTION_TASK"} 33358756093
doris_be_thread_pool_task_wait_worker_time_ns_total{id="724c0abf75cf90a9-3e3abeeab6d16d98",workload_group="",thread_pool_name="TaskWP_PUBLISH_VERSION"} 180859150381
doris_be_thread_pool_task_wait_worker_time_ns_total{id="0c409b77075507b7-f92c764e773849b8",workload_group="",thread_pool_name="TaskWP_DROP_TABLE"} 9665108798986
doris_be_thread_pool_task_wait_worker_time_ns_total{id="df486dfa9966a110-05f9dab8d713b4b9",workload_group="",thread_pool_name="TaskWP_CREATE_TABLE"} 112382590166
doris_be_thread_pool_task_wait_worker_time_ns_total{id="684e3b22745421a6-53750ba7baa2cbb8",workload_group="",thread_pool_name="TaskWP_SUBMIT_TABLE_COMPACTION"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="7149ee9859975eea-4af42c73f045668e",workload_group="",thread_pool_name="TaskWP_MOVE"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="9a451cd7fa961da0-18eb5a59fdb2f0b4",workload_group="",thread_pool_name="TaskWP_RELEASE_SNAPSHOT"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="b4449831f5c86e35-232a4f577ad1c2b1",workload_group="",thread_pool_name="TaskWP_MAKE_SNAPSHOT"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="8f429b7ec6727d5f-98d9283d12964abb",workload_group="",thread_pool_name="TaskWP_DOWNLOAD"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="2d4ea06989746114-dc6cad8fed5e708b",workload_group="",thread_pool_name="TaskWP_UPLOAD"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e14a24282105b80d-7dc351b0847e628d",workload_group="",thread_pool_name="TaskWP_CHECK_CONSISTENCY"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="bf4ab940992f57ee-c1a7a6a77ac7548e",workload_group="",thread_pool_name="TaskWP_ALTER_INVERTED_INDEX"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="0e4273e2531ec0a7-25c937cfd8070a81",workload_group="",thread_pool_name="mf_normal"} 184970449185
doris_be_thread_pool_task_wait_worker_time_ns_total{id="5343a5006eb4c059-d1a051493124a5a3",workload_group="",thread_pool_name="ColdDataCompactionTaskThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="3a4baeca76f42c54-b49891e90010e6bc",workload_group="",thread_pool_name="SegCompactionTaskThreadPool"} 333762346
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c34079f5fbe73cd3-e5d5c0b901a1b89e",workload_group="",thread_pool_name="CumuCompactionTaskThreadPool"} 580243242148
doris_be_thread_pool_task_wait_worker_time_ns_total{id="ff43ee17b534ebcd-19e6c9f61c0274a1",workload_group="",thread_pool_name="BaseCompactionTaskThreadPool"} 402504703315
doris_be_thread_pool_task_wait_worker_time_ns_total{id="b84b7308fcad257d-8f21992136b299ac",workload_group="",thread_pool_name="MemTableHighPriorityFlushThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c849b8445ad5dbc1-78fd5bcdc2f75798",workload_group="",thread_pool_name="MemTableFlushThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="914761c28ea6c0e4-51884f5bd2ce309e",workload_group="",thread_pool_name="TaskWP_UPDATE_TABLET_META_INFO"} 20342
doris_be_thread_pool_task_wait_worker_time_ns_total{id="5245378eef663bd4-35d27b98e414a591",workload_group="",thread_pool_name="TaskWP_CLEAN_TRASH"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c94ae366cb2b5c71-9806439ffa9ff3bb",workload_group="",thread_pool_name="p_normal_simple_scheduler"} 1244596
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c147d13bfaebebd5-4c97cc174010479c",workload_group="",thread_pool_name="DownloadThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="61482d670aaedb3e-71d277e4dc89bbb9",workload_group="normal",thread_pool_name="rs_normal"} 158369477187930
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c64e5533441a7acd-6dac81bdc607d09c",workload_group="",thread_pool_name="SegmentPrefetchThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="6348d593ebcf26e4-e6d97b81fec29194",workload_group="",thread_pool_name="NonBlockCloseThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="324d594d3102e87b-6474a510cc810980",workload_group="",thread_pool_name="S3FileSystemThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="af4ae0af82899e35-5ae0b68e45a8b7b4",workload_group="",thread_pool_name="S3FileUploadThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e9408edac6fbc85b-e894d205fce4a491",workload_group="",thread_pool_name="BufferedReaderPrefetchThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c84c0f5f75f5dbaa-60b8022ab5f36697",workload_group="",thread_pool_name="SendBatchThreadPool"} 67230059001
doris_be_thread_pool_task_wait_worker_time_ns_total{id="514810825f75d9bc-5b98ecada2a82e8a",workload_group="",thread_pool_name="SegmentFileWriterThreadPool"} 52565005708126
doris_be_thread_pool_task_wait_worker_time_ns_total{id="7148711d669985d9-95d2456e85b036b9",workload_group="",thread_pool_name="LazyReleaseMemoryThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="854b3ca3d5c3edcc-553e4074cd220196",workload_group="",thread_pool_name="routine_load"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="694b373de3a58e55-1b6914e79ae2a7a8",workload_group="",thread_pool_name="UDFCloseWorkers"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="464fb817654fa0fd-fe8b81ace56008ba",workload_group="",thread_pool_name="GroupCommitThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="df451a3774b69230-455cd12205925ab2",workload_group="",thread_pool_name="GroupCommitReplayWalThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="8b49b9fb74d07988-2de021519038d9a2",workload_group="",thread_pool_name="FragmentMgrAsyncWorkThreadPool"} 606325148503
doris_be_thread_pool_task_wait_worker_time_ns_total{id="b541e11a1ffb0677-6f0c7937d9d46da1",workload_group="",thread_pool_name="TaskWP_UPDATE_VISIBLE_VERSION"} 274132646597
doris_be_thread_pool_task_wait_worker_time_ns_total{id="fb49dd683ba09d51-5068a141c6e6468f",workload_group="",thread_pool_name="TaskWP_PUSH_INDEX_POLICY"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="1844717bd1b922b3-0bc3e21f0a38678d",workload_group="",thread_pool_name="TaskWP_STORAGE_MEDIUM_MIGRATE"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="7248a414c59bb8d1-7a59a88dd8a44993",workload_group="",thread_pool_name="SingleReplicaCompactionTaskThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="06465cbebd6f4710-0225b156f5e698ad",workload_group="normal",thread_pool_name="ls_normal"} 46990108624074401
doris_be_thread_pool_task_wait_worker_time_ns_total{id="b349ab8d5357a1a4-b0d11a555b348792",workload_group="",thread_pool_name="TaskWP_ALTER_TABLE"} 211676483
doris_be_thread_pool_task_wait_worker_time_ns_total{id="7c4600b288cddf72-bb4aa692c2c42f87",workload_group="",thread_pool_name="TaskWP_PUSH_STORAGE_POLICY"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="4145541e0df790ba-544d845bb33721b5",workload_group="",thread_pool_name="p_normal_blocking_scheduler"} 2894590
doris_be_thread_pool_task_wait_worker_time_ns_total{id="6646dfa5d6425a03-86ca7e7fa64eb4a6",workload_group="",thread_pool_name="TaskWP_CLEAN_UDF_CACHE"} 5510895
doris_be_thread_pool_task_wait_worker_time_ns_total{id="454b6c7a20ffbee8-1421edae6769b1be",workload_group="",thread_pool_name="TabletPublishTxnThreadPool"} 661544272237
doris_be_thread_pool_task_wait_worker_time_ns_total{id="524498e6f574bd3a-f4f74af94289c8bc",workload_group="",thread_pool_name="TaskWP_GC_BINLOG"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e44c7e94f40d0fff-8167ff996a3697b0",workload_group="",thread_pool_name="ReportProfileThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="0a4de764390539bc-1bd209fdde4fcab4",workload_group="",thread_pool_name="TabletCalcDeleteBitmapThreadPool"} 48431858638
doris_be_thread_pool_task_wait_worker_time_ns_total{id="4543c697b3e06e57-836059f814e0f689",workload_group="",thread_pool_name="TabletCalcDeleteBitmapThreadPool"} 6883315
doris_be_thread_pool_task_wait_worker_time_ns_total{id="e7497874a56e2077-809953fe87da388a",workload_group="",thread_pool_name="SendTableStatsThreadPool"} 0
doris_be_thread_pool_task_wait_worker_time_ns_total{id="c94405c4d0ba40ae-31a1c638c8c46f81",workload_group="",thread_pool_name="TabletMetaCheckpointTaskThreadPool"} 893730586
doris_be_thread_pool_task_wait_worker_time_ns_total{id="1441c2083caa45b9-f873ad1bcb1672a3",workload_group="",thread_pool_name="TaskWP_PUSH_COOLDOWN_CONF"} 0
# TYPE doris_be_disks_total_capacity gauge
doris_be_disks_total_capacity{path="/opt/apache-doris/be/storage"} 3036857835520
# TYPE doris_be_memory_pswpout gauge
doris_be_memory_pswpout 0
# TYPE doris_be_num_io_bytes_read_from_cache counter
doris_be_num_io_bytes_read_from_cache 0
# TYPE doris_be_snmp_tcp_in_errs counter
doris_be_snmp_tcp_in_errs 30
# TYPE doris_be_disks_avail_capacity gauge
doris_be_disks_avail_capacity{path="/opt/apache-doris/be/storage"} 740745945088
# TYPE doris_be_cache_hit_ratio gauge
doris_be_cache_hit_ratio{name="AnnIndexIVFListCache"} 0.000000
doris_be_cache_hit_ratio{name="CreateTabletRRIdxCache"} 0.186305
doris_be_cache_hit_ratio{name="IndexPageCache"} 0.790660
doris_be_cache_hit_ratio{name="LoadStateChannelCache "} 0.000000
doris_be_cache_hit_ratio{name="PointQueryRowCache"} 0.000000
doris_be_cache_hit_ratio{name="TabletColumnObjectPool"} 0.990013
doris_be_cache_hit_ratio{name="ConditionCache"} 0.550316
doris_be_cache_hit_ratio{name="QueryCache"} 0.000000
doris_be_cache_hit_ratio{name="PointQueryLookupConnectionCache"} 0.000000
doris_be_cache_hit_ratio{name="InvertedIndexSearcherCache"} 0.000000
doris_be_cache_hit_ratio{name="InvertedIndexQueryCache"} 0.000000
doris_be_cache_hit_ratio{name="TabletSchemaCache"} 0.999434
doris_be_cache_hit_ratio{name="SegmentCache"} 0.724454
doris_be_cache_hit_ratio{name="MowTabletVersionCache"} 0.000813
doris_be_cache_hit_ratio{name="MowDeleteBitmapAggCache"} 0.649584
doris_be_cache_hit_ratio{name="DataPageCache"} 0.465680
doris_be_cache_hit_ratio{name="PKIndexPageCache"} 0.996648
doris_be_cache_hit_ratio{name="CommonObjLRUCache"} 0.542619
# TYPE doris_be_workload_group_local_scan_bytes counter
doris_be_workload_group_local_scan_bytes{id="1",path="local_data_dir_0",workload_group="normal"} 1083230277300
doris_be_workload_group_local_scan_bytes{id="1",path="spill_data_dir_0",workload_group="normal"} 1083230277300
# TYPE doris_be_engine_requests_total counter
doris_be_engine_requests_total{status="failed",type="alter_inverted_index"} 0
doris_be_engine_requests_total{status="total",type="alter_inverted_index"} 0
doris_be_engine_requests_total{status="failed",type="publish"} 0
doris_be_engine_requests_total{status="total",type="publish"} 681887
doris_be_engine_requests_total{status="cancelled",type="single_compaction"} 0
doris_be_engine_requests_total{status="failed",type="single_compaction"} 0
doris_be_engine_requests_total{status="total",type="single_compaction"} 0
doris_be_engine_requests_total{status="failed",type="cumulative_compaction"} 0
doris_be_engine_requests_total{status="total",type="cumulative_compaction"} 88606568
doris_be_engine_requests_total{status="failed",type="base_compaction"} 0
doris_be_engine_requests_total{status="total",type="base_compaction"} 13342208
doris_be_engine_requests_total{status="total",type="create_tablet"} 184952
doris_be_engine_requests_total{status="failed",type="create_tablet"} 0
doris_be_engine_requests_total{status="total",type="drop_tablet"} 153023
doris_be_engine_requests_total{status="skip",type="report_all_tablets"} 10
doris_be_engine_requests_total{status="total",type="schema_change"} 0
doris_be_engine_requests_total{status="failed",type="schema_change"} 0
doris_be_engine_requests_total{status="total",type="create_rollup"} 17
doris_be_engine_requests_total{status="failed",type="create_rollup"} 0
doris_be_engine_requests_total{status="total",type="storage_migrate"} 0
doris_be_engine_requests_total{status="total",type="storage_migrate_v2"} 0
doris_be_engine_requests_total{status="failed",type="storage_migrate_v2"} 0
doris_be_engine_requests_total{status="total",type="delete"} 0
doris_be_engine_requests_total{status="failed",type="delete"} 0
doris_be_engine_requests_total{status="total",type="clone"} 57
doris_be_engine_requests_total{status="failed",type="clone"} 0
doris_be_engine_requests_total{status="total",type="finish_task"} 936732
doris_be_engine_requests_total{status="failed",type="finish_task"} 0
# TYPE doris_be_disks_local_used_capacity gauge
doris_be_disks_local_used_capacity{path="/opt/apache-doris/be/storage"} 61679386371
# TYPE doris_be_memory_jemalloc_retained_bytes gauge
doris_be_memory_jemalloc_retained_bytes 201721307136
# TYPE doris_be_light_work_pool_max_queue_size gauge
doris_be_light_work_pool_max_queue_size 18446744073709551615
# TYPE doris_be_thrift_used_clients gauge
doris_be_thrift_used_clients{name="master_fe"} 41
doris_be_thrift_used_clients{name="backend"} 0
doris_be_thrift_used_clients{name="frontend"} 19
doris_be_thrift_used_clients{name="broker"} 0
# TYPE doris_be_thread_pool_submit_failed counter
doris_be_thread_pool_submit_failed{id="ca466ac793f51dab-d16848274bfa8f81",workload_group="",thread_pool_name="AsyncFetchAutoIncIDExecutor"} 0
doris_be_thread_pool_submit_failed{id="a44de8673d93c2f4-c86218196e073983",workload_group="",thread_pool_name="EvHttpServer"} 0
doris_be_thread_pool_submit_failed{id="e74dbddb49dbe552-cec7d125c8fa3fb4",workload_group="",thread_pool_name="MigrationTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="c94a356e0777a0c7-21b509f9bd65b991",workload_group="",thread_pool_name="TaskWP_DELETE"} 0
doris_be_thread_pool_submit_failed{id="e541481740247b42-d5804043e53b2eb0",workload_group="",thread_pool_name="TaskWP_CLEAR_TRANSACTION_TASK"} 0
doris_be_thread_pool_submit_failed{id="724c0abf75cf90a9-3e3abeeab6d16d98",workload_group="",thread_pool_name="TaskWP_PUBLISH_VERSION"} 0
doris_be_thread_pool_submit_failed{id="0c409b77075507b7-f92c764e773849b8",workload_group="",thread_pool_name="TaskWP_DROP_TABLE"} 0
doris_be_thread_pool_submit_failed{id="df486dfa9966a110-05f9dab8d713b4b9",workload_group="",thread_pool_name="TaskWP_CREATE_TABLE"} 0
doris_be_thread_pool_submit_failed{id="684e3b22745421a6-53750ba7baa2cbb8",workload_group="",thread_pool_name="TaskWP_SUBMIT_TABLE_COMPACTION"} 0
doris_be_thread_pool_submit_failed{id="7149ee9859975eea-4af42c73f045668e",workload_group="",thread_pool_name="TaskWP_MOVE"} 0
doris_be_thread_pool_submit_failed{id="9a451cd7fa961da0-18eb5a59fdb2f0b4",workload_group="",thread_pool_name="TaskWP_RELEASE_SNAPSHOT"} 0
doris_be_thread_pool_submit_failed{id="b4449831f5c86e35-232a4f577ad1c2b1",workload_group="",thread_pool_name="TaskWP_MAKE_SNAPSHOT"} 0
doris_be_thread_pool_submit_failed{id="8f429b7ec6727d5f-98d9283d12964abb",workload_group="",thread_pool_name="TaskWP_DOWNLOAD"} 0
doris_be_thread_pool_submit_failed{id="2d4ea06989746114-dc6cad8fed5e708b",workload_group="",thread_pool_name="TaskWP_UPLOAD"} 0
doris_be_thread_pool_submit_failed{id="e14a24282105b80d-7dc351b0847e628d",workload_group="",thread_pool_name="TaskWP_CHECK_CONSISTENCY"} 0
doris_be_thread_pool_submit_failed{id="bf4ab940992f57ee-c1a7a6a77ac7548e",workload_group="",thread_pool_name="TaskWP_ALTER_INVERTED_INDEX"} 0
doris_be_thread_pool_submit_failed{id="0e4273e2531ec0a7-25c937cfd8070a81",workload_group="",thread_pool_name="mf_normal"} 0
doris_be_thread_pool_submit_failed{id="5343a5006eb4c059-d1a051493124a5a3",workload_group="",thread_pool_name="ColdDataCompactionTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="3a4baeca76f42c54-b49891e90010e6bc",workload_group="",thread_pool_name="SegCompactionTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="c34079f5fbe73cd3-e5d5c0b901a1b89e",workload_group="",thread_pool_name="CumuCompactionTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="ff43ee17b534ebcd-19e6c9f61c0274a1",workload_group="",thread_pool_name="BaseCompactionTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="b84b7308fcad257d-8f21992136b299ac",workload_group="",thread_pool_name="MemTableHighPriorityFlushThreadPool"} 0
doris_be_thread_pool_submit_failed{id="c849b8445ad5dbc1-78fd5bcdc2f75798",workload_group="",thread_pool_name="MemTableFlushThreadPool"} 0
doris_be_thread_pool_submit_failed{id="914761c28ea6c0e4-51884f5bd2ce309e",workload_group="",thread_pool_name="TaskWP_UPDATE_TABLET_META_INFO"} 0
doris_be_thread_pool_submit_failed{id="5245378eef663bd4-35d27b98e414a591",workload_group="",thread_pool_name="TaskWP_CLEAN_TRASH"} 0
doris_be_thread_pool_submit_failed{id="c94ae366cb2b5c71-9806439ffa9ff3bb",workload_group="",thread_pool_name="p_normal_simple_scheduler"} 0
doris_be_thread_pool_submit_failed{id="c147d13bfaebebd5-4c97cc174010479c",workload_group="",thread_pool_name="DownloadThreadPool"} 0
doris_be_thread_pool_submit_failed{id="61482d670aaedb3e-71d277e4dc89bbb9",workload_group="normal",thread_pool_name="rs_normal"} 0
doris_be_thread_pool_submit_failed{id="c64e5533441a7acd-6dac81bdc607d09c",workload_group="",thread_pool_name="SegmentPrefetchThreadPool"} 0
doris_be_thread_pool_submit_failed{id="6348d593ebcf26e4-e6d97b81fec29194",workload_group="",thread_pool_name="NonBlockCloseThreadPool"} 0
doris_be_thread_pool_submit_failed{id="324d594d3102e87b-6474a510cc810980",workload_group="",thread_pool_name="S3FileSystemThreadPool"} 0
doris_be_thread_pool_submit_failed{id="af4ae0af82899e35-5ae0b68e45a8b7b4",workload_group="",thread_pool_name="S3FileUploadThreadPool"} 0
doris_be_thread_pool_submit_failed{id="e9408edac6fbc85b-e894d205fce4a491",workload_group="",thread_pool_name="BufferedReaderPrefetchThreadPool"} 0
doris_be_thread_pool_submit_failed{id="c84c0f5f75f5dbaa-60b8022ab5f36697",workload_group="",thread_pool_name="SendBatchThreadPool"} 0
doris_be_thread_pool_submit_failed{id="514810825f75d9bc-5b98ecada2a82e8a",workload_group="",thread_pool_name="SegmentFileWriterThreadPool"} 0
doris_be_thread_pool_submit_failed{id="7148711d669985d9-95d2456e85b036b9",workload_group="",thread_pool_name="LazyReleaseMemoryThreadPool"} 0
doris_be_thread_pool_submit_failed{id="854b3ca3d5c3edcc-553e4074cd220196",workload_group="",thread_pool_name="routine_load"} 0
doris_be_thread_pool_submit_failed{id="694b373de3a58e55-1b6914e79ae2a7a8",workload_group="",thread_pool_name="UDFCloseWorkers"} 0
doris_be_thread_pool_submit_failed{id="464fb817654fa0fd-fe8b81ace56008ba",workload_group="",thread_pool_name="GroupCommitThreadPool"} 0
doris_be_thread_pool_submit_failed{id="df451a3774b69230-455cd12205925ab2",workload_group="",thread_pool_name="GroupCommitReplayWalThreadPool"} 0
doris_be_thread_pool_submit_failed{id="8b49b9fb74d07988-2de021519038d9a2",workload_group="",thread_pool_name="FragmentMgrAsyncWorkThreadPool"} 0
doris_be_thread_pool_submit_failed{id="b541e11a1ffb0677-6f0c7937d9d46da1",workload_group="",thread_pool_name="TaskWP_UPDATE_VISIBLE_VERSION"} 0
doris_be_thread_pool_submit_failed{id="fb49dd683ba09d51-5068a141c6e6468f",workload_group="",thread_pool_name="TaskWP_PUSH_INDEX_POLICY"} 0
doris_be_thread_pool_submit_failed{id="1844717bd1b922b3-0bc3e21f0a38678d",workload_group="",thread_pool_name="TaskWP_STORAGE_MEDIUM_MIGRATE"} 0
doris_be_thread_pool_submit_failed{id="7248a414c59bb8d1-7a59a88dd8a44993",workload_group="",thread_pool_name="SingleReplicaCompactionTaskThreadPool"} 0
doris_be_thread_pool_submit_failed{id="06465cbebd6f4710-0225b156f5e698ad",workload_group="normal",thread_pool_name="ls_normal"} 0
doris_be_thread_pool_submit_failed{id="b349ab8d5357a1a4-b0d11a555b348792",workload_group="",thread_pool_name="TaskWP_ALTER_TABLE"} 0
doris_be_thread_pool_submit_failed{id="7c4600b288cddf72-bb4aa692c2c42f87",workload_group="",thread_pool_name="TaskWP_PUSH_STORAGE_POLICY"} 0
doris_be_thread_pool_submit_failed{id="4145541e0df790ba-544d845bb33721b5",workload_group="",thread_pool_name="p_normal_blocking_scheduler"} 0
doris_be_thread_pool_submit_failed{id="6646dfa5d6425a03-86ca7e7fa64eb4a6",workload_group="",thread_pool_name="TaskWP_CLEAN_UDF_CACHE"} 0
doris_be_thread_pool_submit_failed{id="454b6c7a20ffbee8-1421edae6769b1be",workload_group="",thread_pool_name="TabletPublishTxnThreadPool"} 0
doris_be_thread_pool_submit_failed{id="524498e6f574bd3a-f4f74af94289c8bc",workload_group="",thread_pool_name="TaskWP_GC_BINLOG"} 0
doris_be_thread_pool_submit_failed{id="e44c7e94f40d0fff-8167ff996a3697b0",workload_group="",thread_pool_name="ReportProfileThreadPool"} 0
...

字符太长,我截掉了后面一半