第一次玩,请教各位大佬!!!
kubernetes部署foundationDB,所有pod运行正常,但是pod内日志报错:
➜ doris kubectl -n doris get fdb
NAME GENERATION RECONCILED AVAILABLE FULLREPLICATION VERSION AGE
fdb-cluster 1 7.4.5 59m
yaml如下:
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
apiVersion: apps.foundationdb.org/v1beta2
kind: FoundationDBCluster
metadata:
name: fdb-cluster
finalizers:
- doris.io/compute-storage-separation
spec:
databaseConfiguration:
redundancy_mode: "double"
automationOptions:
replacements:
enabled: true
#imageType: split
minimumUptimeSecondsForBounce: 60
processCounts:
cluster_controller: 1
log: 4
storage: 3
stateless: 6
processes:
general:
podTemplate:
spec:
containers:
- name: foundationdb
resources:
requests:
cpu: 1
memory: 1Gi
securityContext:
runAsUser: 0
privileged: true
- name: foundationdb-kubernetes-sidecar
# image: swr.cn-north-4.myhuaweicloud.com/pixiu-public/lbemi/foundationdb-kubernetes-sidecar
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
runAsUser: 0
privileged: true
storage:
podTemplate:
spec:
containers:
- name: foundationdb
resources:
requests:
cpu: 1
memory: 8Gi
securityContext:
runAsUser: 0
privileged: true
- name: foundationdb-kubernetes-sidecar
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
runAsUser: 0
privileged: true
volumeClaimTemplate:
spec:
#storageClassName: ${storageClassName}
resources:
requests:
storage: 200Gi
log:
podTemplate:
spec:
containers:
- name: foundationdb
resources:
requests:
cpu: 1
memory: 2Gi
securityContext:
runAsUser: 0
privileged: true
- name: foundationdb-kubernetes-sidecar
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
runAsUser: 0
privileged: true
volumeClaimTemplate:
spec:
#storageClassName: ${storageClassName}
resources:
requests:
storage: 100Gi
routing:
useDNSInClusterFile: true
mainContainer:
# use the foundationdb/fdb-kubernetes-monitor:7.1.38 as example to display how to config private repository image.
imageConfigs:
- baseImage: swr.cn-north-4.myhuaweicloud.com/pixiu-public/lbemi/fdb-kubernetes-monitor
tag: 7.4.5
sidecarContainer:
imageConfigs:
- baseImage: swr.cn-north-4.myhuaweicloud.com/pixiu-public/lbemi/fdb-kubernetes-monitor
tag: 7.4.5
enableLivenessProbe: true
enableReadinessProbe: false
useExplicitListenAddress: true
version: 7.4.5
fdb-cluster-log-44930:
"level":"info","ts":1765036226.5572069,"msg":"Starting subprocess","processNumber":1,"area":"runProcess","arguments":["/usr/bin/fdbserver","--cluster_file=/var/fdb/data/fdb.cluster","--seed_cluster_file=/var/dynamic-conf/fdb.cluster","--public_address=[10.174.53.15]:4501","--class=log","--logdir=/var/log/fdb-tra
│ {"level":"info","ts":1765036226.557463,"msg":"Subprocess started","processNumber":1,"area":"runProcess","PID":194}
│ {"level":"error","ts":1765036226.5993257,"msg":"Subprocess error log","processNumber":1,"area":"runProcess","msg":"ERROR: error creating or opening process id file `/var/fdb/data/1/processId'.","PID":194}
│ {"level":"error","ts":1765036226.5994596,"msg":"Subprocess error log","processNumber":1,"area":"runProcess","msg":"Fatal Error: Unable to lock the file","PID":194}
│ {"level":"error","ts":1765036226.6114664,"msg":"Error from subprocess","processNumber":1,"area":"runProcess","PID":194,"error":"exit status 1"}
│ {"level":"info","ts":1765036226.6114936,"msg":"Subprocess terminated","processNumber":1,"area":"runProcess","exitCode":1,"PID":194,"lastExecutionDurationSeconds":"54.028142ms"}
│ {"level":"info","ts":1765036226.6115005,"msg":"Backing off from restarting subprocess","processNumber":1,"area":"runProcess","backoffDuration":"1m0s","lastExecutionDurationSeconds":"54.028142ms","errorCounter":54,"exitCode":1}
│ {"level":"info","ts":1765036229.4436235,"msg":"waiting for file to be created","path":"/var/fdb/data/fdb.cluster"}
│ {"level":"info","ts":1765036234.447676,"msg":"waiting for file to be created","path":"/var/fdb/data/fdb.cluster"}
│ {"level":"info","ts":1765036239.4516165,"msg":"waiting for file to be created","path":"/var/fdb/data/fdb.cluster"}
进入pod内查看日志: tail -n 300 be.WARNING
RuntimeLogger W20251206 14:12:20.193630 759 status.h:415] meet error status: [E-3001]rocks db open failed, reason: Corruption: Can't access /000228.sst: IO error: while stat a file for size: /opt/apache-doris/be/storage/meta/000228.sst: No such file or directory
Can't access /000224.sst: IO error: while stat a file for size: /opt/apache-doris/be/storage/meta/000224.sst: No such file or directory
0# doris::OlapMeta::init() at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
1# doris::DataDir::_init_meta() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# doris::DataDir::init(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::StorageEngine::_init_store_map()::$_0> > >::_M_run() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
4# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85
5# ?
6# ?
RuntimeLogger W20251206 14:12:20.195677 759 status.h:415] meet error status: [IO_ERROR]open rocksdb failed, path=/opt/apache-doris/be/storage
0# doris::DataDir::_init_meta() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
1# doris::DataDir::init(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::StorageEngine::_init_store_map()::$_0> > >::_M_run() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85
4# ?
5# ?
RuntimeLogger W20251206 14:12:20.195714 759 data_dir.cpp:206] init OlapMeta failed, error: [IO_ERROR]open rocksdb failed, path=/opt/apache-doris/be/storage
0# doris::DataDir::_init_meta() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
1# doris::DataDir::init(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::StorageEngine::_init_store_map()::$_0> > >::_M_run() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85
4# ?
5# ?
RuntimeLogger W20251206 14:12:20.195724 759 data_dir.cpp:131] _init_meta failed, error: [IO_ERROR]open rocksdb failed, path=/opt/apache-doris/be/storage
0# doris::DataDir::_init_meta() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
1# doris::DataDir::init(bool) at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
2# std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::StorageEngine::_init_store_map()::$_0> > >::_M_run() at /home/zcp/repo_center/doris_release/doris/be/src/common/status.h:494
3# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85
4# ?
5# ?
RuntimeLogger W20251206 14:12:20.195731 759 storage_engine.cpp:224] Store load failed, status=[IO_ERROR]open rocksdb failed, path=/opt/apache-doris/be/storage
fdb-cluster相关pod运行日志都提示 {"level":"info","ts":1765036544.9878392,"msg":"waiting for file to be created","path":"/var/fdb/data/fdb.cluster"}
