3.0.3的异常停止后,启动fe报错,有大佬遇见过这个问题吗?

Viewed 10

报如下错误,网上有说是fe的jvm没有设置对,我把JAVA_OPTS增大这个了,-Xmx24576m -XX:+UseG1GC -XX:MaxGCPauseMillis=100,还是不起作用完整的JAVA_OPTS="-Xmx24576m -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
错误如下

2026-01-08 15:37:27,572 WARN (MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)|15) [BDBStateChangeListener.stateChange():59] this node is DETACHED
2026-01-08 15:37:27,572 INFO (MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)|15) [LogUtils.stdout():50] StdoutLogger 2026-01-08 15:37:27,572 this node is DETACHED
2026-01-08 15:37:36,580 WARN (stateListener|15) [BDBHA.fencing():95] fencing failed. tried 0 times
com.sleepycat.je.rep.InsufficientAcksException: (JE 18.3.12) Transaction: -373408486  VLSN: 744,918,440, initiated at: 15:37:26.  Insufficient acks for policy:SIMPLE_MAJORITY. Need replica acks: 1. Missing replica acks: 1. Timeout: 10000ms. FeederState=fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)[DETACHED]
No feeders.

        at com.sleepycat.je.rep.impl.node.DurabilityQuorum.ensureSufficientAcks(DurabilityQuorum.java:205) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.stream.FeederTxns.awaitReplicaAcks(FeederTxns.java:188) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.RepImpl.postLogCommitHookInternal(RepImpl.java:1444) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.RepImpl.postLogCommitHook(RepImpl.java:1403) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.txn.MasterTxn.postLogCommitHook(MasterTxn.java:228) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.txn.Txn.commit(Txn.java:778) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.txn.Txn.commit(Txn.java:631) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.txn.Txn.operationEnd(Txn.java:1773) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.put(Database.java:1638) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.putNoOverwrite(Database.java:1737) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at org.apache.doris.ha.BDBHA.fencing(BDBHA.java:85) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.transferToMaster(Env.java:1578) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.access$1200(Env.java:348) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2878) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119) ~[doris-fe.jar:1.2-SNAPSHOT]
2026-01-08 15:37:36,597 INFO (UNKNOWN fe_28d77995_c87e_4789_8faa_3f64a01e2093(-1)|15) [Env.waitForReady():1137] wait catalog to be ready. feType:INIT isReady:false, counter:101 reason: (JE 18.3.12) fe_28d77995_c87e_4789_8faa_3f64a01e2093(1):/data/doris-3.0.3/fe/doris-meta/bdb Uncaught exception in feeder thread:Thread[Feeder Input for fe_5ed15a3c_2183_4186_b08b_ec9a9b47d152,5,main] UNCAUGHT_EXCEPTION: Uncaught Exception in internal thread, unable to continue. Environment is invalid and must be closed. Originally thrown by HA thread: MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)
2026-01-08 15:37:38,581 WARN (stateListener|15) [BDBHA.fencing():95] fencing failed. tried 1 times
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) fe_28d77995_c87e_4789_8faa_3f64a01e2093(1):/data/doris-3.0.3/fe/doris-meta/bdb Uncaught exception in feeder thread:Thread[Feeder Input for fe_5ed15a3c_2183_4186_b08b_ec9a9b47d152,5,main] UNCAUGHT_EXCEPTION: Uncaught Exception in internal thread, unable to continue. Environment is invalid and must be closed. Originally thrown by HA thread: MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)
        at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:230) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1835) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.checkOpen(EnvironmentImpl.java:1844) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.checkOpen(Environment.java:2697) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.checkEnv(Database.java:2413) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.count(Database.java:2039) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at org.apache.doris.ha.BDBHA.fencing(BDBHA.java:77) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.transferToMaster(Env.java:1578) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.access$1200(Env.java:348) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2878) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) fe_28d77995_c87e_4789_8faa_3f64a01e2093(1):/data/doris-3.0.3/fe/doris-meta/bdb Uncaught exception in feeder thread:Thread[Feeder Input for fe_5ed15a3c_2183_4186_b08b_ec9a9b47d152,5,main] UNCAUGHT_EXCEPTION: Uncaught Exception in internal thread, unable to continue. Environment is invalid and must be closed. Originally thrown by HA thread: MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)
        at com.sleepycat.je.EnvironmentFailureException.promote(EnvironmentFailureException.java:340) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$IOThreadsHandler.uncaughtException(Feeder.java:1590) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at java.lang.Thread.dispatchUncaughtException(Thread.java:2019) ~[?:?]
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -122971540 out of bounds for length 1001
        at java.lang.invoke.VarHandle$1.apply(VarHandle.java:2187) ~[?:?]
        at java.lang.invoke.VarHandle$1.apply(VarHandle.java:2184) ~[?:?]
        at jdk.internal.util.Preconditions$1.apply(Preconditions.java:177) ~[?:?]
        at jdk.internal.util.Preconditions$1.apply(Preconditions.java:174) ~[?:?]
        at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:62) ~[?:?]
        at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?]
        at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266) ~[?:?]
        at java.lang.invoke.VarHandleLongs$Array.getAndAdd(VarHandleLongs.java:926) ~[?:?]
        at java.util.concurrent.atomic.AtomicLongArray.incrementAndGet(AtomicLongArray.java:234) ~[?:?]
        at com.sleepycat.je.utilint.LatencyPercentile.add(LatencyPercentile.java:288) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.processHeartbeatResponse(Feeder.java:981) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.runResponseLoop(Feeder.java:856) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.run(Feeder.java:755) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
2026-01-08 15:37:40,582 WARN (stateListener|15) [BDBHA.fencing():95] fencing failed. tried 2 times
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) fe_28d77995_c87e_4789_8faa_3f64a01e2093(1):/data/doris-3.0.3/fe/doris-meta/bdb Uncaught exception in feeder thread:Thread[Feeder Input for fe_5ed15a3c_2183_4186_b08b_ec9a9b47d152,5,main] UNCAUGHT_EXCEPTION: Uncaught Exception in internal thread, unable to continue. Environment is invalid and must be closed. Originally thrown by HA thread: MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)
        at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:230) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1835) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.checkOpen(EnvironmentImpl.java:1844) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.checkOpen(Environment.java:2697) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.checkEnv(Database.java:2413) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Database.count(Database.java:2039) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at org.apache.doris.ha.BDBHA.fencing(BDBHA.java:77) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.transferToMaster(Env.java:1578) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.access$1200(Env.java:348) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2878) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) fe_28d77995_c87e_4789_8faa_3f64a01e2093(1):/data/doris-3.0.3/fe/doris-meta/bdb Uncaught exception in feeder thread:Thread[Feeder Input for fe_5ed15a3c_2183_4186_b08b_ec9a9b47d152,5,main] UNCAUGHT_EXCEPTION: Uncaught Exception in internal thread, unable to continue. Environment is invalid and must be closed. Originally thrown by HA thread: MASTER fe_28d77995_c87e_4789_8faa_3f64a01e2093(1)
        at com.sleepycat.je.EnvironmentFailureException.promote(EnvironmentFailureException.java:340) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$IOThreadsHandler.uncaughtException(Feeder.java:1590) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at java.lang.Thread.dispatchUncaughtException(Thread.java:2019) ~[?:?]
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -122971540 out of bounds for length 1001
        at java.lang.invoke.VarHandle$1.apply(VarHandle.java:2187) ~[?:?]
        at java.lang.invoke.VarHandle$1.apply(VarHandle.java:2184) ~[?:?]
        at jdk.internal.util.Preconditions$1.apply(Preconditions.java:177) ~[?:?]
        at jdk.internal.util.Preconditions$1.apply(Preconditions.java:174) ~[?:?]
        at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:62) ~[?:?]
        at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?]
        at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266) ~[?:?]
        at java.lang.invoke.VarHandleLongs$Array.getAndAdd(VarHandleLongs.java:926) ~[?:?]
        at java.util.concurrent.atomic.AtomicLongArray.incrementAndGet(AtomicLongArray.java:234) ~[?:?]
        at com.sleepycat.je.utilint.LatencyPercentile.add(LatencyPercentile.java:288) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.processHeartbeatResponse(Feeder.java:981) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.runResponseLoop(Feeder.java:856) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Feeder$InputThread.run(Feeder.java:755) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
2026-01-08 15:37:42,583 ERROR (stateListener|15) [Env.transferToMaster():1579] fencing failed. will exit
1 Answers

这个报错和JAVA_OPTS 没关系,主要是: Need replica acks: 1. Missing replica acks: 1. Timeout: 10000ms。

ack超时了,多数派丢失导致宕机,现在是几个FE呢?需要同时拉起半数以上的 FE 节点才能进行选主。