集群情况:
Doris 版本 3.0.7,FE和BE都是独立节点部署。
FE 8C32G 3台
BE 12C48G 5台
JAVA_OPTS_FOR_JDK_17="-Djava.security.krb5.conf=/etc/krb5.conf -Dallow_weak_crypto=true -Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xmx23552m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR -Xlog:gc*,classhisto*=trace:$LOG_DIR/fe.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.ref=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED"
FE节点内存总是有这种直线上升情况,看gc日志,发现没有fullgc。这个下降是强制重启FE恢复正常,手工触发fullgc 也会正常。
jmap -histo top 20
num #instances #bytes class name (module)
-------------------------------------------------------
1: 235393575 7352541896 [B (java.base@17.0.2)
2: 4374583 104989992 com.google.common.base.Suppliers$NonSerializableMemoizingSupplier
3: 78152 82698672 [I (java.base@17.0.2)
4: 1154062 54774104 [Ljava.lang.Object; (java.base@17.0.2)
5: 482205 42434040 org.apache.doris.nereids.trees.expressions.SlotReference
6: 1617733 38825592 java.lang.String (java.base@17.0.2)
7: 862048 27585536 java.util.HashMap$Node (java.base@17.0.2)
8: 397685 25451840 org.apache.doris.nereids.trees.expressions.Alias
9: 380025 24321600 org.apache.doris.nereids.trees.expressions.literal.VarcharLiteral
10: 781698 18760752 java.util.ArrayList (java.base@17.0.2)
11: 1020715 16331440 java.lang.Integer (java.base@17.0.2)
12: 20190 16074688 [C (java.base@17.0.2)
13: 108509 14011120 [Ljava.util.HashMap$Node; (java.base@17.0.2)
14: 865799 13852784 org.apache.doris.nereids.trees.expressions.ExprId
15: 551426 13234224 org.apache.doris.nereids.util.MutableState$SingleMutableState
16: 818429 13094864 org.apache.doris.nereids.trees.expressions.Expression$$Lambda$445/0x00000008016bd840
17: 818240 13091840 org.apache.doris.nereids.trees.expressions.Expression$$Lambda$444/0x00000008016bd390
18: 813592 13017472 org.apache.doris.nereids.trees.expressions.Expression$$Lambda$446/0x00000008016bdcf0
如果手工触发fullgc,内存就会下降。这种情况应该如何排查处理。