GL-MT6000 reboot every 10-12 days

christech · March 12, 2025, 1:46pm

Hello,

First post here but long time follower of OpenWRT. I installed 24.10 on my GL-MT6000 shortly after release and have had reboots every 10-12 days. With the frequency, these do not seem to be power related as the same surge protector is used on several other devices.

Whats the best place to start troubleshooting?

Running version OpenWrt 24.10.0 r28427-6df0e3d02a

~35 WiFi clients served in a main subnet and isolated guest subnet
AdguardHome and AttendedSysupgrade installed
4 WiFi 2.4 Ghz networks
1 WiFi 5 GHZ network
500/500 Mbps internet connectivity

brada4 · March 12, 2025, 1:58pm

What additional services you run on it?
Number of wifi clients?
Freshly booted OpenWrt uses 25-30MB RAM, so what are spevific things you added?

christech · March 12, 2025, 3:24pm

Good question. I have about 35 WiFi clients and the only additional services I utilize is AdguardHome and Attended Sysupgrade.

romanovj · March 12, 2025, 3:50pm

Check /sys/fs/pstore for logs.
Track memory usage
It's could be memory leak in adg.

christech · March 12, 2025, 4:01pm

Good idea on /sys/fs/pstore. Looks like a kernel panic but I don't see a reason.

<6>[323378.695005] mt798x-wmac 18000000.wifi phy1-ap0: entered promiscuous mode
<6>[323461.659387] br-lan: port 6(phy1-ap0) entered blocking state
<6>[323461.665057] br-lan: port 6(phy1-ap0) entered forwarding state
<0>[913557.617302] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000007
<2>[913557.625038] SMP: stopping secondary CPUs
<0>[913557.629036] Kernel Offset: disabled
<0>[913557.632595] CPU features: 0x0,00000000,00000000,1000400b
<0>[913557.637975] Memory Limit: none

brada4 · March 12, 2025, 4:02pm

Show the OOM-s before that.

christech · March 12, 2025, 4:07pm

Apologies, but what is OOM? Out of memory? How to show?

romanovj · March 12, 2025, 4:26pm

There should be many more lines in the files.

christech · March 12, 2025, 4:42pm

That's honestly about it but will add below the entire output. There is a gap between 5 days and 10 days as these are in seconds.

root@OpenWrt:/sys/fs/pstore# cat dmesg-ramoops-0 
Panic#1 Part1
<6>[   25.438451] br-iot: port 3(phy0-ap3) entered blocking state
<6>[   25.444069] br-iot: port 3(phy0-ap3) entered forwarding state
<6>[   25.450687] mt798x-wmac 18000000.wifi phy0-ap3: left allmulticast mode
<6>[   25.457234] mt798x-wmac 18000000.wifi phy0-ap3: left promiscuous mode
<6>[   25.463850] br-iot: port 3(phy0-ap3) entered disabled state
<6>[   25.511613] br-iot: port 3(phy0-ap3) entered blocking state
<6>[   25.517184] br-iot: port 3(phy0-ap3) entered disabled state
<6>[   25.522789] mt798x-wmac 18000000.wifi phy0-ap3: entered allmulticast mode
<6>[   25.529737] mt798x-wmac 18000000.wifi phy0-ap3: entered promiscuous mode
<6>[   25.536551] br-iot: port 3(phy0-ap3) entered blocking state
<6>[   25.542116] br-iot: port 3(phy0-ap3) entered forwarding state
<6>[323313.325825] br-iot: port 3(phy0-ap3) entered disabled state
<6>[323313.404237] mt798x-wmac 18000000.wifi phy0-ap3 (unregistering): left allmulticast mode
<6>[323313.412676] mt798x-wmac 18000000.wifi phy0-ap3 (unregistering): left promiscuous mode
<6>[323313.420729] br-iot: port 3(phy0-ap3) entered disabled state
<6>[323313.705137] br-iot: port 2(phy0-ap2) entered disabled state
<6>[323313.783725] mt798x-wmac 18000000.wifi phy0-ap2 (unregistering): left allmulticast mode
<6>[323313.791853] mt798x-wmac 18000000.wifi phy0-ap2 (unregistering): left promiscuous mode
<6>[323313.799909] br-iot: port 2(phy0-ap2) entered disabled state
<6>[323314.843478] br-iot: port 1(phy0-ap1) entered disabled state
<6>[323314.912359] mt798x-wmac 18000000.wifi phy0-ap1 (unregistering): left allmulticast mode
<6>[323314.920655] mt798x-wmac 18000000.wifi phy0-ap1 (unregistering): left promiscuous mode
<6>[323314.928640] br-iot: port 1(phy0-ap1) entered disabled state
<6>[323315.814726] mt798x-wmac 18000000.wifi phy0-ap0: left allmulticast mode
<6>[323315.821380] mt798x-wmac 18000000.wifi phy0-ap0: left promiscuous mode
<6>[323315.827996] br-lan: port 7(phy0-ap0) entered disabled state
<6>[323316.495050] br-lan: port 7(phy0-ap0) entered blocking state
<6>[323316.500762] br-lan: port 7(phy0-ap0) entered disabled state
<6>[323316.506448] mt798x-wmac 18000000.wifi phy0-ap0: entered allmulticast mode
<6>[323316.513515] mt798x-wmac 18000000.wifi phy0-ap0: entered promiscuous mode
<6>[323316.520403] br-lan: port 7(phy0-ap0) entered blocking state
<6>[323316.526046] br-lan: port 7(phy0-ap0) entered forwarding state
<6>[323316.610082] br-lan: port 7(phy0-ap0) entered disabled state
<6>[323324.326831] br-lan: port 7(phy0-ap0) entered blocking state
<6>[323324.332505] br-lan: port 7(phy0-ap0) entered forwarding state
<6>[323324.483107] br-iot: port 1(phy0-ap1) entered blocking state
<6>[323324.488990] br-iot: port 1(phy0-ap1) entered disabled state
<6>[323324.494752] mt798x-wmac 18000000.wifi phy0-ap1: entered allmulticast mode
<6>[323324.501952] mt798x-wmac 18000000.wifi phy0-ap1: entered promiscuous mode
<6>[323324.510380] br-iot: port 1(phy0-ap1) entered blocking state
<6>[323324.516066] br-iot: port 1(phy0-ap1) entered forwarding state
<6>[323324.523202] mt798x-wmac 18000000.wifi phy0-ap1: left allmulticast mode
<6>[323324.529913] mt798x-wmac 18000000.wifi phy0-ap1: left promiscuous mode
<6>[323324.536568] br-iot: port 1(phy0-ap1) entered disabled state
<6>[323324.592940] br-iot: port 1(phy0-ap1) entered blocking state
<6>[323324.598600] br-iot: port 1(phy0-ap1) entered disabled state
<6>[323324.604302] mt798x-wmac 18000000.wifi phy0-ap1: entered allmulticast mode
<6>[323324.611310] mt798x-wmac 18000000.wifi phy0-ap1: entered promiscuous mode
<6>[323324.618208] br-iot: port 1(phy0-ap1) entered blocking state
<6>[323324.623870] br-iot: port 1(phy0-ap1) entered forwarding state
<6>[323324.782813] br-iot: port 2(phy0-ap2) entered blocking state
<6>[323324.788568] br-iot: port 2(phy0-ap2) entered disabled state
<6>[323324.794421] mt798x-wmac 18000000.wifi phy0-ap2: entered allmulticast mode
<6>[323324.801488] mt798x-wmac 18000000.wifi phy0-ap2: entered promiscuous mode
<6>[323324.809604] br-iot: port 2(phy0-ap2) entered blocking state
<6>[323324.815279] br-iot: port 2(phy0-ap2) entered forwarding state
<6>[323324.822219] mt798x-wmac 18000000.wifi phy0-ap2: left allmulticast mode
<6>[323324.828846] mt798x-wmac 18000000.wifi phy0-ap2: left promiscuous mode
<6>[323324.835447] br-iot: port 2(phy0-ap2) entered disabled state
<6>[323324.892630] br-iot: port 2(phy0-ap2) entered blocking state
<6>[323324.898289] br-iot: port 2(phy0-ap2) entered disabled state
<6>[323324.903995] mt798x-wmac 18000000.wifi phy0-ap2: entered allmulticast mode
<6>[323324.911147] mt798x-wmac 18000000.wifi phy0-ap2: entered promiscuous mode
<6>[323324.918102] br-iot: port 2(phy0-ap2) entered blocking state
<6>[323324.923766] br-iot: port 2(phy0-ap2) entered forwarding state
<6>[323324.983099] br-iot: port 3(phy0-ap3) entered blocking state
<6>[323324.988768] br-iot: port 3(phy0-ap3) entered disabled state
<6>[323324.994526] mt798x-wmac 18000000.wifi phy0-ap3: entered allmulticast mode
<6>[323325.001974] mt798x-wmac 18000000.wifi phy0-ap3: entered promiscuous mode
<6>[323325.010910] br-iot: port 3(phy0-ap3) entered blocking state
<6>[323325.016583] br-iot: port 3(phy0-ap3) entered forwarding state
<6>[323325.023726] mt798x-wmac 18000000.wifi phy0-ap3: left allmulticast mode
<6>[323325.030369] mt798x-wmac 18000000.wifi phy0-ap3: left promiscuous mode
<6>[323325.036998] br-iot: port 3(phy0-ap3) entered disabled state
<6>[323325.142398] br-iot: port 3(phy0-ap3) entered blocking state
<6>[323325.148064] br-iot: port 3(phy0-ap3) entered disabled state
<6>[323325.153765] mt798x-wmac 18000000.wifi phy0-ap3: entered allmulticast mode
<6>[323325.160922] mt798x-wmac 18000000.wifi phy0-ap3: entered promiscuous mode
<6>[323325.167825] br-iot: port 3(phy0-ap3) entered blocking state
<6>[323325.173485] br-iot: port 3(phy0-ap3) entered forwarding state
<6>[323377.307392] mt798x-wmac 18000000.wifi phy1-ap0: left allmulticast mode
<6>[323377.314128] mt798x-wmac 18000000.wifi phy1-ap0: left promiscuous mode
<6>[323377.320827] br-lan: port 6(phy1-ap0) entered disabled state
<6>[323378.676430] br-lan: port 6(phy1-ap0) entered blocking state
<6>[323378.682100] br-lan: port 6(phy1-ap0) entered disabled state
<6>[323378.687866] mt798x-wmac 18000000.wifi phy1-ap0: entered allmulticast mode
<6>[323378.695005] mt798x-wmac 18000000.wifi phy1-ap0: entered promiscuous mode
<6>[323461.659387] br-lan: port 6(phy1-ap0) entered blocking state
<6>[323461.665057] br-lan: port 6(phy1-ap0) entered forwarding state
<0>[913557.617302] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000007
<2>[913557.625038] SMP: stopping secondary CPUs
<0>[913557.629036] Kernel Offset: disabled
<0>[913557.632595] CPU features: 0x0,00000000,00000000,1000400b
<0>[913557.637975] Memory Limit: none

egc · March 12, 2025, 4:48pm

If you cannot pin-point it further I would remove Adguard from the equation and see if the reboots are gone.

If you want Adblocking use and Adblocker for secure DNS consider https-dns-proxy

brada4 · March 12, 2025, 6:24pm

Or stretch agh using zram-swap - in 20 days youll see swap used and ?adguard? just running very slow.

Dante · March 12, 2025, 6:34pm

AFAICR, the default Adguard Home installation writes the logs to /var, which is an alias for /tmp, which in turn is backed by RAM, and with 35+ clients these might add up pretty fast, depending on what these clients do. So consider adjusting the log retention interval in Adguard and/or moving the logs somewhere else.

christech · March 12, 2025, 7:22pm

Ooo, good thinking! I will try limiting my log retention. I'm willing to say you probably solved this as I am overly generous in this department usually.

Thank you all!

Dante · March 21, 2025, 3:11pm

Did adjusting the log retention/location improve the situation?

christech · March 27, 2025, 4:38pm

I have an uptime of 16+ days now. New record for me and its working fine. Thanks for checking in!

Decreasing DNS cache size and logging to lowest levels resolved the issue.

phinn · March 27, 2025, 5:45pm

Have lots of services and clients on my MT6000 and currently 49 day uptime on 24.10. I would consider dropping adguard home and using adblock maybe it's that.

trevormcooper · March 31, 2025, 12:15pm

Sorry to hijack, but what services are you running and how is the performance with them? I’m running ultra basic right now, but debating adding

phinn · March 31, 2025, 3:41pm

Performance is excellent. Using adblock, ksmbd and usb3 file sharing, hdd idle, irqbalance, hardware flow offloading, packet steering, 2.4GHz / 5GHz wifi with wpa3/ krack/ etc. all the usual other stuff.

trevormcooper · March 31, 2025, 3:46pm

are you using the full adblock package or adblock fast? i currently have adblock fast with just the oisd big package, seems to do a decent job

phinn · March 31, 2025, 5:24pm

I've used both extensively they are similar since adblock fast has gotten a bit larger. Currently running luci-app-adblock-fast yea.