Zyxel NR7101 kern.warn / OOM issue

Installing and Using OpenWrt
1

After many months of working just fine my Zyxel NR7101 went into a bad state. Seems like a memory leak? Can the source be identified from the log below?

Here are some excerpts:

Fri May  2 23:47:53 2025 daemon.notice [2656]: <msg> [modem0/bearer48] verbose call end reason (6,36): [3gpp] regular-deactivation
Fri May  2 23:47:54 2025 daemon.notice [2656]: <msg> [modem0] state changed (connected -> registered)
Fri May  2 23:47:58 2025 daemon.notice [2656]: <msg> [modem0/bearer48] connection #1 finished: duration 172806s, tx: 2301955147 bytes, rx: 18308280628 bytes
Fri May  2 23:48:04 2025 daemon.warn [2656]: <wrn> [modem0/bearer48] reloading stats failed: Couldn't get packet statistics: QMI protocol error (15): 'OutOfCall'
Fri May  2 23:48:04 2025 daemon.warn [2656]: <wrn> [connection status report dispatcher] forcing exit on connection status report operation
Fri May  2 23:48:06 2025 daemon.warn [2656]: <wrn> [connection status report dispatcher] Cannot run connection status report operation from /usr/lib/ModemManager/connection.d/10-report-down: connection status report operation aborted with signal 9
Fri May  2 23:48:06 2025 daemon.warn [2656]: <wrn> [modem0/bearer48] errors detected in dispatcher: Failed 1 connection status report operations
Sat May  3 01:12:40 2025 daemon.notice hostapd: nl80211: nl80211_recv_beacons->nl_recvmsgs failed: -5
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.792426] procd invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.814139] CPU: 2 PID: 1 Comm: procd Tainted: G           O       6.6.73 #0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.828573] Stack : 00000000 00000001 809ce168 80089f7c 00000000 00000000 00000000 00000000
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.845622]         00000000 00000000 00000000 00000000 00000000 00000001 81837b88 818a6400
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.862667]         81837c20 00000000 00000000 81837ac8 00000038 80822a84 ffffffea 00000000
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.879726]         81837ad4 000001d1 8094b060 ffffffff 80841920 81837b68 82851f40 00000840
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.896781]         00000000 00140cca 809ce168 80943440 00000001 80496db4 00000001 000f96af
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.913814]         ...
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.919048] Call Trace:
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.919060] [<80089f7c>] 0x80089f7c
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.931576] [<80822a84>] 0x80822a84
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.938981] [<80496db4>] 0x80496db4
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.946288] [<800073b8>] 0x800073b8
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.953620] [<800073c0>] 0x800073c0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.960962] [<807f8c54>] 0x807f8c54
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.968287] [<80176460>] 0x80176460
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.975690] [<80176d78>] 0x80176d78
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.983098] [<801770f0>] 0x801770f0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.990429] [<80177310>] 0x80177310
Sat May  3 04:29:54 2025 kern.warn kernel: [7139625.997751] [<801d0028>] 0x801d0028
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.005242] [<80172944>] 0x80172944
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.012977] [<801731d8>] 0x801731d8
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.020299] [<8017070c>] 0x8017070c
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.027673] [<80829d20>] 0x80829d20
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.035175] [<801ae474>] 0x801ae474
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.042611] [<801b4ad4>] 0x801b4ad4
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.050150] [<800166c8>] 0x800166c8
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.057596] [<8001b9a8>] 0x8001b9a8
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.065014]
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.069394] Mem-Info:
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343] active_anon:1308 inactive_anon:1116 isolated_anon:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  active_file:353 inactive_file:13 isolated_file:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  unevictable:0 dirty:0 writeback:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  slab_reclaimable:689 slab_unreclaimable:10560
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  mapped:93 shmem:138 pagetables:143
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  sec_pagetables:0 bounce:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  kernel_misc_reclaimable:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.074343]  free:5420 free_pcp:56 free_cma:0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.152635] Node 0 active_anon:5232kB inactive_anon:4464kB active_file:2252kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:380kB dirty:0kB writeback:0kB shmem:552kB writeback_tmp:0kB kernel_stack:968kB pagetables:572kB sec_pagetables:0kB all_unreclaimable? yes
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.205072] Normal free:20600kB boost:0kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:5232kB inactive_anon:4464kB active_file:2464kB inactive_file:500kB unevictable:0kB writepending:0kB present:262144kB managed:249100kB mlocked:0kB bounce:0kB free_pcp:232kB local_pcp:0kB free_cma:0kB
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.261205] lowmem_reserve[]: 0 0
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.268207] Normal: 322*4kB (UM) 212*8kB (M) 73*16kB (UM) 70*32kB (M) 30*64kB (UM) 12*128kB (UM) 4*256kB (M) 5*512kB (UM) 1*1024kB (M) 3*2048kB (UM) 0*4096kB 0*8192kB = 20600kB
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.300138] 779 total pagecache pages
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.307795] 0 pages in swap cache
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.314776] Free swap  = 0kB
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.320987] Total swap = 0kB
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.327086] 65536 pages RAM
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.333043] 0 pages HighMem/MovableOnly
Sat May  3 04:29:54 2025 kern.warn kernel: [7139626.341187] 3261 pages reserved
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.348163] Tasks state (memory values in pages):
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.358054] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.376009] [    748]    81   748      412       32    16384        0             0 ubusd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.392750] [    749]     0   749      272       32    20480        0             0 askfirst
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.409958] [    784]     0   784      310       32    20480        0             0 urngd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.426863] [   1641]   514  1641      572       32    20480        0             0 logd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.443469] [   1695]     0  1695      982      192    24576        0             0 rpcd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.460189] [   1749]     0  1749     1187      606    24576        0             0 bridger
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.477354] [   2032]     0  2032      326       32    16384        0             0 dropbear
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.495496] [   2138]     0  2138      735       32    20480        0             0 hostapd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.513002] [   2139]     0  2139      735       32    16384        0             0 wpa_supplicant
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.531298] [   2171]   101  2171     1292       64    20480        0             0 wpa_supplicant
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.549602] [   2172]   101  2172     1411      148    24576        0             0 hostapd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.566674] [   2203]     0  2203      702       64    24576        0             0 netifd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.583550] [   2373]     0  2373      449       29    20480        0             0 odhcpd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.600428] [   2437]     0  2437      354        0    20480        0             0 crond
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.617142] [   2519]     0  2519      510      123    12288        0             0 uhttpd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.634045] [   2568]     0  2568      494       32    20480        0             0 dbus-daemon
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.651820] [   2614]     0  2614      397       64    20480        0             0 ModemManager-wr
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.670980] [   2615]     0  2615      403       96    20480        0             0 ModemManager-mo
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.689523] [   2656]     0  2656     3167      384    32768        0             0 ModemManager
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.707507] [   2808]     0  2808     1581       64    24576        0             0 mmcli
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.724349] [   2809]     0  2809      404      119    20480        0             0 ModemManager-mo
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.742816] [   2981]     0  2981      337        0    16384        0             0 irqbalance
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.760427] [   3612]     0  3612      735       50    16384        0             0 ntpd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.777341] [   3654]   123  3654      350        0    16384        0             0 ntpd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.793978] [   3859]     0  3859      274       32    20480        0             0 ubihealthd
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.811597] [   4211]     0  4211     2310       64    20480        0             0 qmi-proxy
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.829128] [   5067]     0  5067      735       53    16384        0             0 dnsmasq
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.846350] [   5079]   453  5079      462       32    16384        0             0 dnsmasq
Sat May  3 04:29:54 2025 kern.info kernel: [7139626.863580] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=bridger,pid=1749,uid=0
Sat May  3 04:29:54 2025 kern.err kernel: [7139626.889408] Out of memory: Killed process 1749 (bridger) total-vm:4748kB, anon-rss:2424kB, file-rss:256kB, shmem-rss:0kB, UID:0 pgtables:24kB oom_score_adj:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.347771] ModemManager invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.370101] CPU: 1 PID: 2656 Comm: ModemManager Tainted: G           O       6.6.73 #0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.386260] Stack : 00000000 00000a60 809ce168 80089f7c 00000000 00000000 00000000 00000000
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.403326]         00000000 00000000 00000000 00000000 00000000 00000001 83dc3b88 818a1f40
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.420382]         83dc3c20 00000000 00000000 83dc3ac8 00000038 80822a84 ffffffea 00000000
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.437464]         83dc3ad4 0000021b 8094b060 ffffffff 80841920 83dc3b68 81c09900 00000840
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.454535]         00000000 00140cca 809ce168 80943440 00000001 80496db4 00000004 80b10004
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.471595]         ...
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.476829] Call Trace:
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.476845] [<80089f7c>] 0x80089f7c
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.489367] [<80822a84>] 0x80822a84
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.496765] [<80496db4>] 0x80496db4
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.504082] [<800073b8>] 0x800073b8
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.511374] [<800073c0>] 0x800073c0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.518663] [<807f8c54>] 0x807f8c54
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.525954] [<80176460>] 0x80176460
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.533357] [<80176d78>] 0x80176d78
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.540706] [<801770f0>] 0x801770f0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.548007] [<80177310>] 0x80177310
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.555303] [<801d0028>] 0x801d0028
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.562776] [<80172944>] 0x80172944
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.570315] [<801731d8>] 0x801731d8
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.577609] [<8017070c>] 0x8017070c
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.584996] [<801ae474>] 0x801ae474
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.592305] [<801b4ad4>] 0x801b4ad4
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.599641] [<80065c4c>] 0x80065c4c
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.607000] [<800166c8>] 0x800166c8
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.614448] [<8001b9a8>] 0x8001b9a8
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.621894]
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.625341] Mem-Info:
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257] active_anon:1020 inactive_anon:815 isolated_anon:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  active_file:489 inactive_file:19 isolated_file:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  unevictable:0 dirty:0 writeback:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  slab_reclaimable:683 slab_unreclaimable:10657
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  mapped:268 shmem:141 pagetables:142
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  sec_pagetables:0 bounce:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  kernel_misc_reclaimable:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.630257]  free:5365 free_pcp:99 free_cma:0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.708634] Node 0 active_anon:3712kB inactive_anon:3628kB active_file:1728kB inactive_file:776kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1260kB dirty:0kB writeback:0kB shmem:564kB writeback_tmp:0kB kernel_stack:976kB pagetables:568kB sec_pagetables:0kB all_unreclaimable? no
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.761204] Normal free:20140kB boost:0kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:4080kB inactive_anon:3260kB active_file:2044kB inactive_file:1140kB unevictable:0kB writepending:0kB present:262144kB managed:249100kB mlocked:0kB bounce:0kB free_pcp:420kB local_pcp:0kB free_cma:0kB
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.816955] lowmem_reserve[]: 0 0
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.824033] Normal: 1*4kB (M) 9*8kB (U) 74*16kB (UM) 82*32kB (UM) 47*64kB (UM) 15*128kB (UM) 9*256kB (UM) 3*512kB (M) 3*1024kB (UM) 2*2048kB (UM) 0*4096kB 0*8192kB = 19820kB
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.855524] 1141 total pagecache pages
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.863397] 0 pages in swap cache
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.870411] Free swap  = 0kB
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.876498] Total swap = 0kB
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.882642] 65536 pages RAM
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.888552] 0 pages HighMem/MovableOnly
Sat May  3 06:51:16 2025 kern.warn kernel: [7148108.896578] 3261 pages reserved
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.903231] Tasks state (memory values in pages):
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.913017] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.930612] [    748]    81   748      412       64    16384        0             0 ubusd
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.947488] [    749]     0   749      272       32    20480        0             0 askfirst
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.964864] [    784]     0   784      310       32    20480        0             0 urngd
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.981683] [   1641]   514  1641      572      160    20480        0             0 logd
Sat May  3 06:51:16 2025 kern.info kernel: [7148108.998370] [   1695]     0  1695      982      192    24576        0             0 rpcd
Sat May  3 06:51:16 2025 kern.info kernel: [7148109.015117] [   2032]     0  2032      326       32    16384        0             0 dropbear
Sat May  3 06:51:16 2025 kern.info kernel: [7148109.032462] [   2138]     0  2138      735       32    20480        0             0 hostapd
Sat May  3 06:51:16 2025 kern.info kernel: [7148109.049653] [   2139]     0  2139      735       32    16384        0             0 wpa_supplicant
Sat May  3 06:51:16 2025 kern.info kernel: [7148109.067972] [   2171]   101  2171     1292      128    20480        0             0 wpa_supplicant
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.086299] [   2172]   101  2172     1411      227    24576        0             0 hostapd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.103420] [   2203]     0  2203      702      224    24576        0             0 netifd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.120387] [   2373]     0  2373      449      125    20480        0             0 odhcpd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.137324] [   2437]     0  2437      354       32    20480        0             0 crond
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.154100] [   2519]     0  2519      510      123    12288        0             0 uhttpd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.171004] [   2568]     0  2568      494       64    20480        0             0 dbus-daemon
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.188775] [   2614]     0  2614      397       64    20480        0             0 ModemManager-wr
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.207245] [   2615]     0  2615      403       96    20480        0             0 ModemManager-mo
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.225779] [   2656]     0  2656     3167      384    32768        0             0 ModemManager
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.243756] [   2808]     0  2808     1581       96    24576        0             0 mmcli
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.260533] [   2809]     0  2809      404      119    20480        0             0 ModemManager-mo
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.279031] [   2981]     0  2981      337        0    16384        0             0 irqbalance
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.296632] [   3612]     0  3612      735       50    16384        0             0 ntpd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.313202] [   3654]   123  3654      350        0    16384        0             0 ntpd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.329806] [   3859]     0  3859      274       64    20480        0             0 ubihealthd
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.347398] [   4211]     0  4211     2310      224    20480        0             0 qmi-proxy
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.364848] [   5067]     0  5067      735       53    16384        0             0 dnsmasq
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.381961] [   5079]   453  5079      462       64    16384        0             0 dnsmasq
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.399071] [   5794]     0  5794      591       64    20480        0             0 bridger
Sat May  3 06:51:17 2025 kern.info kernel: [7148109.416192] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=ModemManager,pid=2656,uid=0
Sat May  3 06:51:17 2025 kern.err kernel: [7148109.442736] Out of memory: Killed process 2656 (ModemManager) total-vm:12668kB, anon-rss:1408kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:32kB oom_score_adj:0

Sat May  3 07:03:33 2025 kern.warn kernel: [7148844.990578] ------------[ cut here ]------------
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.000180] WARNING: CPU: 1 PID: 26759 at lib/list_debug.c:62 0x80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.013919] list_del corruption. prev->next should be 85af9708, but was 836c7e80. (prev=836c7e80)
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.031971] Modules linked in: pppoe ppp_async option nft_fib_inet nf_flow_table_inet usb_wwan qmi_wwan pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7603e(O) mt76(O) mac80211(O) cfg80211(O) usbserial usbnet slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc_ccitt compat(O) cls_flower cdc_wdm act_vlan cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact sha512_generic seqiv sha3_generic jitterentropy_rng drbg hmac geniv rng cmac leds_gpio xhci_plat_hcd xhci_pci xhci_mtk_hcd xhci_hcd gpio_button_hotplug(O) usbcore nls_base usb_common mii crc32c_generic
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.194728] CPU: 1 PID: 26759 Comm: kworker/1:0 Tainted: G           O       6.6.73 #0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.210945] Workqueue: events 0x83a830f8 [cfg80211@ccf7c4e8+0x46000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.224188] Stack : 00000000 00006887 80943440 80841920 00000000 00000000 00000000 00000000
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.241326]         00000000 00000000 00000000 00000000 00000000 00000001 8adbbb78 818a1f40
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.258503]         8adbbc10 00000000 00000000 8adbba50 00000038 80822a84 ffffffea 00000000
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.275717]         8adbba5c 000002ba 8094b060 ffffffff 80841920 8adbbb58 8adbbc80 80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.292860]         00000009 808605f8 80943440 8100fc00 00000001 80496db4 00000004 80b10004
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.310083]         ...
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.315383] Call Trace:
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.315447] [<80822a84>] 0x80822a84
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.328042] [<80388fcc>] 0x80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.335390] [<80496db4>] 0x80496db4
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.342741] [<800073b8>] 0x800073b8
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.350109] [<800073c0>] 0x800073c0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.357450] [<80388fcc>] 0x80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.364767] [<807f8c54>] 0x807f8c54
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.372097] [<8002d9b0>] 0x8002d9b0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.379407] [<80388fcc>] 0x80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.386709] [<8002db64>] 0x8002db64
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.394154] [<80388fcc>] 0x80388fcc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.401877] [<83994d64>] 0x83994d64 [mt76@db298452+0xd000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.413594] [<83b200c0>] 0x83b200c0 [mac80211@0871516b+0x8a000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.425910] [<800a6010>] 0x800a6010
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.433288] [<83b20240>] 0x83b20240 [mac80211@0871516b+0x8a000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.445589] [<805b7784>] 0x805b7784
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.453047] [<8005e4b0>] 0x8005e4b0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.460455] [<805b78ac>] 0x805b78ac
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.467856] [<800600d4>] 0x800600d4
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.475266] [<805b79f8>] 0x805b79f8
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.482637] [<8005e8d0>] 0x8005e8d0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.489944] [<83a8305c>] 0x83a8305c [cfg80211@ccf7c4e8+0x46000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.502230] [<83a83114>] 0x83a83114 [cfg80211@ccf7c4e8+0x46000]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.514431] [<80048920>] 0x80048920
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.521736] [<80048db8>] 0x80048db8
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.529020] [<800b24b0>] 0x800b24b0
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.536409] [<800498fc>] 0x800498fc
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.543757] [<80053698>] 0x80053698
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.551097] [<80824940>] 0x80824940
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.558433] [<80049560>] 0x80049560
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.565768] [<80053698>] 0x80053698
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.573086] [<8005379c>] 0x8005379c
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.580400] [<80053698>] 0x80053698
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.587712] [<80053698>] 0x80053698
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.595013] [<80002a58>] 0x80002a58
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.602357]
Sat May  3 07:03:33 2025 kern.warn kernel: [7148845.605771] ---[ end trace 0000000000000000 ]---

image
image1171×1209 145 KB

Larger excerpt here:

2

Something is consuming almost all available memory but the logs don't seem to say what. Might be the kernel? Anyway, this is the first thing to figure out.

1 Like
3

Thanks @antonk. This was a freak event that occurred on my router device that has our 4G modem. The only slightly unusual thing installed on that is ModemManager. I had to reboot the router to restore internet connectivity, but I did retain a copy of logread and that screenshot from top. Anything I can do at this point?

Is this relevant from dmesg:

[    3.923850] Freeing unused kernel image (initmem) memory: 1284K
[    3.935719] This architecture does not have kernel memory protection.
4

Judging by this line:

the user-space part of modem-manager (not sure if it has a kernel-space part) was consuming ~12MiB total, so not really a big deal, compared to total memory (256MiB). And apparently the machine still had almost no available memory after modem-manager was killed. So I think the kernel simply killed the user-space application which was consuming the most memory, but it was not the culprit. Hence I'm wondering whether the culprit was in the kernel.

After a machine reboot, I don't think I can suggest any way to investigate this further, except perhaps monitoring the RAM usage and seeing whether it's increasing.

1 Like
5

Yes makes sense. Thanks for looking at this.

And I suppose the next question is: what sort of thing might cause the kernel to leak memory (assuming that this relates to a memory leak) or otherwise cause a catastrophic out of memory situation like this?

@slh perhaps this is something that you might be familiar with?

1 Like
6

I don't think the kernel is leaking memory, but one of the drivers could.

7

Does this point to the mt76 2.4GHz chip driver?

Or is it just that this call gave rise to the out of memory killer getting invoked?

8

I don't know how to interpret call traces. Hopefully someone smarter will answer this question.