I have the qsdk partition layout and I can flash own build qsdk and also official xiaomi firmware, but I would prefer to go back to official layout?
You can see the difference in the wiki ( https://openwrt.org/inbox/toh/xiaomi/xiaomi_ax3600 ) or just use panko of mtd12 is different size then mtd13
For ipq806x, the nss-crypto is pretty much standalone. Once it gets loaded, I can use its internal benchmark tool to check if the crypto core is running.
nss-cfi just provide kernel crypto integration for higher level kernel drivers to use. nss-cfi needs nss-crypto.
Ok, that was my logic as well.
I can load the benchmark kmod, but as soon as start command is issued it will cause a NSS core1 coredump and then kernel panic and reboot.
I will try bumping the crypto log level
1 Like
I remember pulling my hairs out when toying with the crypto drivers, as I was really hoping to accelerate OpenVPN. Alas …
Anyway, my issue turns out to be compatibility between driver and firmware versions. Once I restart and ensure QSDK and firmware version matches, everything started working.
I feel so incredibly stupid right now. Had to to the flag_ota_reboot to get the international fw running and stuff so my brain automatically did the same for openwrt. Thanks!
I did like that too.
Yeah, we got burned by the mismatched FW version as well earlier.
However, I tried using 11.4 FW as well and the end result is the same.
BTW, my AX9000 finally arrived, with packaging its like 3.4kg.
Box is huge compared to the AX3600, much more fancy.
Its gonna be a fun weekend trying to get UART TX working.
Horrible adapters as well, these are completely useless as the pins are completely bent.
2 Likes
That one AX9000 is on my watch list... Curious to know how much it can handle.
During the process of soldering the pins for the UART access I noticed that the quality of the thermal pads is quite improvable, so I took the opportunity to change them.
Before:
The old ones fall apart when touched, the residue must be cleaned with isopropanol:
New ones:
I use those pads:
2 Likes
Thanks, did they mark the pins on the silkscreen like on AX3600?
Not really, only the right pin has some mark.
I used an oscilloscope to find the RX
Ok, then we will just document it in the wiki.
And i already had a look if we can use the same nvram inject method like the ax3600.
But "set_config_iotdev" is missing. But there are a lot of other lua api methods.
But this is really not my area.
EDIT
set_config_iotdev is still there in misystem.lua
cfg_backup is encrypted (of cause)
/lua/xiaoqiang/module/XQBackup.lua seems to handle this, but is was only able to partially decrypt the lua. (with luadec)
There are two additional screws (to the other ones under the rubber feet) under the device label, lift the top carefully, there is a small cable/plug from the leds to the board.
Massive OOM, after just 24 hours of uptime (in case you want to take a look):
[102645.375788] netifd invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
[102645.375833] CPU: 1 PID: 1863 Comm: netifd Not tainted 5.10.42 #0
[102645.383290] Hardware name: Xiaomi AX3600 (DT)
[102645.389278] Call trace:
[102645.393624] dump_backtrace+0x0/0x1b0
[102645.396310] show_stack+0x18/0x30
[102645.399869] dump_stack+0xdc/0x11c
[102645.403253] dump_header+0x44/0x194
[102645.406642] oom_kill_process+0x1d0/0x1d8
[102645.410459] out_of_memory+0x20c/0x320
[102645.414366] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[102645.418101] __alloc_pages_nodemask+0x1f4/0x238
[102645.424001] __get_free_pages+0x1c/0x68
[102645.428775] proc_pid_readlink+0x74/0x2b8
[102645.432683] vfs_readlink+0x11c/0x138
[102645.436589] do_readlinkat+0xac/0x128
[102645.440318] __arm64_sys_readlinkat+0x20/0x30
[102645.444056] el0_svc_common.constprop.3+0xac/0x170
[102645.448482] do_el0_svc+0x70/0x90
[102645.453255] el0_svc+0x18/0x28
[102645.456726] el0_sync_handler+0x90/0xb8
[102645.459764] el0_sync+0x184/0x1c0
[102645.464040] Mem-Info:
[102645.467148] active_anon:69 inactive_anon:677 isolated_anon:0
[102645.467148] active_file:15 inactive_file:76 isolated_file:0
[102645.467148] unevictable:0 dirty:0 writeback:0
[102645.467148] slab_reclaimable:1878 slab_unreclaimable:11436
[102645.467148] mapped:15 shmem:70 pagetables:138 bounce:0
[102645.467148] free:4801 free_pcp:194 free_cma:0
[102645.478778] Node 0 active_anon:276kB inactive_anon:2708kB active_file:60kB inactive_file:304kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:60kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1392kB all_unreclaimable? yes
[102645.501543] DMA free:19204kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:276kB inactive_anon:2708kB active_file:188kB inactive_file:432kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:552kB bounce:0kB free_pcp:688kB local_pcp:0kB free_cma:0kB
[102645.529551] lowmem_reserve[]: 0 0 0 0
[102645.551798] DMA: 452*4kB (UME) 207*8kB (UME) 124*16kB (UME) 56*32kB (UME) 24*64kB (UME) 9*128kB (UME) 4*256kB (M) 6*512kB (UM) 2*1024kB (UM) 1*2048kB (U) 0*4096kB = 18120kB
[102645.555366] 300 total pagecache pages
[102645.570885] 0 pages in swap cache
[102645.574526] Swap cache stats: add 0, delete 0, find 0/0
[102645.577912] Free swap = 0kB
[102645.583382] Total swap = 0kB
[102645.586237] 131072 pages RAM
[102645.589213] 0 pages HighMem/MovableOnly
[102645.592145] 37102 pages reserved
[102645.596137] Tasks state (memory values in pages):
[102645.599351] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[102645.604088] [ 842] 81 842 304 28 36864 0 0 ubusd
[102645.612740] [ 843] 0 843 205 9 28672 0 0 askfirst
[102645.620996] [ 877] 0 877 237 11 24576 0 0 urngd
[102645.629662] [ 1326] 514 1326 296 38 32768 0 0 logd
[102645.637754] [ 1397] 0 1397 566 84 32768 0 0 rpcd
[102645.646000] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[102645.654155] [ 1802] 0 1802 943 52 45056 0 0 hostapd
[102645.662728] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[102645.671185] [ 1863] 0 1863 452 54 32768 0 0 netifd
[102645.679985] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[102645.688255] [ 2037] 0 2037 850 73 40960 0 0 uhttpd
[102645.696627] [ 2585] 0 2585 306 15 28672 0 0 ntpd
[102645.704921] [ 3075] 453 3075 346 34 32768 0 0 dnsmasq
[102645.713291] [ 3327] 0 3327 319 117 32768 0 0 pppd
[102645.721762] [ 3442] 0 3442 393 103 32768 0 0 sh
[102645.729870] [ 19354] 0 19354 305 10 28672 0 0 sleep
[102645.737648] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=dnsmasq,pid=3075,uid=453
[102645.746036] Out of memory: Killed process 3075 (dnsmasq) total-vm:1384kB, anon-rss:132kB, file-rss:4kB, shmem-rss:0kB, UID:453 pgtables:32kB oom_score_adj:0
[102645.758871] oom_reaper: reaped process 3075 (dnsmasq), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[102771.797710] hostapd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[102771.797755] CPU: 2 PID: 1802 Comm: hostapd Not tainted 5.10.42 #0
[102771.806254] Hardware name: Xiaomi AX3600 (DT)
[102771.812415] Call trace:
[102771.816848] dump_backtrace+0x0/0x1b0
[102771.819533] show_stack+0x18/0x30
[102771.823093] dump_stack+0xdc/0x11c
[102771.826476] dump_header+0x44/0x194
[102771.829865] oom_kill_process+0x1d0/0x1d8
[102771.833682] out_of_memory+0x20c/0x320
[102771.837588] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[102771.841340] __alloc_pages_nodemask+0x1f4/0x238
[102771.847227] pagecache_get_page.part.71+0xf8/0x2b8
[102771.852000] filemap_fault+0x774/0x978
[102771.856600] __do_fault+0x3c/0xf8
[102771.860418] handle_mm_fault+0x6b4/0xc38
[102771.863890] do_page_fault+0x1ac/0x3d8
[102771.867968] do_translation_fault+0x50/0x60
[102771.871616] do_mem_abort+0x40/0xa0
[102771.876132] el0_ia+0x64/0xd0
[102771.879690] el0_sync_handler+0x78/0xb8
[102771.882554] el0_sync+0x184/0x1c0
[102771.886749] Mem-Info:
[102771.889861] active_anon:70 inactive_anon:671 isolated_anon:0
[102771.889861] active_file:20 inactive_file:58 isolated_file:0
[102771.889861] unevictable:0 dirty:0 writeback:0
[102771.889861] slab_reclaimable:1874 slab_unreclaimable:11442
[102771.889861] mapped:43 shmem:70 pagetables:138 bounce:0
[102771.889861] free:4076 free_pcp:5 free_cma:0
[102771.901133] Node 0 active_anon:280kB inactive_anon:2684kB active_file:132kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:20kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1456kB all_unreclaimable? no
[102771.924067] DMA free:16816kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:280kB inactive_anon:2684kB active_file:220kB inactive_file:508kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:552kB bounce:0kB free_pcp:148kB local_pcp:20kB free_cma:0kB
[102771.951994] lowmem_reserve[]: 0 0 0 0
[102771.974226] DMA: 656*4kB (UME) 242*8kB (UME) 127*16kB (UME) 69*32kB (UME) 28*64kB (UME) 9*128kB (ME) 5*256kB (M) 5*512kB (M) 2*1024kB (UM) 0*2048kB 0*4096kB = 17632kB
[102771.977893] 139 total pagecache pages
[102771.992715] 0 pages in swap cache
[102771.996533] Swap cache stats: add 0, delete 0, find 0/0
[102771.999922] Free swap = 0kB
[102772.005385] Total swap = 0kB
[102772.008251] 131072 pages RAM
[102772.011195] 0 pages HighMem/MovableOnly
[102772.014156] 37102 pages reserved
[102772.018147] Tasks state (memory values in pages):
[102772.021361] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[102772.026079] [ 842] 81 842 304 28 36864 0 0 ubusd
[102772.034656] [ 843] 0 843 205 9 28672 0 0 askfirst
[102772.042988] [ 877] 0 877 237 11 24576 0 0 urngd
[102772.051584] [ 1326] 514 1326 296 38 32768 0 0 logd
[102772.059742] [ 1397] 0 1397 566 84 32768 0 0 rpcd
[102772.067989] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[102772.076150] [ 1802] 0 1802 943 52 45056 0 0 hostapd
[102772.084656] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[102772.093163] [ 1863] 0 1863 452 54 32768 0 0 netifd
[102772.101930] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[102772.110261] [ 2037] 0 2037 850 73 40960 0 0 uhttpd
[102772.118597] [ 2585] 0 2585 306 15 28672 0 0 ntpd
[102772.126929] [ 3327] 0 3327 319 37 32768 0 0 pppd
[102772.135262] [ 3442] 0 3442 393 103 32768 0 0 sh
[102772.143424] [ 19354] 0 19354 305 10 28672 0 0 sleep
[102772.151238] [ 19363] 453 19363 341 28 32768 0 0 dnsmasq
[102772.159569] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=sh,pid=3442,uid=0
[102772.168187] Out of memory: Killed process 3442 (sh) total-vm:1572kB, anon-rss:408kB, file-rss:4kB, shmem-rss:0kB, UID:0 pgtables:32kB oom_score_adj:0
[102772.180271] oom_reaper: reaped process 3442 (sh), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[102784.049579] netifd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[102784.049628] CPU: 3 PID: 1863 Comm: netifd Not tainted 5.10.42 #0
[102784.058140] Hardware name: Xiaomi AX3600 (DT)
[102784.064199] Call trace:
[102784.068544] dump_backtrace+0x0/0x1b0
[102784.071230] show_stack+0x18/0x30
[102784.074790] dump_stack+0xdc/0x11c
[102784.078173] dump_header+0x44/0x194
[102784.081563] oom_kill_process+0x1d0/0x1d8
[102784.085380] out_of_memory+0x20c/0x320
[102784.089287] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[102784.093021] __alloc_pages_nodemask+0x1f4/0x238
[102784.098924] pagecache_get_page.part.71+0xf8/0x2b8
[102784.103697] filemap_fault+0x774/0x978
[102784.108299] __do_fault+0x3c/0xf8
[102784.112115] handle_mm_fault+0x6b4/0xc38
[102784.115587] do_page_fault+0x1ac/0x3d8
[102784.119665] do_translation_fault+0x50/0x60
[102784.123314] do_mem_abort+0x40/0xa0
[102784.127830] el0_ia+0x64/0xd0
[102784.131408] el0_sync_handler+0x78/0xb8
[102784.134252] el0_sync+0x184/0x1c0
[102784.138480] Mem-Info:
[102784.141575] active_anon:68 inactive_anon:558 isolated_anon:0
[102784.141575] active_file:14 inactive_file:93 isolated_file:0
[102784.141575] unevictable:0 dirty:0 writeback:0
[102784.141575] slab_reclaimable:1873 slab_unreclaimable:11441
[102784.141575] mapped:41 shmem:70 pagetables:123 bounce:0
[102784.141575] free:4080 free_pcp:1 free_cma:0
[102784.152834] Node 0 active_anon:272kB inactive_anon:2232kB active_file:48kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:12kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1472kB all_unreclaimable? yes
[102784.175806] DMA free:16564kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:272kB inactive_anon:2232kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:492kB bounce:0kB free_pcp:908kB local_pcp:0kB free_cma:0kB
[102784.203263] lowmem_reserve[]: 0 0 0 0
[102784.225537] DMA: 359*4kB (UME) 251*8kB (UME) 132*16kB (UME) 75*32kB (UME) 27*64kB (UME) 10*128kB (ME) 7*256kB (UM) 6*512kB (UM) 1*1024kB (M) 0*2048kB 0*4096kB = 16852kB
[102784.229090] 175 total pagecache pages
[102784.244240] 0 pages in swap cache
[102784.247882] Swap cache stats: add 0, delete 0, find 0/0
[102784.251265] Free swap = 0kB
[102784.256737] Total swap = 0kB
[102784.259593] 131072 pages RAM
[102784.262552] 0 pages HighMem/MovableOnly
[102784.265505] 37102 pages reserved
[102784.269511] Tasks state (memory values in pages):
[102784.272710] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[102784.277429] [ 842] 81 842 304 28 36864 0 0 ubusd
[102784.286027] [ 843] 0 843 205 9 28672 0 0 askfirst
[102784.294340] [ 877] 0 877 237 11 24576 0 0 urngd
[102784.302934] [ 1326] 514 1326 296 42 32768 0 0 logd
[102784.311118] [ 1397] 0 1397 566 84 32768 0 0 rpcd
[102784.319339] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[102784.327501] [ 1802] 0 1802 943 52 45056 0 0 hostapd
[102784.336007] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[102784.344515] [ 1863] 0 1863 452 54 32768 0 0 netifd
[102784.353303] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[102784.361719] [ 2037] 0 2037 850 73 40960 0 0 uhttpd
[102784.369955] [ 2585] 0 2585 306 15 28672 0 0 ntpd
[102784.378366] [ 3327] 0 3327 319 37 32768 0 0 pppd
[102784.386627] [ 19363] 453 19363 341 28 32768 0 0 dnsmasq
[102784.394871] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=rpcd,pid=1397,uid=0
[102784.403343] Out of memory: Killed process 1397 (rpcd) total-vm:2264kB, anon-rss:332kB, file-rss:4kB, shmem-rss:0kB, UID:0 pgtables:32kB oom_score_adj:0
[102784.415872] oom_reaper: reaped process 1397 (rpcd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[102827.829785] pppd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[102827.829828] CPU: 3 PID: 3327 Comm: pppd Not tainted 5.10.42 #0
[102827.837981] Hardware name: Xiaomi AX3600 (DT)
[102827.843882] Call trace:
[102827.848401] dump_backtrace+0x0/0x1b0
[102827.851086] show_stack+0x18/0x30
[102827.854646] dump_stack+0xdc/0x11c
[102827.858029] dump_header+0x44/0x194
[102827.861418] oom_kill_process+0x1d0/0x1d8
[102827.865235] out_of_memory+0x20c/0x320
[102827.869143] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[102827.872876] __alloc_pages_nodemask+0x1f4/0x238
[102827.878779] pagecache_get_page.part.71+0xf8/0x2b8
[102827.883553] filemap_fault+0x774/0x978
[102827.888154] __do_fault+0x3c/0xf8
[102827.891971] handle_mm_fault+0x6b4/0xc38
[102827.895444] do_page_fault+0x1ac/0x3d8
[102827.899522] do_translation_fault+0x50/0x60
[102827.903170] do_mem_abort+0x40/0xa0
[102827.907687] el0_ia+0x64/0xd0
[102827.911242] el0_sync_handler+0x78/0xb8
[102827.914108] el0_sync+0x184/0x1c0
[102827.918324] Mem-Info:
[102827.921428] active_anon:70 inactive_anon:535 isolated_anon:0
[102827.921428] active_file:23 inactive_file:59 isolated_file:0
[102827.921428] unevictable:0 dirty:0 writeback:0
[102827.921428] slab_reclaimable:1872 slab_unreclaimable:11441
[102827.921428] mapped:0 shmem:70 pagetables:129 bounce:0
[102827.921428] free:4104 free_pcp:261 free_cma:0
[102827.933068] Node 0 active_anon:280kB inactive_anon:2140kB active_file:92kB inactive_file:564kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:40kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1488kB all_unreclaimable? yes
[102827.955697] DMA free:16956kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:280kB inactive_anon:2140kB active_file:0kB inactive_file:40kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:516kB bounce:0kB free_pcp:208kB local_pcp:120kB free_cma:0kB
[102827.983395] lowmem_reserve[]: 0 0 0 0
[102828.005626] DMA: 334*4kB (UME) 229*8kB (UME) 138*16kB (UME) 73*32kB (UME) 29*64kB (ME) 11*128kB (UME) 7*256kB (UM) 6*512kB (UM) 1*1024kB (M) 0*2048kB 0*4096kB = 16864kB
[102828.009447] 304 total pagecache pages
[102828.024618] 0 pages in swap cache
[102828.028274] Swap cache stats: add 0, delete 0, find 0/0
[102828.031641] Free swap = 0kB
[102828.037112] Total swap = 0kB
[102828.039977] 131072 pages RAM
[102828.042922] 0 pages HighMem/MovableOnly
[102828.045882] 37102 pages reserved
[102828.049874] Tasks state (memory values in pages):
[102828.053089] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[102828.057808] [ 842] 81 842 304 28 36864 0 0 ubusd
[102828.066389] [ 843] 0 843 205 9 28672 0 0 askfirst
[102828.074765] [ 877] 0 877 237 11 24576 0 0 urngd
[102828.083343] [ 1326] 514 1326 296 38 32768 0 0 logd
[102828.091601] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[102828.099722] [ 1802] 0 1802 943 52 45056 0 0 hostapd
[102828.108320] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[102828.116736] [ 1863] 0 1863 452 54 32768 0 0 netifd
[102828.125600] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[102828.133840] [ 2037] 0 2037 850 73 40960 0 0 uhttpd
[102828.142310] [ 2585] 0 2585 306 14 28672 0 0 ntpd
[102828.150506] [ 3327] 0 3327 319 37 32768 0 0 pppd
[102828.158924] [ 19363] 453 19363 341 28 32768 0 0 dnsmasq
[102828.167001] [ 19368] 0 19368 532 48 32768 0 0 rpcd
[102828.175647] [ 19371] 0 19371 305 12 24576 0 0 sh
[102828.183576] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=uhttpd,pid=2037,uid=0
[102828.191408] Out of memory: Killed process 2037 (uhttpd) total-vm:3400kB, anon-rss:292kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:40kB oom_score_adj:0
[102828.204104] oom_reaper: reaped process 2037 (uhttpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[103370.343728] netifd invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
[103370.343773] CPU: 1 PID: 1863 Comm: netifd Not tainted 5.10.42 #0
[103370.351232] Hardware name: Xiaomi AX3600 (DT)
[103370.357219] Call trace:
[103370.361564] dump_backtrace+0x0/0x1b0
[103370.364251] show_stack+0x18/0x30
[103370.367811] dump_stack+0xdc/0x11c
[103370.371194] dump_header+0x44/0x194
[103370.374582] oom_kill_process+0x1d0/0x1d8
[103370.378400] out_of_memory+0x20c/0x320
[103370.382307] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[103370.386042] __alloc_pages_nodemask+0x1f4/0x238
[103370.391942] __get_free_pages+0x1c/0x68
[103370.396716] proc_pid_readlink+0x74/0x2b8
[103370.400623] vfs_readlink+0x11c/0x138
[103370.404529] do_readlinkat+0xac/0x128
[103370.408258] __arm64_sys_readlinkat+0x20/0x30
[103370.411997] el0_svc_common.constprop.3+0xac/0x170
[103370.416421] do_el0_svc+0x70/0x90
[103370.421197] el0_svc+0x18/0x28
[103370.424666] el0_sync_handler+0x90/0xb8
[103370.427706] el0_sync+0x184/0x1c0
[103370.432069] Mem-Info:
[103370.435104] active_anon:69 inactive_anon:492 isolated_anon:0
[103370.435104] active_file:0 inactive_file:21 isolated_file:0
[103370.435104] unevictable:0 dirty:0 writeback:0
[103370.435104] slab_reclaimable:1867 slab_unreclaimable:11472
[103370.435104] mapped:35 shmem:70 pagetables:123 bounce:0
[103370.435104] free:4920 free_pcp:0 free_cma:0
[103370.446371] Node 0 active_anon:276kB inactive_anon:1968kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:140kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1456kB all_unreclaimable? yes
[103370.469211] DMA free:19680kB min:16384kB low:20480kB high:24576kB reserved_highatomic:4096KB active_anon:276kB inactive_anon:1968kB active_file:308kB inactive_file:160kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:492kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[103370.497190] lowmem_reserve[]: 0 0 0 0
[103370.519386] DMA: 581*4kB (UMEH) 212*8kB (UMEH) 110*16kB (UMEH) 100*32kB (UMEH) 28*64kB (UMEH) 13*128kB (UME) 3*256kB (M) 4*512kB (UM) 1*1024kB (M) 0*2048kB 1*4096kB (M) = 20372kB
[103370.523046] 94 total pagecache pages
[103370.538910] 0 pages in swap cache
[103370.542726] Swap cache stats: add 0, delete 0, find 0/0
[103370.546027] Free swap = 0kB
[103370.551491] Total swap = 0kB
[103370.554356] 131072 pages RAM
[103370.557306] 0 pages HighMem/MovableOnly
[103370.560254] 37102 pages reserved
[103370.564253] Tasks state (memory values in pages):
[103370.567471] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[103370.572188] [ 842] 81 842 304 28 36864 0 0 ubusd
[103370.580764] [ 843] 0 843 205 9 28672 0 0 askfirst
[103370.589096] [ 877] 0 877 237 11 24576 0 0 urngd
[103370.597689] [ 1326] 514 1326 296 38 32768 0 0 logd
[103370.605849] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[103370.614099] [ 1802] 0 1802 943 52 45056 0 0 hostapd
[103370.622604] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[103370.631110] [ 1863] 0 1863 452 54 32768 0 0 netifd
[103370.639875] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[103370.648209] [ 2585] 0 2585 306 14 28672 0 0 ntpd
[103370.656541] [ 3327] 0 3327 319 37 32768 0 0 pppd
[103370.664880] [ 19363] 453 19363 341 28 32768 0 0 dnsmasq
[103370.673035] [ 19368] 0 19368 532 48 32768 0 0 rpcd
[103370.681546] [ 19381] 0 19381 827 36 40960 0 0 uhttpd
[103370.689620] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=hostapd,pid=1802,uid=0
[103370.697797] Out of memory: Killed process 1802 (hostapd) total-vm:3772kB, anon-rss:208kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:44kB oom_score_adj:0
[103370.710894] oom_reaper: reaped process 1802 (hostapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[103370.724642] netifd invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
[103370.733868] CPU: 2 PID: 1863 Comm: netifd Not tainted 5.10.42 #0
[103370.742120] Hardware name: Xiaomi AX3600 (DT)
[103370.748104] Call trace:
[103370.752451] dump_backtrace+0x0/0x1b0
[103370.755136] show_stack+0x18/0x30
[103370.758694] dump_stack+0xdc/0x11c
[103370.762079] dump_header+0x44/0x194
[103370.765482] oom_kill_process+0x1d0/0x1d8
[103370.769288] out_of_memory+0x20c/0x320
[103370.773192] __alloc_pages_slowpath.constprop.150+0x6c8/0x9b0
[103370.776927] __alloc_pages_nodemask+0x1f4/0x238
[103370.782828] __get_free_pages+0x1c/0x68
[103370.787602] proc_pid_readlink+0x74/0x2b8
[103370.791509] vfs_readlink+0x11c/0x138
[103370.795415] do_readlinkat+0xac/0x128
[103370.799146] __arm64_sys_readlinkat+0x20/0x30
[103370.802883] el0_svc_common.constprop.3+0xac/0x170
[103370.807309] do_el0_svc+0x70/0x90
[103370.812084] el0_svc+0x18/0x28
[103370.815552] el0_sync_handler+0x90/0xb8
[103370.818590] el0_sync+0x184/0x1c0
[103370.822796] Mem-Info:
[103370.825981] active_anon:69 inactive_anon:454 isolated_anon:0
[103370.825981] active_file:16 inactive_file:21 isolated_file:0
[103370.825981] unevictable:0 dirty:0 writeback:0
[103370.825981] slab_reclaimable:1867 slab_unreclaimable:11472
[103370.825981] mapped:35 shmem:70 pagetables:123 bounce:0
[103370.825981] free:5051 free_pcp:74 free_cma:0
[103370.837614] Node 0 active_anon:276kB inactive_anon:1816kB active_file:0kB inactive_file:272kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:140kB dirty:0kB writeback:0kB shmem:280kB writeback_tmp:0kB kernel_stack:1488kB all_unreclaimable? yes
[103370.860350] DMA free:19692kB min:16384kB low:20480kB high:24576kB reserved_highatomic:0KB active_anon:276kB inactive_anon:1816kB active_file:460kB inactive_file:0kB unevictable:0kB writepending:0kB present:524288kB managed:375880kB mlocked:0kB pagetables:492kB bounce:0kB free_pcp:580kB local_pcp:0kB free_cma:0kB
[103370.887959] lowmem_reserve[]: 0 0 0 0
[103370.910182] DMA: 335*4kB (UME) 152*8kB (UME) 115*16kB (UME) 96*32kB (UME) 29*64kB (UME) 13*128kB (UME) 3*256kB (M) 4*512kB (UM) 1*1024kB (M) 0*2048kB 1*4096kB (M) = 18924kB
[103370.913949] 151 total pagecache pages
[103370.929465] 0 pages in swap cache
[103370.933083] Swap cache stats: add 0, delete 0, find 0/0
[103370.936477] Free swap = 0kB
[103370.941940] Total swap = 0kB
[103370.944805] 131072 pages RAM
[103370.947757] 0 pages HighMem/MovableOnly
[103370.950709] 37102 pages reserved
[103370.954703] Tasks state (memory values in pages):
[103370.957916] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[103370.962636] [ 842] 81 842 304 71 36864 0 0 ubusd
[103370.971234] [ 843] 0 843 205 9 28672 0 0 askfirst
[103370.979623] [ 877] 0 877 237 11 24576 0 0 urngd
[103370.988141] [ 1326] 514 1326 296 42 32768 0 0 logd
[103370.996380] [ 1682] 0 1682 254 10 28672 0 0 dropbear
[103371.004555] [ 1803] 0 1803 922 31 36864 0 0 wpa_supplicant
[103371.013130] [ 1863] 0 1863 452 54 32768 0 0 netifd
[103371.021907] [ 1917] 0 1917 358 29 36864 0 0 odhcpd
[103371.030304] [ 2585] 0 2585 306 14 28672 0 0 ntpd
[103371.038576] [ 3327] 0 3327 319 37 32768 0 0 pppd
[103371.046912] [ 19363] 453 19363 341 28 32768 0 0 dnsmasq
[103371.055130] [ 19368] 0 19368 532 48 32768 0 0 rpcd
[103371.063574] [ 19381] 0 19381 827 36 40960 0 0 uhttpd
[103371.071739] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=netifd,pid=1863,uid=0
[103371.079832] Out of memory: Killed process 1863 (netifd) total-vm:1808kB, anon-rss:216kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:32kB oom_score_adj:0
[103371.092648] oom_reaper: reaped process 1863 (netifd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
@kirdes Thanks, I hope to have some time even later today to finally open the box.
Damn, this thing is like 2+ kgs on its own.
@dchard Yeah, we are aware, unfortunately ath11k is really complicated
Thought maybe you can dig something out of the log that helps 
Thanks for sharing, I will check it in detail later but it's just OOM panic.
And it will print only the userspace usage per app so that's not useful.
I mean, we narrowed the leak to ath11k WMI buffer which never gets freed.
Now it's a matter of spending a lot of man-hours trying to figure out why is the callback not being triggered to clear the buffer.
Unfortunately since we do this after work there is never enough time.
4 Likes