Adding OpenWrt support for Xiaomi AX3600 (Part 1)

For Developers
2966

Haha, not its at 152MB.
iperf3 is still horrible, but like how when speedtesting to upstream gets me full uplink speed?

2967

in my case iperf from one device to another 174 mb

but if the router does the iperf server, 400+ mb LOL

2968

LOL, with UDP iperf3 I get like 1Mbitps.
Wait what?
With parallel streams each one gets like 45-50Mbits and they sum at 200Mbits?
But why iperf3, why?

So, with many streams the WLAN perfroms great actually, I get 400+Mbits on 40MHz AX.
That is really good, but why do I need parallel streams iperf?

2969

it honestly seems that there is a problem with transfer between device in the same network?

about parallel streams... yesterday i got 1gbps with 5 parallel stream... with a single one i got 600 max

2970

To me it looks like a wpa_supplicant/hostapd issue as with parallel streams it gets the full radio speed.
Anyway, it looks that RAM stabilized at around 170MB, weirdly it goes down with load.

2971

So I added both PCIe's and I am seeing:

# lspci -v
00:00.0 Class 0604: 17cb:0302
00:00.0 Class 0604: 17cb:1002

I tried to compile ath11k-pci but got an error about the .ko file being missing, need to see what's going on there and run a complete clean make over night I guess.

2972

You want to include the full pci-utils instead of the Busybox one.
But, I think that only the bridges are visible.

For ath11k, you are probably running an old tree.
Got to kernel modules->wireless drivers and enable the thermal option under ath11k

2973

I enabled the pciutils now, rebuilding again....

I have kmod-ath11k (including thermal sensor and throttling support) and kmod-ath11k-ahb enabled, those compile fine. Once I enable ath11k-pci additionally I start seeing that issue.

2974

let me guess, its missing mhi?

2975

The error I am getting:

SHELL= flock /home/flole/OpenWRTSXK80/tmp/.backports-5.10.34-1.tar.xz.flock -c '        /home/flole/OpenWRTSXK80/scripts/download.pl "/home/flole/OpenWRTSXK80/dl" "backports-5.10.34-1.tar.xz" "03c4ca6bf47d4e50b91b61bc2943a98c788439e56ce2b4080bc4c94141c2c15b" "" "@KERNEL/linux/kernel/projects/backports/stable/v5.10.34/"    '
mkdir -p /home/flole/OpenWRTSXK80/bin/targets/ipq807x/generic/packages /home/flole/OpenWRTSXK80/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/backports-5.10.34-1/ipkg-aarch64_cortex-a53/kmod-ath11k-pci/CONTROL /home/flole/OpenWRTSXK80/staging_dir/target-aarch64_cortex-a53_musl/pkginfo
ERROR: module '/home/flole/OpenWRTSXK80/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/backports-5.10.34-1/drivers/net/wireless/ath/ath11k/ath11k_pci.ko' is missing.
Makefile:555: recipe for target '/home/flole/OpenWRTSXK80/bin/targets/ipq807x/generic/packages/kmod-ath11k-pci_5.10.41+5.10.34-1-1_aarch64_cortex-a53.ipk' failed
make[3]: *** [/home/flole/OpenWRTSXK80/bin/targets/ipq807x/generic/packages/kmod-ath11k-pci_5.10.41+5.10.34-1-1_aarch64_cortex-a53.ipk] Error 1
make[3]: Leaving directory '/home/flole/OpenWRTSXK80/package/kernel/mac80211'
time: package/kernel/mac80211/compile#0.30#0.17#0.44
    ERROR: package/kernel/mac80211 failed to build.
package/Makefile:114: recipe for target 'package/kernel/mac80211/compile' failed
make[2]: *** [package/kernel/mac80211/compile] Error 1

Uhmmm:

0000:00:00.0 PCI bridge: Qualcomm Device 1002
0001:00:00.0 PCI bridge: Qualcomm MDM9x55 LTE Modem [Snapdragon X16]

No clue why it thinks that there is an LTE Modem in it, quite sure that it's not there.

2976

It's not being built as it has dependencies that are not satisfied, most likely MHI bus support.
I am gonna fixup the PCI support as well, but when I don't really know.

ID is just being reused, that is why its seeing it as a modem

2977

@robimarko you know what? i'm adding the memory stats patch so we can check who is actually leaking

1 Like
2978

I enabled MHI Bus support in make kernel_menuconfig, didn't help. Why is it not showing a meaningful error message though?

Also that dp6 didn't fix the 2.5Gbit/s port.

2979

@Ansuel Not a bad idea.

@Flole Because it quite literaly is missing the .ko file.
Its still missing something to enable the symbol, its hard to tell blindly.
You can try checking manually in the backports build dir

2980

I manually went through the dependencies:

CONFIG_QRTR_MHI

was missing. Should that be added somewhere so it automatically gets selected?

2981

memory stats ported... lets see if it doesn't crash the system ahahah

anyway the gcc 11 static analyzer reported some skb dereferenced but i don't think they are related

2982

It's usually auto-selected, but the thing with backports is that they cannot select stuff outside of the backports itself, so the logic was changed to depend on it instead.
And it's usually fine, but ath11k depends on stuff that is sometimes not even user-selectable like QMI helpers which I had to make visible first.
MHI could be packaged as a kernel module as its tristate and then CONFIG_QRTR and CONFIG_QRTR_MHI could be selected by that kernel module.

2983

@robimarko

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 187648
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18862157

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 195072
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18869581

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 198144
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18872653

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 198144
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18872653

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 205568
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18880077

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 205568
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18880077

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 205568
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18880077

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 205568
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18880077

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 227840
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18902349

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 227840
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18902349

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 227840
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18902349

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 227840
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18902349

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 235264
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18909773

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 235264
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18909773

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 235264
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18909773

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 235264
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18909773

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 235264
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18909773

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 238336
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18912845

root@OpenWrt:/# cat /sys/kernel/debug/ath11k/ipq8074\ hw2.0/memory_stats
MEMORY STATS IN BYTES:
malloc size : 3145728
ce_ring_alloc size: 134488
dma_alloc size:: 9899829
htc_skb_alloc size: 6400
wmi tx skb alloc size: 245760
per peer object: 4544
rx_post_buf size: 5483520
Total size: 18920269
2984

If i haven't messed the wmi track part of the patch (i need to check)
it looks like wmi tx sbk is the one leaking

2985

Hmm, that looks interesting.
I doubt you messed with WMI as it leaks without any kind of NSS stuff.

Can you share the patch?