Adding OpenWrt support for Xiaomi AX3600 (Part 1)

@robimarko / @Ansuel Im totally out of ideas for tracking the memory leak.

If it is within ath11k (seems that way) then hunting within that driver is beyond my understanding for that driver :frowning:

if you want i can give some try...

to reproduce the leak i can use open wifi with 2.4 correct?

@Apache14 Yeah, me as well.
It all points to ath11k or QRTR leaking, but no idea where.

@Ansuel Yeah, 2 or 5G works, just enable it and you will see it leaking after couple of minutes.

Maybe reaching out to Qualcomm devs through the ath11k mailing list could help? Chances not too high but might be worth a try

Did I just get one of the PCIe ports to work?

[    0.179380] qcom-qmp-phy 84000.phy: Registered Qcom-QMP phy
[    0.181137] qcom-pcie 20000000.pci: host bridge /soc/pci@20000000 ranges:
[    0.181188] qcom-pcie 20000000.pci:       IO 0x0020200000..0x00202fffff -> 0x0020200000
[    0.181215] qcom-pcie 20000000.pci:      MEM 0x0020220000..0x002fffffff -> 0x0020220000
[    1.300917] qcom-pcie 20000000.pci: Phy link never came up
[    1.301074] qcom-pcie 20000000.pci: PCI host bridge to bus 0000:00
[    1.301091] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.301105] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0x20200000-0x202fffff])
[    1.301116] pci_bus 0000:00: root bus resource [mem 0x20220000-0x2fffffff]
[    1.301161] pci 0000:00:00.0: [17cb:1002] type 01 class 0x060400
[    1.301185] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    1.301265] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    1.306690] pci 0000:00:00.0: BAR 8: assigned [mem 0x20300000-0x204fffff]
[    1.306710] pci 0000:00:00.0: BAR 9: assigned [mem 0x20500000-0x206fffff 64bit pref]
[    1.306723] pci 0000:00:00.0: BAR 0: assigned [mem 0x20220000-0x20220fff]
[    1.306736] pci 0000:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[    1.306749] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    1.306760] pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    1.306772] pci 0000:00:00.0:   bridge window [mem 0x20300000-0x204fffff]
[    1.306784] pci 0000:00:00.0:   bridge window [mem 0x20500000-0x206fffff 64bit pref]
[    1.306808] pci 0000:00:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  128
[    1.308676] pcieport 0000:00:00.0: PME: Signaling with IRQ 103
[    1.309761] bam-dma-engine 704000.dma: num-channels unspecified in dt
[    1.309778] bam-dma-engine 704000.dma: num-ees unspecified in dt 

The wifi radio is not visible though...

Is there really no easy way to recompile the dts and make an initramfs from it only without deleting and recompiling the entire kernel every time? That is such a waste of time

Also one of my ethernet ports is causing a warning, connecting a cable causes:

[  284.063371] ------------[ cut here ]------------
[  284.063408] nss_port5_rx_clk_src: rcg didn't update its configuration.
[  284.063443] WARNING: CPU: 0 PID: 973 at update_config+0xb8/0xc8
[  284.073394] Modules linked in: pppoe ppp_async iptable_nat ath11k_ahb ath11k ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 qca_nss_dp qca_ssdk michael_mic leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom gpio_button_hotplug
[  284.116355] CPU: 0 PID: 973 Comm: kworker/0:2 Tainted: G        W         5.10.41 #0
[  284.138591] Hardware name: Netgear SXR80 (DT)
[  284.146739] Workqueue: events qca_mac_sw_sync_work_task [qca_ssdk]
[  284.150570] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
[  284.156647] pc : update_config+0xb8/0xc8
[  284.162802] lr : update_config+0xb8/0xc8
[  284.166706] sp : ffffffc01206bb00
[  284.170611] x29: ffffffc01206bb00 x28: 0000000000000000
[  284.173828] x27: 0000000000000001 x26: 0000000000109100
[  284.179210] x25: ffffffc01206bd38 x24: 0000000000000001
[  284.184505] x23: ffffff8003319c00 x22: ffffffc010850748
[  284.189800] x21: 0000000000000000 x20: ffffffc011e058e0
[  284.195095] x19: 0000000000000000 x18: 0000000000000000
[  284.200390] x17: 0000000000000000 x16: 0000000000000000
[  284.205684] x15: 0000000000000000 x14: 0000000000000000
[  284.210981] x13: 0000000000000000 x12: ffffffc011de7c68
[  284.216275] x11: 0000000000000003 x10: ffffffc011dcfc28
[  284.221570] x9 : ffffffc011dcfc80 x8 : 0000000000017fe8
[  284.226866] x7 : c0000000ffffefff x6 : 0000000000000001
[  284.232160] x5 : ffffff803edc26d0 x4 : 0000000000000000
[  284.237456] x3 : 0000000000000027 x2 : 0000000000000027
[  284.242751] x1 : 0000000000000023 x0 : 000000000000003a
[  284.248047] Call trace:
[  284.253338]  update_config+0xb8/0xc8
[  284.255510]  clk_rcg2_configure+0x58/0x88
[  284.259330]  __clk_rcg2_set_rate+0x34/0x58
[  284.263237]  clk_rcg2_set_rate_and_parent+0x14/0x20
[  284.267234]  clk_change_rate+0x7c/0x2a0
[  284.272003]  clk_core_set_rate_nolock+0x180/0x1b8
[  284.275824]  clk_set_rate+0x38/0xa8
[  284.281045]  ssdk_uniphy_clock_rate_set+0x80/0x130 [qca_ssdk]
[  284.284288]  ssdk_port_speed_clock_set+0xb0/0x148 [qca_ssdk]
[  284.290178]  qca_hppe_mac_sw_sync_task+0x3b0/0x904 [qca_ssdk]
[  284.295900]  qca_mac_sw_sync_work_task+0x40/0x308 [qca_ssdk]
[  284.301266]  process_one_work+0x200/0x3b0
[  284.306987]  worker_thread+0x54/0x4e8
[  284.310893]  kthread+0x124/0x128
[  284.314539]  ret_from_fork+0x10/0x30
[  284.317836] ---[ end trace 339466354e08d8a5 ]---
[  284.322360] ------------[ cut here ]------------
[  284.326029] nss_port5_tx_clk_src: rcg didn't update its configuration.
[  284.326061] WARNING: CPU: 0 PID: 973 at update_config+0xb8/0xc8
[  284.336935] Modules linked in: pppoe ppp_async iptable_nat ath11k_ahb ath11k ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 qca_nss_dp qca_ssdk michael_mic leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom gpio_button_hotplug
[  284.379895] CPU: 0 PID: 973 Comm: kworker/0:2 Tainted: G        W         5.10.41 #0
[  284.402130] Hardware name: Netgear SXR80 (DT)
[  284.410266] Workqueue: events qca_mac_sw_sync_work_task [qca_ssdk]
[  284.414111] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
[  284.420190] pc : update_config+0xb8/0xc8
[  284.426343] lr : update_config+0xb8/0xc8
[  284.430249] sp : ffffffc01206bb00
[  284.434153] x29: ffffffc01206bb00 x28: 0000000000000000
[  284.437370] x27: 0000000000000001 x26: 0000000000109100
[  284.442752] x25: ffffffc01206bd38 x24: 0000000000000001
[  284.448046] x23: ffffff8003319d00 x22: ffffffc010850730
[  284.453341] x21: 0000000000000000 x20: ffffffc011e057d8
[  284.458636] x19: 0000000000000000 x18: 0000000000000000
[  284.463931] x17: 0000000000000000 x16: 0000000000000000
[  284.469227] x15: 0000000000000000 x14: 0000000000000000
[  284.474522] x13: 0000000000000000 x12: ffffffc011de7c68
[  284.479817] x11: 0000000000000003 x10: ffffffc011dcfc28
[  284.485112] x9 : ffffffc011dcfc80 x8 : 0000000000017fe8
[  284.490407] x7 : c0000000ffffefff x6 : 0000000000000001
[  284.495702] x5 : ffffff803edc26d0 x4 : 0000000000000000
[  284.500998] x3 : 0000000000000027 x2 : 0000000000000027
[  284.506294] x1 : 0000000000000023 x0 : 000000000000003a
[  284.511589] Call trace:
[  284.516880]  update_config+0xb8/0xc8
[  284.519053]  clk_rcg2_configure+0x58/0x88
[  284.522872]  __clk_rcg2_set_rate+0x34/0x58
[  284.526779]  clk_rcg2_set_rate_and_parent+0x14/0x20
[  284.530774]  clk_change_rate+0x7c/0x2a0
[  284.535545]  clk_core_set_rate_nolock+0x180/0x1b8
[  284.539366]  clk_set_rate+0x38/0xa8
[  284.544584]  ssdk_uniphy_clock_rate_set+0x80/0x130 [qca_ssdk]
[  284.547832]  ssdk_port_speed_clock_set+0xc0/0x148 [qca_ssdk]
[  284.553717]  qca_hppe_mac_sw_sync_task+0x3b0/0x904 [qca_ssdk]
[  284.559441]  qca_mac_sw_sync_work_task+0x40/0x308 [qca_ssdk]
[  284.564807]  process_one_work+0x200/0x3b0
[  284.570528]  worker_thread+0x54/0x4e8
[  284.574434]  kthread+0x124/0x128
[  284.578080]  ret_from_fork+0x10/0x30
[  284.581378] ---[ end trace 339466354e08d8a6 ]---

I saw that the same issue was observed on the AX3600, was this ever fixed and what fixed it?

Nope, you can see that phy link never came up.
You can use lspci to see what the kernel sees.

I am gonna guess that the radio is on the Gen3 port, that is &pcie0.
perst-gpio for it is: perst-gpio = <&tlmm 58 GPIO_ACTIVE_LOW>;

If that does not cause a PHY linkup then you can try &pcie1
perst-gpio for it is: perst-gpio = <&tlmm 61 GPIO_ACTIVE_LOW>;
But for it, you also must enable the &qmp_pcie_phy1

Then simply use lspci to see what pops up.
Radio is not gonna work just by itself, you gotta enable the pci ath11k kmod as well and the firmware for QCA6390 and hope that your QCA6290 is close enough and supported.

That clock only pops up when its from initramfs.

BTW, you are missing DP6 node for the 2.5G PHY.

dp6 {
			device_type = "network";
			compatible = "qcom,nss-dp";
			qcom,id = <0x06>;
			reg = <0x3a001a00 0x200>;
			qcom,mactype = <0x00>;
			local-mac-address = [00 00 00 00 00 00];
			qcom,link-poll = <0x01>;
			qcom,phy-mdio-addr = <0x1c>;
			phy-mode = "sgmii";
		};
Mem: 233708K used, 137832K free, 76K shrd, 0K buff, 21804K cached
Mem: 250564K used, 120976K free, 76K shrd, 0K buff, 21808K cached
Mem: 259600K used, 111940K free, 76K shrd, 0K buff, 21808K cached

this is the leak observed correct?

Sure looks like it, 16MB of memory used more.
Can you try doing a crazy thing and connecting a client to WLAN, running speedtest and then checking memory usage?

the strange thing is that for some reason...

Mem: 214480K used, 157060K free, 76K shrd, 0K buff, 21808K cached

this is before the speedtest so not related... currently i have my phone and my laptop connected to the router (wifi with no additional patch so no offload and all)

Having clients will increase the RAM consumption, but it shouldn't just increase on its own after that.
BTW, 214MB is too much

i'm back to 234 235 238 HALP

anyway this is with ksan and memory poisoning so it i don't think it's something from memory allocation... but ioremap or page allocation

anyway from the bootup memory is 198 mb

But ath11k calls ioremap like only 2 times.
On start it uses the devmem ioremap for the base resource from DTS, and once more for the BDF.

BTW, I am doing 443/92 Mbits on HE40 5G radio on my phone.
Offloading NAT really helped as now CPU load will be like 60% on one core and 15% on others.
Note, that this is not iperf3, but speedtest to my ISP IXP, so its topping out my uplink at this point as the network gets really loaded and I cant get the 500/120Mbits like in other times of the day.
iperf from the phone to desktop is still horrendus at 40Mbits.

@Ansuel My RAM consumption was 134MB, and then after starting ath11k and connecting a client it went to 145MB.
And now its just climbing at 200+MB in like 2 minutes.
So it looks like its leaking badly, like you can see it with your eyes.

ok so HE40 on 5G... we need to find the worst case (if that is even related)

WTF, it now dropped to 170MB of RAM usage?

same happened to me... like randomly memory free...

in my case from 260 to 210

let me check if it will happen the same to me... for now i'm at 248mb

Haha, not its at 152MB.
iperf3 is still horrible, but like how when speedtesting to upstream gets me full uplink speed?

in my case iperf from one device to another 174 mb

but if the router does the iperf server, 400+ mb LOL

LOL, with UDP iperf3 I get like 1Mbitps.
Wait what?
With parallel streams each one gets like 45-50Mbits and they sum at 200Mbits?
But why iperf3, why?

So, with many streams the WLAN perfroms great actually, I get 400+Mbits on 40MHz AX.
That is really good, but why do I need parallel streams iperf?

it honestly seems that there is a problem with transfer between device in the same network?

about parallel streams... yesterday i got 1gbps with 5 parallel stream... with a single one i got 600 max

To me it looks like a wpa_supplicant/hostapd issue as with parallel streams it gets the full radio speed.
Anyway, it looks that RAM stabilized at around 170MB, weirdly it goes down with load.