Adding OpenWrt support for Xiaomi AX3600 (Part 1)

snoova · 2022-02-13T18:34:07.261Z

Oh man, thanks!
i activated SW offloading (Network > Firewall > General Settings > Routing/NAT Offloading) and it moves in the right direction. 720 Mbits DL.
How can i check if packet steering is enabled?

EDIT:
Packet steering is now enabled, however 720Mbits DL is currently the max i can reach.

r3pek · 2022-02-13T18:35:30.361Z

And me trying to understand why jumping from one router to the other took like 5 minutes for being able to get the DHCP ACK. This is why

sqrwv · 2022-02-13T18:59:38.707Z

With the script it's instantaneous again.
If the original script gives trouble, try this version from Edrikk here:

Adding OpenWrt support for Xiaomi AX3600 (Part 1) For Developers

So some observations, and consolidation of info from across this thread that seems to be going well for me that this point, on my AX3600, running the latest (as of Jan 29th) Robi K5.15.15 codebase. It is acting as the main router against a Cable Modem which is bridged: Memory seems to be behaving much better now with the 512MB patch in place.at 13 hours uptime, the device is sitting at 128MB free out of 407.75MB total. I was having the issue where some wireless clients couldn't connect/transi…

dchard · 2022-02-13T19:06:25.376Z

If you are not reaching 90+% on a single CPU core, the limitations is somewhere else. Maybe your provider is not able to keep up in busy hours. It can be crappy anti virus, or firewalling issues, bad cabling etc. All off topic here. Maybe you can try again when Robi merges the NAPI patches.

Wait a minute: you are trying on direct cable connection and not on wifi, do you?

cjom · 2022-02-13T19:24:07.686Z

Edrikk:

I was having the issue where some wireless clients couldn't connect/transition/use the internet. I had forgottent hat I had read about this in another thread, and completely missed that you need "qca-ssdk-shell to workaround the issue. I have included my build config below.

Thank you, thank you, thank you, thank you, thank you, thank you!
I was banging my head and my phone because of this! Of all my house devices including other phones equal to mine, only mine was loosing connection after a couple of minutes in standby and even worse when I moved a bit far away from the router. And I was thinking the reason had something to do with my phone being the only one using a wireguard VPN while the router was also serving VPN to some devices (not mine).

You pointed me in the right direction, but now I have some data to add that may help development, I hope.

Build yesterday with "nss-firmware-ipq8074", "kmod-qca-nss-dp" and "qca-ssdk-shell", also user @dchard IRQ optimizations, but in my hurry forgot to add the wifi_transition_workaround.sh!

I am trying to note here that even without the script, my connectivity issues were fixed. Before this I missing nss-firmware-ipq8074 (and qca-ssdk-shell), I suppose this firmware package could be set to yes by default?

Edrikk · 2022-02-13T19:31:56.244Z

I'd love to take the credit, but the script is fully thanks to @joba-1 per his post here. I just packaged it so that it can be easily put in the startup and sent to the background.

The scripts are identical. i.e. they both depend on qca-ssdk-shell. The script keeps an eye on the rolling log file and if the substring "AP-STA" is seen calls a utility (from qca-ssdk-shell) to to a flush.

Crect · 2022-02-13T19:40:06.743Z

Went through to code to find out what could have happened but I'm not sure if it's the kernel messing up or uboot. Basically uboot will write the fdt into memory and pass its location to the kernel in the r2 register. If kernel was compiled with fdt support, it will load the fdt from there and unflatten it to build its internal tree representation.

elixir.bootlin.com

fdt.c - drivers/of/fdt.c - Linux source code (v5.16.9) - Bootlin

Elixir Cross Referencer - Explore source code in your browser - Particularly useful for the Linux kernel and other low-level projects in C/C++ (bootloaders, C libraries...)

I can think of 2 ways you could debug this further. Either add some logging to where the kernel unflattens the fdt to see if the nodes actually reach the kernel or inspect the fdt in uboot with the fdt command. After all it might be uboot that deletes the partition nodes from the fdt for some Xiaomi-ish reason.

snoova · 2022-02-13T19:47:39.421Z

You are right, is cable connected PC.

I will test the speed with an ASUS router again, than switch back to the Xiaomi. However I don’t think, that the ISP is the problem. My connection is normally very stable.

With 5.10 I got full speed of my ISP provided speed, so I'm afraid it has something to do with the current version or setting.

I will update to a new version, as soon robi will provide something new.

robimarko · 2022-02-13T20:03:09.680Z

Added the NAPI patch and pushed.

robimarko · 2022-02-13T20:05:14.382Z

Thanks for looking into it, yeah U-boot will modify what it wants, and then it should just pass the memory pointer in r2.

I think I will try dumping the DTB from the running kernel, that way its clear if U-boot messed something up.

dchard · 2022-02-13T20:16:11.433Z

Great, build is on its way. Any sysfs or other config flags that might be needed?

robimarko · 2022-02-13T20:23:40.299Z

No, its enabled by default in the driver for IPQ8074 and QCN9074 so it should just work

dchard · 2022-02-13T20:54:17.983Z

On wired networking the speed and CPU load is the same as it was.

On Wifi the load is slightly lower, but it is spread quite a bit better then it was before across the cores.

Single client, AC @ 330Mbits Iperf3 DL (4 threads, WLAN-LAN):

Single client, AC @ 342Mbits Iperf3 UL (4 threads, WLAN-LAN):

80MHz@SISO in both the above cases. What I noticed is that Speedtest starts faster, and the ramp-up phase is also quicker. Tomorrow I can do AX tests if someone is not doing it in the mean time

robimarko · 2022-02-13T21:00:01.482Z

Yeah, wired won't be affected it's just enabling threaded NAPI for ath11k.

I did a couple of rounds of AX@80MHz and I can get around 720Mbps compared to around 550Mbps without threaded NAPI and it spreads the load much better.
This is with my phone (Poco F2 Pro) and with 5 parallel connections as with only 1 I cant really get more than 330-ish usually
And it also includes NAT overhead as well as iperf3 server was intentionally connected to WAN.

dchard · 2022-02-13T21:06:15.760Z

Will do some 160MHz AX tests tomorrow. The diff should be very visible in that case. The load spread is already visible in AC mode, but maybe it was not enough traffic for NAPI to show a significantly lowered CPU load. We will see tomorrow.

Edrikk · 2022-02-14T01:42:25.409Z

Edrikk:

Same here (AX3600). Always been like this, and always 2.

I have even tried reboot first followed by immediate first flash, but even then, a second flash is required.

So I should have just kept quite. For this first time, it took me more than 2 attempts tonight.
Updating to the latest compiled version (r18856-d21fd5135c ) took a solid NINE flashes.

Edrikk · 2022-02-14T03:30:36.458Z

About 2 hours in, with the same build packages as before (actually +htop), I’m seeing free memory staying up at between 140MB and 170MB.

I’ve never seen free memory stay this high post boot for more than a couple of minutes. Prior builds the free memory would fluctuate between 90MB and 110MB.

@dchard are you seeing similar?

--
Update: At 14 hours uptime point, 157MB free

ronison · 2022-02-14T03:38:35.682Z

Same here!

dchard · 2022-02-14T09:34:48.823Z

image931×261 3.16 KB

The memory consumption is better on my end as well. Never seen this low memory usage after half a day ever before. So yes, it improved quite nicely.

dspalu32 · 2022-02-14T10:02:40.815Z

I'm seeing lower RAM usage too.

Related to this, the low-traffic AP (ap3 below, click to zoom) seems to have the margin to stably sit in the higher usage RAM state without OOM. Still too early to conclusively, but it seems the iperf3 cron workaround may not be strictly needed any more: the ~100MB effective loss of RAM isn't a functional problem.

sysupdate ran OK on the first attempt on two routers, with a "wifi down; killall -9 wpad; killall -9 hostapd" prefix. It failed on first attempt on the third router where WiFi wasn't first forced offline, and worked on the next try with WiFi taken offline. These three AX3600 are only WiFi/WDS connected, but that isn't a barrier to issuing a typo-free compound command that force-kills your ssh session.

mem397×811 17 KB

Edit: 6 hours later, it's still stable at the higher level of usage, 100Mb free:

8b085be7-ad3f-4730-97a7-f74bedfbbaa5