Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

diogosena · 2022-03-30T13:12:55.392Z

Just retested, my setup is using wifi as wwan and repeater, wwan max speed is +-330mbps

ath10k is the worst, even iperf inside my network is limitted to +-50mbps
ath10k-ct is not limiting in iperf (+-350mbps), but is limiting in speedtest.net or fast.com (+-70mbps).
openwrt19, is not limiting anything, speedtest.net or fast.com(+-200bmps).

any ideas? this happenned in both v21 and v22 versions.

anon98444528 · 2022-03-30T13:49:06.189Z

diogosena:

any ideas?

Not really.

If possible, try your v21 or v22 speed tests with a client wired to the AP. Also, try repeating the wireless speed test but ensure one and only one client is connected to the 5GHz band. My own observations are if more than one client is connected on the same wifi band, I can see significant variation in throughput even if the other clients are not doing much.

I gave up on the non ct ath10k (for qca9980) a long time ago.

The biggest difference I see between 19 and 21 or 22 is the kernel went from 4.14 to 5.4. Kernels 5.4+ are good for me in terms of overall stability (AP uptimes > 100+ days). However, during that transition is also when I noticed a degradation in "wifi quality." Unfortunately, about this time is when I think ATF/AQL was introduced (which is intended to improved "quality"). I can't (yet) pinpoint what the root cause is for the change in wifi behavior - it could just be a combination of how my family uses wifi now and changes in how I assess "quality."

FWIW I'm willing to trade off throughput for "quality" (i.e. low bufferbloat - stable video conferencing).

HTH

diogosena · 2022-03-30T14:08:10.541Z

wired speed test is ok (+-330)

my tests where using just one client already, but the router itself is the client of another router, maybe that triggers something.

i don't really have a problem with v19 throughput or latency, but i'll keep using v21 for a while just to see if I notice some kind of improvement

anon98444528 · 2022-03-30T14:09:12.815Z

diogosena:

but the router itself is the client of another router

A wireless bridge? If so, on the same wifi band (5GHz)?

diogosena · 2022-03-30T14:12:08.105Z

Yes, only using 5ghz

anon98444528 · 2022-03-30T14:27:27.931Z

Short term, I don't think there is much you can do. I'm guessing ditching the wireless bridge and connecting your AP via a wire is not an option you want.

For the purpose of a test, you could rearrange your AP's such that the r7500v2 is connected to your isp via wire, temporarily turn off the bridge, and test just using 5 GHz as an AP only. But this will only confirm that post v19, wifi throughput variation increases (I call it throughput jitter).

You could also try the v19 ath10k-ct firmware on v21 or v22. If there is an improvement with v19 ath10k-ct firmeware, you can ask for assistance from the ct dev but you will likely have to spend a fair amount of time working with them.

EDIT: you can also try disabling AQL on v21 or v22

echo 0 > /sys/kernel/debug/ieee80211/phy0/aql_enable
echo 0 > /sys/kernel/debug/ieee80211/phy1/aql_enable

REF

ACwifidude · 2022-03-30T15:20:07.637Z

Speed tests with the router as the server or the client for iperf? Using the router as the iperf server or client will lead to inaccurate numbers when trying to test max 5ghz or max wired speeds. Instead test it with a wired computer as the iperf server and a 5ghz wireless client (iperf client).

Wireless bridges insert latency and is the biggest source of your performance degradation. Any way you can wire to the source?

diogosena · 2022-03-30T16:29:00.039Z

Already using a wired gigabit client as the server, disabling AQL made no perceivable difference.

After some time using v21 ath10k-ct , wifi started to behave slow in iperf (like ath10k)

I know about the issues of wireless bridge, but we are talking v19 bridge vs v21+ bridge, wiring is not possible in this case.

Thanks for all the help, but I'll keep v19 for a little more..

xeonpj · 2022-03-30T18:49:05.458Z

with the NON NSS versions do you get the same results?21.02, 20.02, etc.

diogosena · 2022-03-30T19:10:49.382Z

Yes, it's not related to NSS (sorry for hijacking the thread)

ACwifidude · 2022-03-30T19:30:43.836Z

If there is any way to fish a wire or move the access point so that you could fish a wire - it is well well well well worth it. I hired help and paid for less than 2 hours of electrician labor (tough to fish lines by yourself!) and I’m able to get full performance for all my APs.

Best of luck!

GollyJer · 2022-03-31T02:40:10.448Z

Is Master or 22.03 the "newest" build? It seems most people have clicked Master but I'm not sure which to use. Thanks!

Looks like maybe Master is newest and uses the updated kernel?

aamadorj · 2022-03-31T11:53:51.725Z

This previous post has an explanation: Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500) - #1897 by ACwifidude

ACwifidude · 2022-03-31T14:08:46.404Z

@aamadorj post is spot on.

Master is always the most cutting edge “trunk”, followed by the 22.03 branch, and 21.02 branch.

I updated the naming scheme for all the links above so the click counts got reset a couple days ago. The click counts have been reset several times as I make sure the build has the latest stuff (click counts don’t matter). Go with the branch or trunk that best aligns with your goals - cutting edge vs more tried and true. There are pros and cons to both. I like cutting edge so I’m always on master.

GollyJer · 2022-03-31T14:51:16.949Z

Cool. Thanks for the explanation and the builds.
Cutting edge is where I'll be.

Cyanoxygen · 2022-04-01T05:06:18.760Z

Hi ACWifidude,

I am using your tree (openwrt-21.02-nss-qsdk10.0 branch rebased to OpenWrt upstream) to build my custom firmware for my R7800, but I currently encountering some issues with my custom build, and I want to figure out why and try to solve it. Any help is appreciated!

Issues encountered

after flashing it, the kernel log is keep being spammed by the lines shown below[1].
Wifi sometimes stops its transmission - it keeps connected but pinging and performing network operations does not work for a few seconds. Not sure if it is related to NSS, but I will post this anyway.
~~Compared to non-NSS build, Wired Ethernet throughput is way lower (~800Mbps compared to 940Mbps) using iperf3. I tested it with a Raspberry Pi 4, server runs at the router side~~ Have tested with another device, the router can now handle full gigabit with zero CPU.

What I am doing and have done for this router

I am using -ct driver (I have tested non-ct driver and firmware) and performance CPU governor (CPU maxed at 1725MHz).
I am using 5GHz WiFi for both WWAN and AP, not sure if the second issue is related to NSS. Tried disabling AQL but no effect.
I have tested your build from the git repo, and using both WWAN and AP on 5GHz, there is no issue and no repeating lines. I need the iptables-mod-tproxy package but it conflicts because the kernel does not like it, so I can not use your build for me.
With my build, I have enforced 800MHz NSS core frequency by echoing it and cated it (you can see the kernel logs[2]), but these lines keep spamming. I also disabled AQL but no effect whatsoever.

Notes

[1] the repeating lines:

[  183.669431] net_ratelimit: 24 callbacks suppressed
[  183.669435] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  183.706858] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  183.790479] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  183.838664] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  183.950342] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  184.043845] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  184.218573] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  184.262553] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  184.327766] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
[  184.327799] wlan0-1: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED

Those lines keep spamming every 5 seconds.

[2] Kernel log uploaded to Gist
[3] My diffconfig to defconfig (./scripts/diffconfig.sh)
[4] diff of my diffconfig and yours (I enabled lots of packages so it is big)
[5] My complete config

shelterx · 2022-04-01T11:27:04.292Z

Cyanoxygen:

NSS_TX_FAILURE_NOT_ENABLED

I think that's because NSS acceleration isn't enabled. You probably missed something in your custom build.
EDIT: I don't see ECM being loaded in your kernel log.

Cyanoxygen · 2022-04-01T11:45:18.804Z

Hi,
Yes, I did check that ecm is not being loaded - I disabled NSS for wireless by echo 0 > /sys/module/mac80211/parameters/is_nss_enable for now, I will reenable it now and check if loading ecm works. Thanks!
EDIT: Unfortunately loading ECM does not work out. I reenabled by echo 1 and restarted network

image1359×766 101 KB

I use WWAN for internet, so PPPoE is not used.

Cyanoxygen · 2022-04-01T12:19:47.203Z

image936×441 164 KB

I looked at qca-nss-ecm service script, it looks like there are something missing. ecm kmod does exist, but it can not load by insmod, by adding an entry in /etc/modules.d/ it can be modprobe'd after a depmod.

ACwifidude · 2022-04-01T12:31:49.289Z

The NSS frequency is locked at 800mhz by the .dts patch adding it to the ipq8065.dtsi file (should save you a step).

Hopefully your ecm is fully back up and operational. If you don’t want wireless offloading you can remove the mac80211 NSS commit next time you build.