Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

rickkz0r · 2023-05-01T07:12:22.564Z

yes it is a problem in the NSS drivers. I don’t have the links handy right now but the bottom line is that you need to enable promiscuous mode on the lan bridge.

ACwifidude · 2023-05-01T13:40:31.731Z

ACwifidude:

Make custom DNS internal sites work (if you have custom sites):
ifconfig br-lan promisc
turn custom internal sites off:
ifconfig br-lan -promisc

This should help I hope!

CompositeBlues · 2023-05-01T20:55:22.788Z

Thanks for the info. To be honest I think I'm going to drop the R7800 and go with something else, it's been more trouble than it's worth and I'm looking for AX capabilities soon so will probably bite the bullet and get something sooner rather than later. Thanks for the info and the work you and others have already put into this.

M10 · 2023-05-01T21:15:20.793Z

I have dynalink wrx36 as room ap, without nss offloading is slow , wan to wlan only gets me like 500Mbits. I now considering to replace all openwrt aps with some ofw ap's to get real speeds... Already did replace my gateway with pfsense on thin client and it works like a rock since of couple years.

apccv · 2023-05-01T22:05:45.150Z

Similar to what I'm thinking, but I will keep the C2600's as AP. With the NSS build they are solid for gigabit WLAN/LAN routing. I'll implement x86 for core routing.

Once a good AX AP is available, including 6GHz band, stable, at a decent price and with a 2.5GbE I'll replace the C2600. Will be a while, IMO.

apccv · 2023-05-02T00:58:34.672Z

Just when I thought I had the final build for the 22.03 series done, 22.03.5 is out. Here goes "vanilla 22.03.5 plus NSS build including all modules" (still don't have a name for it, probably never will).

GitHub

GitHub - APCCV/OpenWRT-22.03.5-NSS: This is a clone of OpenWRT release...

This is a clone of OpenWRT release 22.03.5 with partial support for NSS cores. Credit goes to all the OpenWRT devs that made it possible for me to build these images, especially for ACwifidude (htt...

Kept commits clean and simple.

Images and all kernel modules in bin directory.

Just as before, this is a rebuild of the 22.03.5 tagged release (22.03.5 tagged build) build plus:

The NSS commits on ACwifidude's repository
Minimal changes to three files to allow all modules to build (this is in a commit of its own)
-- package/qca/qca-nss-clients/Makefile (remove non building packages)
-- package/qca/qca-nss-ecm/Makefile (ditto)
-- target/linux/ipq806x/patches-5.10/999-07b-qca-nss-clients-qdisc-support.patch (remove non building patches to GRE)

The 22.03.5 config.buildinfo (the 22.03.5 diffconfig) with additional NSS packages is checked in as diffconfig-nss.

Also as before, GitHub can be used as a repo for this build's modules. For that:

copy the file in the opkg-key directory to /etc/opkg/keys
update the /etc/opkg/distfeeds.conf to use repo instead by commenting out the first line and adding a second with the URL to this repo, as in:

#src/gz openwrt_core https://downloads.openwrt.org/releases/22.03.5/targets/ipq806x/generic/packages
src/gz openwrt_nss https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.5-NSS/22.03.5-nss/bin/targets/ipq806x/generic/packages
src/gz openwrt_base https://downloads.openwrt.org/releases/22.03.5/packages/arm_cortex-a15_neon-vfpv4/base
src/gz openwrt_luci https://downloads.openwrt.org/releases/22.03.5/packages/arm_cortex-a15_neon-vfpv4/luci
src/gz openwrt_packages https://downloads.openwrt.org/releases/22.03.5/packages/arm_cortex-a15_neon-vfpv4/packages
src/gz openwrt_routing https://downloads.openwrt.org/releases/22.03.5/packages/arm_cortex-a15_neon-vfpv4/routing
src/gz openwrt_telephony https://downloads.openwrt.org/releases/22.03.5/packages/arm_cortex-a15_neon-vfpv4/telephony

opkg will install packages and handle dependencies nicely after that.

The ImageBuilder gets built fine and if the opkg changes above are applied after unarchiving it should work like a charm. It's here: 22.03.5-NSS Image Builder. The images in bin have the same packages as the OpenWRT vanilla ones (using ath10k-ct) and without any add-ons beyond NSS. All modules are available in this repo so in best of theories ImageBuilder can produce whatever image may be required.
For example, to use ath10k instead of the -ct drivers:
make image PROFILE='tplink_c2600' PACKAGES='ath10k-firmware-qca99x0 -ath10k-firmware-qca99x0-ct kmod-ath10k -kmod-ath10k-ct <plus the myriad other packages required like luci>' (profile netgear_r7800 for 7800, make info provides a list of what is supported, and mind that the 7800 uses qca9984 instead.)

The only configuration tweak we use, as suggested by ACwifidude in rc.local, although YMMV and it might not be needed at all is:

# CPU governor fixes
echo 600000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
echo 600000 > /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq
echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

I have been running 22.03.2 and 22.03.4 on TP-Link C2600 routers in different roles (router, pppoe, AP) with great results and minor issues. I didn't bother updating to 22.03.3 and probably will not go through the hassle for 22.03.5 given the change log is relatively skimpy (openssl can be updated separately through image builder or reinstalled on top of 22.03.4... these devices have plenty free flash space).

Here is a test of a 22.03.2 AP using a 22.03.5 client. Looks, feels, smells, tastes more or less the same as before. I'm not the only one using the net right now so maybe that's why speed took a bit of a hit, but still over the 0.5Gbps mark. I'm also not testing from an ideal location... I've seen the wireless bridge operating at 1600.0 Mbit/s both ways before. For anyone using the 7500 with a 160 MHz band width could probably do better.

Screenshot from 2023-05-01 21-18-03

Screenshot from 2023-05-01 21-17-40792×684 65.3 KB

Again, there it is, the 22.03.5 build (will it be the last of the 22.03?) in case anyone wants to give it a try. Cheers!

Mpilon · 2023-05-02T06:21:20.686Z

(still don't have a name for it, probably never will).

How about "Nameless"?

sppmaster · 2023-05-02T06:54:44.590Z

Maybe now there is another/better option to Fix NAT Loopabck with ECM enabled mentioned here.

min2qaz · 2023-05-02T07:01:03.240Z

can you include build for chromium

xeonpj · 2023-05-02T07:34:23.150Z

Good job!!!!

sppmaster · 2023-05-02T07:38:30.025Z

All credits to @AgustinLorenzo.

fruityten · 2023-05-02T16:06:44.155Z

Hey, thanks for the build! I switched from hnyman's build over to this one so I can maximise my 500mbps connection.

All set up and going fine as expected. One small issue I have noticed and I suspect I just need to change something, but sadly I don't know what. I can access my webserver via its LAN IP, but not via my public IP/domain. Everything is working as expected off my Network using 5G.

Perhaps some NAT rule or Firewall needs changing. I stress everything else is working, previously using hnymans build and vanilla OpenWRT I've not come across this before.

Thanks for any suggestions

apccv · 2023-05-02T16:24:03.722Z

Hey there. Happy to hear this works for you, somewhat.

I noticed the issue too… seems to be a problem with the NSS ems service, whatever that is.

Seems that the quick fix is to set the bridge (br-lan) in promiscuous mode (that is, accepting any packages). The config option is in LuCI in bridge config. I have not tried it myself, yet.

The fancier fix is to change some config files and enable “hairpin mode”. I will try to retrofit that in. If it requires a build I’ll leave it to a patch be added to the main repo and once a new release is out I’ll pick it up.

Lemme know if that works for you please. If not, check the NSS thread as some suggestion popped up there already. Mostly this ^, but there may be something new.

Cheers!

fruityten · 2023-05-02T16:53:51.225Z

Thank you very much, thats fixed it for me. I went via the promiscous mode route and I can access my domain/s again at least.

domino138 · 2023-05-02T17:18:16.298Z

Hi! I'm looking for the recommendation on which NSS build to install (Kong/apccv/ACwifidude).

I have an XR500 and always disable the WiFi radios. Cake QoS on a 400/400mbit connection would be great.

Software-wise I enjoy the netdata package as that gives me some nice bandwidth and cpu charts. I haven't yet found a package that allows me to see which device is using most of the bandwidth at any given time, but that's not important. As long as I can add packages without breaking stuff!

Thanks to all of you for doing this. Without you my XR500 would be in the bin!

AgustinLorenzo · 2023-05-02T18:46:01.181Z

Hi boys,

About the alternative fix for NAT Loopback with ECM enabled, I have updated the patch since if it is executed before the network service is active it does not take effect.

COMMIT UPDATED: https://github.com/AgustinLorenzo/openwrt/commit/9c20f0ebb55972d75376c8594a096c49c868b46c

You can add it in your build, you only have to put the script from this link (https://github.com/AgustinLorenzo/openwrt/blob/master/target/linux/ipq807x/base-files/etc/init.d/misc_fixes) in the directory: /etc/init.d give permissions 755 (chmod 755 /etc/init.d/misc_fixes) and then by CLI execute:

service misc_fixes start

In order for it to do it on every reboot you need to access Luci and follow this path:

LUCI -> System > Startup: find the service misc_fixes with the priority 21 and click on disabled for change to enabled

Give me feedback, thanks guys

Regards, Agustin

xeonpj · 2023-05-02T19:08:41.017Z

acwifidude, we hope you will add both ecm and wifi enhancements to your build!! thank you

apccv · 2023-05-03T04:58:57.738Z

Hey,

No, can't. It is not part of the 22.03 release. I am guessing it was added manually to Snapshot. I have not followed the thread regarding how the image for Chromium ended up being built.

For custom builds you are far better off using the build on top of the thread. The builds I posted have what the official release has (or as close to as I could make it)... tweaking that build to add/drop stuff just makes it a bad copy of ACwifidude's build. His is better. Just wait a little for the next rebase and you will be set.

Cheers!

KurashikiAzusa · 2023-05-03T07:05:44.920Z

Hi,

Edit: After posting this, I troubleshooted further and had the idea of disabling IPv6 (6rd). The problem disappeared and my bufferbloat test is now indicating correct speeds, and is not loading my normal cores. I think the problem lies with that IPv6 is not accelerated by NSS (?!). I truly am speechless.

I am trying out SQM using NSS fq_codel with the following startup script:

modprobe nss-ifb

ip link set up nssifb

# Shape ingress traffic to 500 Mbit with chained NSSFQ_CODEL
tc qdisc add dev nssifb root handle 1: nsstbl rate 500Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

# Shape egress traffic to 500 Mbit with chained NSSFQ_CODEL
tc qdisc add dev eth0 root handle 1: nsstbl rate 500Mbit burst 1Mb
tc qdisc add dev eth0 parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

When I run the bufferbloat test (waveform.com), I observed on htop that one of the normal cores (not the NSS cores) are being fully utilized at 100%, and my 500Mbps up/down bandwidth gets reduced to approximately 230/180.

However, when I run speed test (speedtest.net), I observed on htop that the normal cores are barely utilized, if at all. I get a speed of 450Mbps up/down, a reduction from 500Mbps up/down.

Since my speed gets reduced from 500 to 450, it indicates that the startup script is working. However, when hit with the bufferbloat test, the normal cores (not the NSS cores) are being fully utilized.

If my understanding is correct, the NSS cores have SQM accelerators utilizing NSS fq_codel. Why are the main cores being fully utilized in this situation? Is the SQM working as intended?

I am using ACwifidude's 22.03 latest stable build.

Thank you.

rickkz0r · 2023-05-03T09:13:27.442Z

smalldog:

config queue 'eth1'
        option enabled '1'
        option interface 'eth0'
        option download '85000'
        option upload '10000'
        option qdisc 'fq_codel'
        option script 'nss-rk.qos'
        option linklayer 'none'
        option debug_logging '0'
        option verbosity '5'

Well, your tc qdisc output shows that the SQM settings aren't applied at all. If they work you'd see lines referring to nssfq_codel.

Why it doesn't work for you I don't know. But the script outputs a lot of debugging, you should be able to find clues in the logs.

What's also a bit weird is that your config queue references 'eth1' but in the interface config it says 'eth0'. I'm not sure what the effect of this is is, but I'd try changing that to 'eth0' and the interface to 'eth0.2'.