Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

Updated repo with CONFIG_ALL_KMODS=y

Unfortunately the kernel hash is still different (probably because of the NSS kmods) and doesn’t allow the install of kmods in a live system.

Otherwise the build has all the latest master mac80211 updates and the updates to the ipq806x cache driver + CPU. :sunglasses:

2 Likes

So it compiles without issue with this and if I install a package, I may need to force install it but the KMODs should all be built ?

ok, gonna have a new build :slight_smile:
in the meantime, i'm trying to include @KONG SQM scripts. I started adding the 2 config files for luci, i had to add a modprobe for nssifb (probably Kong put it as a default..) but atm i have still also some errors in syslog, need to check those also.. ideas?
the point is that i'm not sure how to test if SQM is working, since i already have a big green A on dsl reports..

Thu Jul  1 14:34:11 2021 user.notice SQM: Starting SQM script: nss.qos on eth0, in: 900000 Kbps, out: 90000 Kbps
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 0 u32 match ip protocol 1 0xff flowid 1:13
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Not supported We have an error talking to the kernel
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 match ip protocol 1 0xff flowid 1:13
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Not supported We have an error talking to the kernel
Thu Jul  1 14:34:13 2021 user.notice SQM: WARNING: sqm_start_default: nss.qos lacks an egress() function
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc add dev nssifb root handle 1: nsstbl rate 900000kbit burst 1Mb
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: File exists
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 4096 flows 1024 quantum 1514 target 5ms interval 100ms set_default
Thu Jul  1 14:34:13 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: File exists
Thu Jul  1 14:34:14 2021 user.notice SQM: nss.qos was started on eth0 successfully```

Just stopped by to say thanks. Excellent work. This finally gave me a gig on the WAN with the R7800, without the governor or affinity shenanigans that max out at 600mbit anyway.

Going to roll a custom build with all the extra stuff that I always disable and will come back in a month with a stability update.

1 Like

I similarly have had difficulty. I don’t quite know how to integrate the .QoS file in correctly for NSS to work with luci (the manual input script above works but doesn’t afford the ease of the luci SQM page). If anyone has any tips for getting the NSS QoS in as an option in to the luci SQM page - I’d love to put it in as a commit to permanently have that capability.

What is nice is that with a gig connection the bufferbloat is practically nothing. Where it would help is for asymmetric connections (DOCSIS) with lousy upload speeds that tend to create some bufferbloat.

1 Like

nss.qos live system

rc.local: delete all nss related commands/script.

cd /usr/lib/sqm/
wget https://raw.githubusercontent.com/ricsc/sqm-scripts/master/src/nss.qos
wget https://raw.githubusercontent.com/ricsc/sqm-scripts/master/src/nss.qos.help


nano defaults.sh
replace
ALL_MODULES="act_ipt sch_$QDISC sch_ingress act_mirred cls_fw cls_flow cls_u32 sch_htb sch_hfsc"

with
ALL_MODULES="act_ipt sch_$QDISC sch_ingress act_mirred cls_fw cls_flow cls_u32 sch_htb sch_hfsc nss-ifb"


nano /etc/sqm/sqm.conf
replace
SQM_CHECK_QDISCS="fq_codel codel pie sfq cake"

with
SQM_CHECK_QDISCS="nssfq_codel fq_codel codel pie sfq cake"

Set sqm interface to eth0 and fq_codel with nss.qos

reboot.
1 Like

Is this working for you?
i got the same errors in my syslog

Fri Jul  2 08:27:51 2021 user.notice SQM: Starting SQM script: nss.qos on eth0, in: 900000 Kbps, out: 90000 Kbps
Fri Jul  2 08:27:51 2021 kern.err kernel: [  180.810197] debugfs: File 'virt_if' in directory 'stats' already present!
Fri Jul  2 08:27:51 2021 kern.info kernel: [  180.810528] Created a NSS virtual interface for dev [nssifb]
Fri Jul  2 08:27:51 2021 kern.info kernel: [  180.816042] NSS IFB data callback registered
Fri Jul  2 08:27:51 2021 kern.info kernel: [  180.821800] NSS IFB transmit callback registered
Fri Jul  2 08:27:51 2021 kern.info kernel: [  180.826673] NSS IFB module loaded.
Fri Jul  2 08:27:52 2021 daemon.err insmod: module is already loaded - nss-ifb
Fri Jul  2 08:27:53 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc filter add dev eth0 parent 1:0 protocol ip prio 0 u32 match ip protocol 1 0xff flowid 1:13
Fri Jul  2 08:27:53 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Not supported We have an error talking to the kernel
Fri Jul  2 08:27:53 2021 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc filter add dev eth0 parent 1:0 protocol ipv6 prio 1 u32 match ip protocol 1 0xff flowid 1:13
Fri Jul  2 08:27:53 2021 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: Not supported We have an error talking to the kernel
Fri Jul  2 08:27:53 2021 user.notice SQM: WARNING: sqm_start_default: nss.qos lacks an egress() function
Fri Jul  2 08:27:53 2021 kern.warn kernel: [  182.508094] nss_qdisc_init[1993]:NSS qdisc 3fde5d55 (type 1) used along with non-nss qdiscs, or the interface is currently down
Fri Jul  2 08:27:53 2021 kern.info kernel: [  182.532938] 7ad670d5: Found net device [eth0]
Fri Jul  2 08:27:53 2021 kern.info kernel: [  182.543304] 7ad670d5: Net device [eth0] has NSS intf_num [1]
Fri Jul  2 08:27:53 2021 kern.info kernel: [  182.568341] Nexthop successfully set for [eth0] to [nssifb]
Fri Jul  2 08:27:54 2021 user.notice SQM: nss.qos was started on eth0 successfully```

if CONFIG_ALL_KMODS=y is on, i saw the diffconfig, why are individual kmods also listed? just want to understand. I thought once this is on, it builds everything.

CONFIG all sets all kmods from “is not set” to “=m” (manual). It sets the config in a sort of “ready to go” but doesn’t install the packages (unless you specify “=y” or another package / part of the config requests that it be installed).

make defconfig will build out a complete .config file and pull in all the required packages depending on your selections. So put a “=y” for the packages you desire. A shortcut most of the time is if there is a luci module for what you are adding - you can say “=y” for the luci packages for the capability you desire and it’ll pull in all the required backside packages.

Unfortunately the config all kmods selection didn’t make a similar hash to master. I believe that is due to the nss kmods. I’m not sure if the solution is adding in the other master seed options (will change the hash?) or if the nss kmods are changing the hash and there is no solution to get a compatible hash to be able to install kmods in a live build.

You probably didn't delete modprobe nss-ifb, ip link xxx incl. tc commands from rc.local (startup) as instructed above and/or new SQM settings (e.g. download/upload) were applied with SQM already running. Just reboot each time you change settings.

well, consider my rc.local is this:

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

exit 0

:slight_smile:
but i see it shows less errors on reboot, so not optimal but it seems it works..
no idea of the source of errors when modifying SQM with no reboot.. maybe @KONG knows?

1 Like

Is there any chance all these mods will make it to OpenWRT master ? That would mitigate the issue as well i assume if they adopt it?

Correct me if I am wrong.

The hash is tied to the kernel in the image you compile so it will never be the same as snapshot image.

The hash as far as I understand is derived from the kernel + configured kmods. The config all kmods = y develops the same hash as master….unless you have all the extra NSS kmods.

Not sure if any of the other features in the master config seed will make a difference. I think the hash will never be the same because of the NSS kernel modifications.

1 Like

Sorry didn't follow up on this, but what is the reason for this custom stuff, why not just use my sqm-scripts package? You can install from my repo or, pull in the code, when you do a custom build.

If you want to include in a custom build, just, change the sqm-scripts Makefile:

PKG_NAME:=sqm-scripts
PKG_SOURCE_VERSION:=27fb04f4a49b80e8ccde27c7587ce2a63bd8ac43
PKG_VERSION:=1.5.1
PKG_RELEASE:=nss

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/ricsc/sqm-scripts.git
PKG_MIRROR_HASH:=d41301ed1e318ea81c6c8f29c1847efdda3663573d12a3e0b855b4b8b8cf0610

3 Likes

Thank you for these NSS builds! I prefer Netgear routers and was a bit let down that I had to look for a different brand to fully utilize my 1Gbit connection. Fast forward two years and I'm happy to learn these builds are now available, so thanks again. I immediately bought a R7800 and it seems to be running well on the latest master build.

A question for you @ACwifidude; Are there plans to make the NSS work part of the standard firmware image? Or do the patches deviate from the main source in such a way, that it could never be part of the main distribution/source tree?

I tried to reproduce the build (with CONFIG_ALL=y and IGNORE_ERRORS="n m") on the openwrt-21.02-nss-qsdk10.0 branch, and failed. The failed package is package/qca/qca-nss-clients, and it fails because it tries to include stuff from qca-ssdk which is not provided by any package. @ACwifidude looks like either a forgotten instruction to fully disable package/qca/qca-nss-clients, or a forgotten inclusion of qca-ssdk.

Why do you need to use CONFIG_ALL=y?

SSDK is not included in this for IPQ806X for now but it works fine without it.

CONFIG_ALL=y is needed so that I can build everything and sidestep the current breakage in the official repositories. See https://lists.openwrt.org/pipermail/openwrt-devel/2021-July/035741.html

Regarding the SSDK situation, OK, it is not included, but let me check my understanding.

Right now in the suggested minimal config (in the first comment, to be expanded by make defconfig), there is CONFIG_PACKAGE_kmod-qca-nss-drv-pppoe=y. This package comes from package/qca/qca-nss-clients, and the minimal config uses nothing else from there, but CONFIG_ALL sets them all to m.

If I understand the build system correctly, the Makefile tries to build exactly the modules that are requested in OpenWRT .config, but some of them fail because they have a #include <fal/fal_*.h> which is in SSDK. They are all m because of CONFIG_ALL=y, but IGNORE_ERRORS="n m" for some reason does not ignore this failure. It does successfully ignore failures in other packages that have no subcomponents set to y. So I have to explicitly disable CONFIG_PACKAGE_kmod-qca-nss-drv-capwapmgr, CONFIG_PACKAGE_kmod-qca-nss-drv-lag-mgr, and CONFIG_PACKAGE_kmod-qca-nss-drv-vlan-mgr.

Edit: there is also CONFIG_PACKAGE_kmod-qca-nss-drv-bridge-mgr, but the Makefile marks it as unavailable for IPQ806X via the DEPENDS clause. So maybe the whole issue is that all other SSDK-dependent modules do not have the proper DEPENDS clause which limits them to the same targets?

Right?

Possibly not a problem of this fork, please point me in the right direction if so.

I noticed that the builds I produce by running

./scripts/feeds update -a
./scripts/feeds install -a
cp diffconfig .config
make defconfig
make -j4 download
make -j4

Won't boot unless flashed via sysupgrade without clearing the settings on the router. Naturally, same thing happens if I flash via TFTP or do a factory reset after flashing via sysupgrade.

Other than this, the builds work fine bar some WiFi stability issues. Is there a step missing from the instructions to reproduce the builds? Is there something that needs to be done for the correct "factory defaults" to be included in the image?