Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

are you using that scripts?

modprobe nss-ifb

ip link set up nssifb

# Shape ingress traffic to 900 Mbit with chained NSSFQ_CODEL
tc qdisc add dev nssifb root handle 1: nsstbl rate 900Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

# Shape egress traffic to 900 Mbit with chained NSSFQ_CODEL
tc qdisc add dev eth0 root handle 1: nsstbl rate 900Mbit burst 1Mb
tc qdisc add dev eth0 parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

No, I've rolled my own that integrate with the GUI: https://github.com/rickkdotnet/sqm-scripts-nss
However, the result is pretty much the same - my solution just makes it easier to fiddle with the settings.

I recommend you lower the nssfq_codel limit to 1024 and the shaper burst to 1514 bytes though - this makes codel kick in much quicker, preventing latency spikes, especially when a download starts.

3 Likes

The 22.03 build was recently updated. Having quite a bit of trouble with openssl and master. The log is not helpful. In my troubleshooting it doesn't seem to be NSS related. All the updates to switch to mbedtls really did a number (or is it something else?). I've been cutting packages to see what the culprit is. In my .config there isn't a single package of any sort associated with openssl yet the build continues to fail due to openssl not building. I've tried make clean, make dirclean, and make distclean. I've removed wpad-openssl and tried using the default wpad. I've removed all vpn and everything crypto. Anyone having openssl problems with their builds and have suggested ways to fix it?

 make[3] -C package/libs/openssl compile
    ERROR: package/libs/openssl failed to build.

2 Likes

I'm currently testing a NSS master build compiled by myself based on @nihilt post with some customizations. So I'm testing SQM with your script too. After approx. 30 hours I had a crash.
I'll test more to see how it goes.

1 Like

How did this work out for you? Is it stable now with promisc enabled ?
I'm in the same boat (on master / k5.15-qsdk11):

  • Hairpin NAT to the egress interface does not work out of the box like it does with the wlan interfaces: No packets end up on the destination host.
  • Hairpinning does work with promisc enabled on the br-lan.
  • When enabling hairpin mode on eth1.1 I don't see an increase in CPU usage but a) hairpinning still does not work (same behavior, packets are not sent to the destination) AND b) all switched LAN traffic that the cpu sees on eth1.1 (because the switch floods it for example) gets dropped by the router.

Rather weird behavior, I can't get my head around it yet.

friends, the other day for testing I put promisc mode in a last image of acwifidude in kernel 5.10. I have gone 19 days without reboots and without problems on the router. it's better than ever. retest promisc mode in brlan in the latest versions of 5.10.

2 Likes

Does anyone know what is this for

and this

This is what I get with SQM enabled (set at 95%) for my 1GBit/700MBit connection.

1 Like

It provides the nssmirred tc action which mirrors input traffic to another interface. I believe that in earlier versions this was used to create a virtual output interface so you can do SQM on ingress traffic, but this has been replaced with the nssifb interface. You probably don't need it.

1 Like

Installing the latest master I keep getting a very broken unstable luci getting 404s... did firstboot and reset the router through the UI a couple times and attempted reconfigure only for it to break again very quickly. All I'm getting to is enabling wifi and uninstalling things like ddns then it breaks again

now it seems like fq_codel isn't helping latency latency at all, but is still applying the cap. My connection is 30/3 and when I set SQM to 20/1 it still gets ridiculous amounts of bufferbloat, it doesn't seem to matter how much I cap my connection. With cake I'm able to get A+ bufferbloat but I can't get above D no matter what on this nss build now... about half a year ago I was using these builds with the same connection without issue

1 Like

As I don't see any visible improvements (according to the page results) and I don't feel there are any real life improvements with SQM enabled, I just start thinking that at some point the final results given by the site depend more on the ISP connection bufferbloat, rather than the SQM capabilities of the router.

If you want help with this I suggest you post the output of tc -s qdisc.

But it's my experience on master that you need to remove and re-add the nssifb interface (or reboot) every time you make changes - if you change the qdisc parameters on the fly it won't throw any errors but it will not work. I wouldn't be surprised if that's your issue.

2 Likes

Hi @ACwifidude , tested v5.15 branch on both R7800 and EA8500, the code with a problem:

The network speed drops to <100Mb in several hours (Wan & Lan). After reboot, recovered to 1Gb.

The 22.03 branch is stable, no such problem.

1 Like

Is it safe to install the latest stable version for R7800? I see some issues reported above and wonder if I should update or not?

1 Like

I’m having issues compiling the 5.15 branch due to errors with compiling openssl. There should be some stability improvements with the updates over the last 2 months but I’m unable to finish the compile.

The 22.03 branch is very steady.

1 Like

Thanks, I compile your 5.15 branch with a new cloned code, no any errors.

When you rebase it to current master it complains about openssl. Some combination of packages in the diffconfig is causing complaints about openssl. I’ve excluded the new mbedtls defaults, I’ve tried extensive cleaning - both with no success. Haven’t had time to further troubleshoot why it is so grouchy. :grin:

If anyone has any diffconfig recommendations for changes that work with a newly rebased master builds - I’m open to suggestions.

1 Like

Tried target 2.5ms /interval 50ms by editing the script, then in luci, but it still sets 2ms instead of 2.5ms.
This tc command works fine though.

tc qdisc add dev eth1 root fq_codel limit 1000 flows 1024 quantum 1514 target 2.5ms interval 50ms

With 2ms/eth0 & 2.5/eth1 (50ms) tests are more consistent, less fluctuations.

nss.qos occasionally doesn't start correctly, to fix this I edited both defaults.sh & sqm.conf like Kong's (ricsc) script.

defaults.sh
ALL_MODULES="sch_$QDISC sch_ingress act_mirred cls_fw cls_flow cls_u32 sch_htb nss-ifb"

sqm.conf
SQM_CHECK_QDISCS="nssfq_codel fq_codel codel pie sfq cake"
BusyBox v1.36.0 (2023-03-10 04:55:46 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt SNAPSHOT, r22256+28-14334c222e
 -----------------------------------------------------
root@OpenWrt:~# lsmod | grep nss
nss_ifb                16384  0
ppp_generic            40960  7 ecm,ppp_async,qca_nss_pptp,pptp,qca_nss_pppoe,pppoe,pppox
pppoe                  24576  2 ecm,qca_nss_pppoe
pptp                   28672  2 ecm,qca_nss_pptp
qca_nss_crypto         36864  0
qca_nss_drv           311296  7 nss_ifb,ecm,mac80211,qca_nss_qdisc,qca_nss_crypto,qca_nss_pptp,qca_nss_pppoe
qca_nss_gmac           61440  1 qca_nss_drv
qca_nss_pppoe          16384  0
qca_nss_pptp           16384  0
qca_nss_qdisc          61440  5
root@OpenWrt:~#
root@OpenWrt:~# cat /sys/kernel/debug/qca-nss-drv/stats/virt_if
if_num 35 stats start:

rx_packets = 543
rx_bytes = 294786
rx_dropped = 0
tx_packets = 889
tx_bytes = 123022
tx_enqueue_failed = 0
shaper_enqueue_failed = 0
ocm_alloc_failed = 0
if_num 35 stats end:

if_num 37 stats start:

rx_packets = 231705
rx_bytes = 341802064
rx_dropped = 0
tx_packets = 73580
tx_bytes = 7956049
tx_enqueue_failed = 0
shaper_enqueue_failed = 0
ocm_alloc_failed = 0
if_num 37 stats end:

if_num 39 stats start:

rx_packets = 246911
rx_bytes = 359148987
rx_dropped = 0
tx_packets = 14918
tx_bytes = 16461621
tx_enqueue_failed = 0
shaper_enqueue_failed = 0
ocm_alloc_failed = 0
if_num 39 stats end:

base node stats begin (shown on if_num 39):

active_interfaces = 6
ocm_alloc_failed = 0
ddr_alloc_failed = 0
base node stats end.

root@OpenWrt:~# tc -s qdisc show dev eth0
qdisc nsstbl 1: root refcnt 2 buffer/maxburst 1514b rate 90Mbit mtu 1514b accel_mode 0
 Sent 6577462 bytes 65748 pkt (dropped 0, overlimits 39 requeues 0)
 backlog 0b 0p requeues 0
qdisc nssfq_codel 10: parent 1: target 2ms limit 1001p interval 50ms flows 1024 quantum 300 set_default accel_mode 0
 Sent 6577462 bytes 65748 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 maxpacket 1518 drop_overlimit 0 new_flow_count 33191 ecn_mark 0
 new_flows_len 0 old_flows_len 1
root@OpenWrt:~#
root@OpenWrt:~# tc -s qdisc show dev nssifb
qdisc nsstbl 1: root refcnt 2 buffer/maxburst 1514b rate 500Mbit mtu 1514b accel_mode 0
 Sent 359163391 bytes 247029 pkt (dropped 0, overlimits 1367 requeues 0)
 backlog 0b 0p requeues 0
qdisc nssfq_codel 10: parent 1: target 2ms limit 1001p interval 50ms flows 1024 quantum 1514 set_default accel_mode 0
 Sent 359163391 bytes 247029 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 maxpacket 1518 drop_overlimit 0 new_flow_count 83996 ecn_mark 0
 new_flows_len 0 old_flows_len 2
root@OpenWrt:~#
root@OpenWrt:~# tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc nsstbl 1: dev eth0 root refcnt 2 buffer/maxburst 1514b rate 90Mbit mtu 1514b accel_mode 0
qdisc nssfq_codel 10: dev eth0 parent 1: target 2ms limit 1001p interval 50ms flows 1024 quantum 300 set_default accel_mode 0
qdisc fq_codel 8003: dev eth1 root refcnt 2 limit 1000p flows 1024 quantum 1514 target 2.5ms interval 50ms memory_limit 4Mb ecn drop_batch 64
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth1.1 root refcnt 2
qdisc noqueue 0: dev eth0.2 root refcnt 2
qdisc noqueue 0: dev phy1-ap0 root refcnt 2
qdisc noqueue 0: dev phy0-ap0 root refcnt 2
qdisc nsstbl 1: dev nssifb root refcnt 2 buffer/maxburst 1514b rate 500Mbit mtu 1514b accel_mode 0
qdisc nssfq_codel 10: dev nssifb parent 1: target 2ms limit 1001p interval 50ms flows 1024 quantum 1514 set_default accel_mode 0
root@OpenWrt:~#
root@OpenWrt:~# iw phy0 get codel
CoDeL target:           5 ms
CoDeL interval:         50 ms
CoDeL ECN CE marking:           1
root@OpenWrt:~# iw phy1 get codel
CoDeL target:           5 ms
CoDeL interval:         50 ms
CoDeL ECN CE marking:           1
root@OpenWrt:~# cat /sys/kernel/debug/ieee80211/phy0/aql_txq_limit
AC      AQL limit low   AQL limit high
VO      5000            12000
VI      5000            12000
BE      5000            12000
BK      5000            12000
root@OpenWrt:~# cat /sys/kernel/debug/ieee80211/phy1/aql_txq_limit
AC      AQL limit low   AQL limit high
VO      5000            12000
VI      5000            12000
BE      5000            12000
BK      5000            12000

sqm restart (nss-rk.qos version 20230313a)

58.993375] 46a1dfa3: Reset nexthop successful for net device [eth0].
[   58.998655] NSS IFB transmit callback unregistered
[   58.999085] NSS virtual interface destroyed for dev [nssifb]
[   65.418841] debugfs: File 'virt_if' in directory 'stats' already present!
[   65.426368] Created a NSS virtual interface for dev [nssifb]
[   65.430207] NSS IFB data callback registered
[   65.436013] NSS IFB transmit callback registered
[   65.580486] __nss_qdisc_init[2175]:parent (65536) and TC_H_ROOT (-1))
[   65.580504] __nss_qdisc_init[2176]:root->ops->owner (00000000) and THIS_MODULE (bf62f600))
[   65.580542] __nss_qdisc_init[2177]:NSS qdisc c5ad2200 (type 1) used along with non-nss qdiscs, or the interface is currently down
[   65.592117] 46a1dfa3: Found net device [eth0]
[   65.605930] 46a1dfa3: Net device [eth0] has NSS intf_num [1]
[   65.610361] Nexthop successfully set for [eth0] to [nssifb]
**[   66.147602] br-lan: port 3(phy0-ap0) entered blocking state**
**[   66.147651] br-lan: port 3(phy0-ap0) entered forwarding state**

Edit:
sqm restart (nss-rk.qos 20230314a)

Tue Mar 14 20:39:21 2023 user.notice SQM: Stopping SQM on eth0
Tue Mar 14 20:39:21 2023 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev eth0 ingress
Tue Mar 14 20:39:21 2023 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory
Tue Mar 14 20:39:21 2023 kern.info kernel: [16612.900516] 46a1dfa3: Reset nexthop successful for net device [eth0].
Tue Mar 14 20:39:21 2023 kern.info kernel: [16612.907879] NSS IFB transmit callback unregistered
Tue Mar 14 20:39:21 2023 kern.info kernel: [16612.908241] NSS virtual interface destroyed for dev [nssifb]
Tue Mar 14 20:39:22 2023 user.notice SQM: Starting SQM script: nss-rk.qos on eth0, in: 500000 Kbps, out: 90000 Kbps
Tue Mar 14 20:39:22 2023 kern.err kernel: [16614.101631] debugfs: File 'virt_if' in directory 'stats' already present!
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.101948] Created a NSS virtual interface for dev [nssifb]
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.107543] NSS IFB data callback registered
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.113149] NSS IFB transmit callback registered
Tue Mar 14 20:39:22 2023 user.notice SQM: ERROR: No ip6tables binary found, please install 'ip6tables' or 'ip6tables-nft' to use this script
Tue Mar 14 20:39:22 2023 kern.warn kernel: [16614.215635] __nss_qdisc_init[2175]:parent (65536) and TC_H_ROOT (-1))
Tue Mar 14 20:39:22 2023 kern.warn kernel: [16614.215653] __nss_qdisc_init[2176]:root->ops->owner (00000000) and THIS_MODULE (bf62f600))
Tue Mar 14 20:39:22 2023 kern.warn kernel: [16614.215690] __nss_qdisc_init[2177]:NSS qdisc cae86a00 (type 1) used along with non-nss qdiscs, or the interface is currently down
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.227436] 46a1dfa3: Found net device [eth0]
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.241064] 46a1dfa3: Net device [eth0] has NSS intf_num [1]
Tue Mar 14 20:39:22 2023 kern.info kernel: [16614.245479] Nexthop successfully set for [eth0] to [nssifb]
Tue Mar 14 20:39:22 2023 user.notice SQM: nss-rk.qos was started on eth0 successfully

Kernel log
[16612.900516] 46a1dfa3: Reset nexthop successful for net device [eth0].
[16612.907879] NSS IFB transmit callback unregistered
[16612.908241] NSS virtual interface destroyed for dev [nssifb]
[16614.101631] debugfs: File 'virt_if' in directory 'stats' already present!
[16614.101948] Created a NSS virtual interface for dev [nssifb]
[16614.107543] NSS IFB data callback registered
[16614.113149] NSS IFB transmit callback registered
[16614.215635] __nss_qdisc_init[2175]:parent (65536) and TC_H_ROOT (-1))
[16614.215653] __nss_qdisc_init[2176]:root->ops->owner (00000000) and THIS_MODULE (bf62f600))
[16614.215690] __nss_qdisc_init[2177]:NSS qdisc cae86a00 (type 1) used along with non-nss qdiscs, or the interface is currently down
[16614.227436] 46a1dfa3: Found net device [eth0]
[16614.241064] 46a1dfa3: Net device [eth0] has NSS intf_num [1]
[16614.245479] Nexthop successfully set for [eth0] to [nssifb]
3 Likes

This is my diffconfig file:

# Use "make defconfig" to expand this to a full .config
CONFIG_TARGET_ipq806x=y
CONFIG_TARGET_ipq806x_generic=y
CONFIG_TARGET_MULTI_PROFILE=y
CONFIG_TARGET_DEVICE_ipq806x_generic_DEVICE_netgear_r7800=y
CONFIG_TARGET_DEVICE_PACKAGES_ipq806x_generic_DEVICE_netgear_r7800=""
CONFIG_TARGET_DEVICE_ipq806x_generic_DEVICE_linksys_ea8500=y
CONFIG_TARGET_DEVICE_PACKAGES_ipq806x_generic_DEVICE_linksys_ea8500=""
CONFIG_TARGET_PER_DEVICE_ROOTFS=y

# exfat is patented
CONFIG_BUILD_PATENTED=y

# NSS Drivers
CONFIG_PACKAGE_kmod-qca-nss-drv=y
CONFIG_PACKAGE_kmod-qca-nss-ecm-standard=y
CONFIG_PACKAGE_kmod-qca-nss-gmac=y
CONFIG_PACKAGE_MAC80211_NSS_SUPPORT=y
CONFIG_PACKAGE_kmod-qca-nss-drv-qdisc=y
CONFIG_PACKAGE_kmod-nss-ifb=y
CONFIG_PACKAGE_kmod-qca-nss-drv-tunipip6=y

# Longer waiting for failsafe button push
CONFIG_IMAGEOPT=y
CONFIG_PREINITOPT=y
CONFIG_TARGET_PREINIT_TIMEOUT=5

# Busybox tweaks
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVEHISTORY=y
CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVE_ON_EXIT=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_FLAGS=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_REGEXP=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_WINCH=y

# Add-on programs
CONFIG_DROPBEAR_ECC=y
CONFIG_PACKAGE_openvpn-openssl=y
CONFIG_PACKAGE_htop=y
CONFIG_PACKAGE_kmod-cryptodev=y
CONFIG_PACKAGE_libopenssl-devcrypto=y
CONFIG_PACKAGE_kmod-dnsresolver=y
CONFIG_PACKAGE_kmod-ramoops=y
CONFIG_PACKAGE_kmod-pstore=y
CONFIG_PACKAGE_kmod-reed-solomon=y
# CONFIG_PACKAGE_kmod-pppol2tp is not set
CONFIG_PACKAGE_ds-lite=y
CONFIG_PACKAGE_curl=y
CONFIG_PACKAGE_kmod-usb-printer=y

# USB device mount & file systems support
CONFIG_PACKAGE_block-mount=y
CONFIG_PACKAGE_cryptsetup=y
CONFIG_PACKAGE_e2fsprogs=y
CONFIG_PACKAGE_f2fs-tools=y
CONFIG_PACKAGE_kmod-crypto-ecb=y
CONFIG_PACKAGE_kmod-crypto-xts=y
CONFIG_PACKAGE_kmod-crypto-iv=y
CONFIG_PACKAGE_kmod-crypto-misc=y
CONFIG_PACKAGE_kmod-crypto-user=y
CONFIG_PACKAGE_kmod-fs-exfat=y
CONFIG_PACKAGE_kmod-fs-ext4=y
CONFIG_PACKAGE_kmod-fs-f2fs=y
CONFIG_PACKAGE_kmod-fs-hfs=y
CONFIG_PACKAGE_kmod-fs-hfsplus=y
CONFIG_PACKAGE_kmod-fs-msdos=y
CONFIG_PACKAGE_kmod-fs-nfs=y
CONFIG_PACKAGE_kmod-fs-nfs-common=y
CONFIG_PACKAGE_kmod-fs-nfs-v3=y
CONFIG_PACKAGE_kmod-fs-nfs-v4=y
CONFIG_PACKAGE_kmod-fs-vfat=y
CONFIG_PACKAGE_kmod-nls-base=y
CONFIG_PACKAGE_kmod-nls-cp1250=y
CONFIG_PACKAGE_kmod-nls-cp437=y
CONFIG_PACKAGE_kmod-nls-cp850=y
CONFIG_PACKAGE_kmod-nls-iso8859-1=y
CONFIG_PACKAGE_kmod-nls-iso8859-15=y
CONFIG_PACKAGE_kmod-nls-utf8=y
CONFIG_PACKAGE_kmod-usb-storage=y
CONFIG_PACKAGE_kmod-usb-storage-uas=y
CONFIG_PACKAGE_libblkid=y
CONFIG_PACKAGE_ntfs-3g=y
CONFIG_PACKAGE_nfs-utils=y

# IPv6 support
CONFIG_PACKAGE_6in4=y
CONFIG_PACKAGE_6to4=y
CONFIG_PACKAGE_6rd=y

# WLAN/WPS support
CONFIG_PACKAGE_hostapd-utils=y
CONFIG_WPA_MSG_MIN_PRIORITY=4
CONFIG_PACKAGE_wpad-openssl=y
# CONFIG_PACKAGE_wpad-basic-wolfssl is not set
# CONFIG_PACKAGE_libustream-wolfssl is not set

# SSL certificates
CONFIG_PACKAGE_ca-certificates=y

# Luci (SSL from OpenSSL)
CONFIG_PACKAGE_luci-ssl-openssl=y
CONFIG_PACKAGE_luci-app-commands=y
CONFIG_PACKAGE_luci-app-ddns=y
CONFIG_PACKAGE_luci-app-wireguard=y
CONFIG_PACKAGE_luci-app-samba4=y
CONFIG_PACKAGE_luci-app-simple-adblock=y
CONFIG_PACKAGE_luci-app-ttyd=y

CONFIG_DEVEL=y
CONFIG_CCACHE=y
CONFIG_PACKAGE_kmod-mtd-rw=y

CONFIG_PACKAGE_kmod-ath10k-ct=y
CONFIG_PACKAGE_kmod-ath10k=m
CONFIG_PACKAGE_ath10k-firmware-qca99x0-ct=m
CONFIG_PACKAGE_ath10k-firmware-qca9984-ct=m
CONFIG_PACKAGE_ath10k-firmware-qca99x0=m
CONFIG_PACKAGE_ath10k-firmware-qca9984=m

# CONFIG_PACKAGE_kmod-nls-cp1250 is not set
# CONFIG_PACKAGE_kmod-nls-cp850 is not set
# CONFIG_PACKAGE_kmod-nls-iso8859-15 is not set
CONFIG_PACKAGE_kmod-usb-storage-extras=y
CONFIG_PACKAGE_kmod-usb-uhci=y

# CONFIG_FEED_telephony is not set

CONFIG_PACKAGE_wget-ssl=y
CONFIG_PACKAGE_bind-host=y
CONFIG_PACKAGE_ddns-scripts-noip=y

CONFIG_PACKAGE_openssh-sftp-server=y
CONFIG_PACKAGE_samba4-admin=y
CONFIG_PACKAGE_samba4-utils=y
CONFIG_PACKAGE_usb-modeswitch=y

one more question, how to mod my diffconfig file for ath10k build, I've tried to install ath10k drivers replaced ath10k-ct on my router(-ct), unsuccessful.

I gave up on it. I'm running an old master build without NSS, but with all other tweaks I could find instead. It's been stable as a rock and no one complains about the Wi-Fi speed anymore.