Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

Hey, I freshly installed your custom build to reach the 1000Mbps my connection allow. It works great, thanks. I'm new to the OpenWRT community as well, just installed it on my R7800.

I've been fiddling around with the SQM settings, and I'm not sure I understand your note:

  • Luci SQM and normal SQM config doesn’t work. Have to use custom config.

What does that mean? The configuration done in the UX is ignored? Where is that custom config? I looked at the /etc/config/sqm file, and it seems it's properly written upon changes in the UX. So maybe there is another file I'm missing.

FYI, here are my test results with waveform:

Directly plugged Ethernet to my ISP modem "Orange" in France, (so ISP -> Computer):

** Plugged Ethernet on the R7800 (so ISP -> R7800 -> Computer)
SQM Disabled in the UX, plugged in Ethernet on the R7800

SQM enabled in the UX, plugged in Ethernet on the R7800 - cf screenshots for configuration of SQM

As you can see, the results are sensibly identical with the last 2 links, whether SQM is enabled or not.
Is there something I'm missing here?

I tried to put absurd configuration as well in SQM UX, it didn't change a thing.

What I'm trying to optimise is lag I have in long video calls in Google Hangout, after ~15 minutes I have a 1 second delay between what I see and what I hear, and also I tend to cut people speaking because I don't hear them speaking... It get way worse in long calls.

Silly question: appart from clicking on "Save & Apply", I do not have to reboot, correct?

Thanks!

Settings, I basically followed this wiki page: https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details#a_little_about_tuning_sqm

Build: SNAPSHOT r21373+13-aa12a0fdd1 with ath10k driver.

  1. Custom NSS fq_codel startup script if SQM is desired (below is set to 900/900). Recommend setting to 5% below your ISP provided speed and then adjusting from there as per your testing. Maximum is ~900Mbit.
modprobe nss-ifb

ip link set up nssifb

# Shape ingress traffic to 900 Mbit with chained NSSFQ_CODEL
tc qdisc add dev nssifb root handle 1: nsstbl rate 900Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

# Shape egress traffic to 900 Mbit with chained NSSFQ_CODEL
tc qdisc add dev eth0 root handle 1: nsstbl rate 900Mbit burst 1Mb
tc qdisc add dev eth0 parent 1: handle 1
1 Like

has anyone tried the qca-nss-cfi package ? it should help accelerate vpn together with qca-nss-crypto

In which file should I save these commands and do I need to make any changes for pppoe?

Oh wow I didn't notice the post below the original with the instructions "how to replicate my build".

Thanks a lot.

1 Like

Hi all,
I have been trying to build the Master with rebase but im running into some compiling errors
Anyone has an idea? Diffconfig is basicly the one from OP but only for R7800.

Error log:

ERROR: modpost: "nss_crypto_pm_notify_register" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
ERROR: modpost: "nss_crypto_notify_register" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
ERROR: modpost: "nss_crypto_pm_notify_unregister" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
ERROR: modpost: "nss_crypto_tx_msg" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
ERROR: modpost: "nss_crypto_data_register" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
ERROR: modpost: "nss_crypto_tx_buf" [/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/v1.0/src/qca-nss-crypto.ko] undefined!
make[5]: *** [scripts/Makefile.modpost:133: /home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/Module.symvers] Error 1
make[5]: *** Deleting file '/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/Module.symvers'
make[4]: *** [Makefile:1818: modules] Error 2
make[4]: Leaving directory '/home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.15.86'
make[3]: *** [Makefile:80: /home/tillo/git/r7800/openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-crypto-2021-03-20-2271a3a/.built] Error 2
make[3]: Leaving directory '/home/tillo/git/r7800/openwrt/feeds/nss/qca-nss-crypto'
time: package/feeds/nss/qca-nss-crypto/compile#2.09#0.22#2.32
    ERROR: package/feeds/nss/qca-nss-crypto failed to build.
make[2]: *** [package/Makefile:116: package/feeds/nss/qca-nss-crypto/compile] Error 1
make[2]: Leaving directory '/home/tillo/git/r7800/openwrt'
make[1]: *** [package/Makefile:110: /home/tillo/git/r7800/openwrt/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/tillo/git/r7800/openwrt'
make: *** [/home/tillo/git/r7800/openwrt/include/toplevel.mk:231: world] Error 2

Thanks in advance!

Under System > Startup > Local Startup then paste the code there. I don't think you need to do anything for pppoe

No problem. I believe you must disable any software SQM and software offloading as well. After pasting the script in Local Startup you then restart your router.

I know @quarky was working hard on those packages in his thread.
From my understanding there were some issues but eventually it did work at some point.

1 Like

I've just got a reboot and put the details here.

Updated the master and 22.03 builds. Let me know how they are working. :sunglasses:

10 Likes

thanks mate!
crossing my fingers to see if i can build python with this..

5.15 is back, do you know if they work now on ea7/8500?

EA7500/EA8500 have a major issue with the 5.15 kernel unfortunately. This problem has not been resolved in master.

If you have one of these devices I recommend using 22.03 until a solution is found for master.

2 Likes

Dependency problem: One needs to include the following packages via .config.
CONFIG_PACKAGE_kmod-cryptodev=y
CONFIG_PACKAGE_libopenssl-devcrypto=y

2 Likes

I've previously seen posts reporting about the bad/low performance of the 2.4GHz radio.
That's why I want once again to share my current experience with latest master build with ath10k drivers.


This time the test was performed with my new Motorola Edge 30 Fusion (Chipset Qualcomm SM8350 Snapdragon 888+ 5G (5 nm); WLAN Wi-Fi 802.11 a/b/g/n/ac/6e).
I've never seen such a good 2.4GHz performance with any other router so far. :slightly_smiling_face:
On 5GHz frequency I have over 500 Mbps up/down.

1 Like

What's your wireguard speed on ipq8065, guys? I have only 130-140, max 160 Mbit. I had 160 Mbit with mt7621. Earlier I assumed that it has less CPU power that ipq8065. nbg6817, 22.03 nss.

As my luck would have it, just as I finished flashing everything with the build I posted, 22.03.3 was released. I am in no rush to flash again but in case anyone wants to try that version out, here it is:

Just as before this image is a rebuild of the 22.03.3 release (22.03.3 tagged build) build plus:

  • The 31 NSS commits on ACwifidude's repository
  • Minimal changes to three files to allow all modules to build (this is in a commit of its own)
    -- package/qca/qca-nss-clients/Makefile (remove non building packages)
    -- package/qca/qca-nss-ecm/Makefile (ditto)
    -- target/linux/ipq806x/patches-5.10/999-07b-qca-nss-clients-qdisc-support.patch (remove non building patches to GRE)
  • the 22.03.3 config.buildinfo (the 22.03.3 diffconfig) with additional NSS packages (checked in as diffconfig-nss)

Images and all kernel modules in bin directory.

Also as before, GitHub can be used as a repo for this build's modules. For that:

  • copy the 242547a2d8baa8b1 file in the opkg-key directory to /etc/opkg/keys
  • update the /etc/opkg/distfeeds.conf to use repo instead, as in:
#src/gz openwrt_core https://downloads.openwrt.org/releases/22.03.3/targets/ipq806x/generic/packages
src/gz openwrt_nss https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages
src/gz openwrt_base https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/base
src/gz openwrt_luci https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/luci
src/gz openwrt_packages https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/packages
src/gz openwrt_routing https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/routing
src/gz openwrt_telephony https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/telephony

opkg again should install nicely after that.

The ImageBuilder gets built fine and with the opkg changes above works like a charm. It's not in the repo as it is way too large to check in.

I've been running the 22.03.2 NSS build for a month on TP-Link C2600 without major issues. Just two blips, both at 30 days uptime, and not sure they were NSS related:

  • Main router rebooted by itself. Pretty weird. Has never happened before. Just died and came back minutes later as if nothing had happened. Nothing on logs as logs did not survive the reboot. I'm guessing this was more of a power issue.
  • An AP started having routing issues. So weird I find it tough to describe. Some wireless clients were suddenly not able to connect to others but only for certain services. For example, VNC worked. ssh didn't. Ping reported "no route to host". Sometimes it was possible to connect to a wireless client by using a third wireless client as an intermediary. Nothing weird on logs. No apparent memory issues. A restart of the wireless interface did not fix the issue. A reboot did.

All in all though we are having less issues with the NSS build than with the normal one. The only tweak we have in place is this, as suggested by ACwifidude in rc.local:

# CPU governor fixes
echo 600000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
echo 600000 > /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq
echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

The next thing I might try is setting up two WAN links and using mwan3 to see if we can push past the 1GBE limit.

Thanks again for such all the work to get NSS working!

EDIT: Flashed one C2600 with 22.03.3-NSS and set it as a wireless client (wlan) to a C2600 AP running 22.0.3.2-NSS. Put a laptop on the LAN of the C2600 acting as client and ran speedtests. Not using SQM anywhere. This happened.

LUCI:

Wireless link:

fast dot com (the numbers are never quite right, but it does make a nice picture):

speedtest:

bufferbloat (not the greatest score but maybe due to how much this was pushing the WiFi board):

CPU usage during one test, first dip is download, second upload:
CPU

It might have been a bit too much for the thing as this was in the AP logs:

[436920.972097] net_ratelimit: 8 callbacks suppressed
[436920.972102] wlan0: NSS TX failed with error: NSS_TX_FAILURE_QUEUE

This was on the wireless client logs:

[ 2391.409835] wlan0: NSS TX failed with error: NSS_TX_FAILURE_QUEUE
[ 2391.410144] wlan0: NSS TX failed with error: NSS_TX_FAILURE_QUEUE

Whatever that was, both recovered and are working fine. So... YMMV...

4 Likes

Thanks apccv
:slight_smile:

Hello again gentlemen.
Just a brief suggestion, as I know I risk to be naive, at the very least.

For all the users of this marvelous FW that are afraid of making the own custom firmwares, I wonder if it is possible to develop a custom version of this very useful tool from OWRT
So that, everyone understanding the benefits of @ACwifidude 's work could furthermore optimize the firmware without the need of a build environment.
Some might say that learning how to build is necessary for the full comprehension of openwrt, and they're right.
Some others might say it is not possible, and I'd be interested to know why, as I guess the remote builder is opensource.

But still, such a tool might simplify a lot the widespread adoption of this precise fork of owrt for the supported hardware, as it squeezes out all the possible efficiency of the routers.

Hope this is welcome as a mere discussion element, as I know I'm only a rookie and I don't know much more than simple howto copy-paste and GUI screens! :sweat_smile:

Humbly yours,
Gabriel

1 Like