Qualcommax NSS Build

csharper2005 · 2023-02-06T19:15:48.222Z

I found the root cause. My ethernet adapter was mistakenly connected to the USB 2.0 port. Now I have ~950 MBit.

McGiverGim · 2023-02-07T15:50:05.991Z

One question: does this NSS build, at this actual state, accelerate PPPoE protocol? Is what all the ISPs use here in Spain.

To install it, can I simply sysupgrade from official build to NSS build and inverse?

rmandrad · 2023-02-08T05:05:56.038Z

I am not making use of PPPoE so can't comment on the benefits .. worth maybe searching the ipq806x nss community build .. but both the nss-drv and nss-ecm are compiling with PPPoE enabled if you check both Makefiles

note you will need to force the sysupgrade as the keys will be different..

larrynz · 2023-02-08T06:05:41.271Z

I am using with pppoe. Offloading works over LAN.

McGiverGim · 2023-02-08T07:27:04.772Z

What speed do you have with the ISP? With a speed test what is the maximum CPU value that you reach?
Without NSS, and 300/300 connection, a simple speedtest goes to over 50-60% at one of the CPUs. Other tests that I've seen maxes it with 1Gbps connections.

larrynz · 2023-02-08T07:41:52.753Z

I have a 1GBs/500MBs plan.

This is what I am getting. Max out at 940 DL / 520 UL.

https://forum.openwrt.org/t/adding-openwrt-support-for-xiaomi-ax3600-part-1/55049/9043

McGiverGim · 2023-02-08T07:46:46.624Z

If I see your capture right, a test from LAN to Internet, can you reach 940 DL with all the CPUs almost idle? That's an incredible test that this works for sure
The only problem is that I suppose that I need to build it by my own, with all the packages/kmods needed.

larrynz · 2023-02-08T07:49:38.486Z

That's right, almost idle

McGiverGim · 2023-02-08T08:09:33.581Z

@dimfish in the old AX3600 thread I think was you who provided an image with all the kmods and packages. Am I wrong? If yes, are you interested in doing the same for the NSS build?

Regards!

larrynz · 2023-02-08T08:40:31.660Z

Updated repo

With more kernel options. We almost need a guide - at least what not to include.

Screenshot from 2023-02-08 21-37-581719×1369 285 KB

rmandrad · 2023-02-08T11:11:39.560Z

in case you want to use nss-qdisc / nss-ifb here's some instructions

# Load the module  

insmod nss-ifb nss_dev_name= 

  example insmod nss-ifb nss_dev_name=10g-1

# Bring up the nssifb interface to active ingress redirect
ip link set up nssifb

# Shape ingress traffic to 1024Mbit with chained NSSFQ_CODEL
tc qdisc add dev nssifb root handle 1: nsstbl rate 1024Mbit burst 1Mb
tc qdisc add dev nssifb parent 1: handle 10: nssfq_codel limit 10240 flows 1024 quantum 1514 target 5ms interval 100ms set_default

note - I haven't found this useful ... I am using mwan3 which is using two wan devices load balanced. At the moment nss-ifb only supports one device ...

dimfish · 2023-02-09T15:04:10.672Z

Tried and got errors:

ERROR: modpost: "nss_ipsec_cmn_get_ifnum_with_coreid" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/netlink/qca-nss-netlink.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_get_context" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_tx_msg_sync" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_tx_msg" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_ppe_port_config" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_notify_register" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_msg_init" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_register_if" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_unregister_if" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
ERROR: modpost: "nss_ipsec_cmn_ppe_mtu_update" [/data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipsecmgr/v2.0/qca-nss-ipsecmgr.ko] undefined!
WARNING: modpost: suppressed 3 unresolved symbol warnings because there were too many)
make[5]: *** [scripts/Makefile.modpost:133: /data/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/Module.symvers] Error 1

bitthief · 2023-02-09T15:40:26.720Z

No problem, happy to contribute to the community!

Here you go:

Pastebin

IPQ807X NSS .config - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

bitthief · 2023-02-09T15:44:23.602Z

rmandrad:

fyi --- regarding the nssinfo utility ... in order to work one has to insmod/modprobe qca-nss-netlink

I wrote an init script for this now, which I hope works, it should be installed as /etc/init.d/qca-nss-netlink (part of the qca-nss-drv-netlink package), it basically does insmod and sets the n2h_queue_limit_core0/core1 to 2048, I got inspired from your sysctl post

rmandrad:

note - I haven't found this useful ... I am using mwan3 which is using two wan devices load balanced. At the moment nss-ifb only supports one device ...

Ah nice, thanks a lot for this, I'll try it soon as I'm seeing some buffer bloat due to my ISP!

I also use mwan3, but mainly to do split-routing for the WAN and 3 other WireGuard VPN interfaces (different networks go out through a different VPN tunnel basically, one of them is pushed out via WAN), so in my case I think I could at least attach it to the physical WAN interface and everything else should behave better as a side effect. WireGuard should technically have built-in QoS, as part of the protocol iirc, so a stable uplink would help the tunnels also.

I think we could definitely add support for more interfaces, it's a super basic module, mainly calling:
nss_virt_if_create_sync_nexthop, nss_virt_if_register and nss_virt_if_xmit_callback_register from the NSS virtual interfaces driver, to register some hooks and stuff. Additionally, you'd need to change the alloc_netdev call, which sets the nssifb interface name. Nothing too fancy, just need to come up with a way to pass multiple interfaces to it, and it'd create nssifb0, nssifb1, nssifb2 or maybe the source interface name with ifb prepended, like nssifbeth0, nssifb10g-1 etc.
Most likely multi-WAN wasn't too popular when this module was written / ripped off from the kernel, it seems to be quite old haha

I just looked at https://github.com/torvalds/linux/blob/master/drivers/net/ifb.c, which is the source of this module, and this already handles multiple IFB devices, as you can see.. it just calls the init function multiple times

rmandrad:

Tried and got errors:

Thanks for pointing these out, the ipsec mgr NSS drv module needs a few more exported symbols and functions in the kernel by the looks of it, I don't use it so I didn't trigger these. I'll have a look in a bit and fix them, you can try a build without ipsecmgr or wait for my fixes!

bitthief · 2023-02-09T16:02:38.776Z

Yeah, I ported the remaining NSS drivers and managers etc., it made sense to have them all in there if I went through the effort to have some working already Also added a few more fixes for the crypto offload and other good stuff, such as the 2.2GHz CPU frequency overclock.

Haha, there is a lot of them for sure, not even I know how many we actually need, but it depends on everyone's network setup. I know how each of these modules hooks into the kernel networking stack with notifiers and hooks etc., but I don't fully understand the flow to be honest, especially for the lower level multicast, NAT, TLS etc. modules. I enable most of them in my build, excluding the GRE and IPSEC tunnels, which I don't use.

I'm seeing some really noticeable LAN traffic improvements in terms of latency, for example browsing to the LuCI interface or pihole is SIGNIFICANTLY snappier now, to the point where I can notice it! I suspect this is coming from the TLS or DTLS mgr modules, or the crypto offload.
Also pushed past 420MBps for WireGuard tunnels, when my ISP isn't throttling too hard, haven't seen that before either.
I finally feel that we're maxing out the performance of this box, as it's designed to perform! Pretty satisfied with what we have now, in terms of performance and stability etc.

Play around with them and see how everything works out for you!

Crect · 2023-02-09T18:44:55.722Z

bitthief:

2.2GHz CPU frequency overclock

Couldn't that potentially decrease the lifespan of the device?

McGiverGim · 2023-02-09T21:34:54.352Z

Is there some problem for enabling all of them? I suppose it's the best option, except if we find some bug at some of them.

michaelchen644 · 2023-02-10T02:56:17.440Z

I didn't overwrite these duplicate packages with the -f parameter. But nss also works as I expected.

larrynz · 2023-02-10T04:04:35.504Z

Cheers will do.

Thanks for posting your config file. This helps as we can see which kernel modules you have selected. Will try these out in a new build

McGiverGim · 2023-02-10T13:33:34.697Z

I think the way to go is to "provide" a "default" config with the kernel modules that work or we think that work. In this way it can be tested by the community, including people like me that don't know too much about this, and confirm if it is stable or there is some bug. This default config I think is better to add too luci and the full wpad version (to let users with mesh AP to upgrade without loosing connectivity). The rest is better to maintain like default OpenWrt.

For othe part, in this thread can be discussed other modules to add for advanced users, and once tested by this users, can be added to the "default" config.

What do you think? Is this a good approach?

PD: I've added a link to this thread in the wiki, to see if more people is interested.