Yes, I know, that's why I apologized to you.
Seriously no problems on my part.

I just wanted to report an existing issue in the main openwrt branch.
But it's true. This is not the right place to do it.

Sorry again.

EDIT:
Issue open on openwrt github

WLAN<-->WAN



WLAN<-->LAN


The upload is a little slower, but that's due to me being 2 rooms over from the router.

Running latest build + bumped kernel to 6.6.26

1 Like

@asvio these are my results. Connected over wifi to the router

Question: I remember seeing somewhere in this thread that it is recommended to disable flow offloading and packet steering...what about bridger?

Bridger is a bit of a mystery to me...all I really know about what it does is that it uses some sort of eBPF magic to make bridges faster.

Would bridger work with NSS or would it prevent NSS from being used for bridge traffic?

NSS is already able to achieve this. You actually wouldn't want the kernel to process anything if you have dedicated hardware for it.

Pretty much anything that tries to accelerate network traffic outside of NSS MUST be disabled.

  • Packet steering
  • FLOWOFFLOAD
  • eBPF

NSS + ECM completely takes over the networking stack from the kernel when it's enabled. It simply adds hooks back in to ensure userspace tools like brctl, iptables, nft, tc still work.

There's just a lot of things being patched for it work with anything else.

For example, when looking at just the *.c files, we're patching over 43

crypto/algapi.c
crypto/authenc.c
drivers/clk/qcom/gcc-ipq8074.c
drivers/net/bonding/bond_3ad.c
drivers/net/bonding/bond_main.c
drivers/net/ifb.c
drivers/net/macvlan.c
drivers/net/ppp/ppp_generic.c
drivers/net/ppp/pppoe.c
drivers/net/ppp/pptp.c
drivers/net/vxlan/vxlan_core.c
net/8021q/vlan_core.c
net/bridge/br.c
net/bridge/br_device.c
net/bridge/br_fdb.c
net/bridge/br_if.c
net/bridge/br_input.c
net/bridge/br_netlink.c
net/core/dev.c
net/core/neighbour.c
net/core/skbuff.c
net/ipv4/esp4.c
net/ipv4/fib_trie.c
net/ipv4/ipmr.c
net/ipv6/addrconf.c
net/ipv6/esp6.c
net/ipv6/ip6mr.c
net/ipv6/ip6_output.c
net/ipv6/ip6_tunnel.c
net/ipv6/ndisc.c
net/ipv6/route.c
net/ipv6/sit.c
net/l2tp/l2tp_core.c
net/l2tp/l2tp_ppp.c
net/netfilter/nf_conntrack_core.c
net/netfilter/nf_conntrack_ecache.c
net/netfilter/nf_conntrack_extend.c
net/netfilter/nf_conntrack_proto_tcp.c
net/netfilter/nf_conntrack_standalone.c
net/netfilter/xt_DSCP.c
net/sched/sch_api.c
net/sched/sch_generic.c

Appreciate the reply.

I had sort of guessed that was the case (disabling all "network acceleration" stuff except NSS), but frankly eBPF is near the top of my "stuff I really dont understand" list, so figured I'd ask.

2 Likes

qca-nss-cfi-cryptoapi + cryptodev or af_alg will cause kernell penic.
disable that and use only software:

root@AX6:~# openssl engine -t -c
(dynamic) Dynamic engine loading support
     [ unavailable ]
(afalg) AFALG engine support
 [AES-128-CBC, AES-192-CBC, AES-256-CBC]
     [ available ]
root@AX6:~# openssl speed -evp aes-128-cbc
Doing AES-128-CBC for 3s on 16 size blocks: 63787 AES-128-CBC's in 0.10s
Doing AES-128-CBC for 3s on 64 size blocks: 65605 AES-128-CBC's in 0.15s
Doing AES-128-CBC for 3s on 256 size blocks: 66592 AES-128-CBC's in 0.15s
Doing AES-128-CBC for 3s on 1024 size blocks: 57222 AES-128-CBC's in 0.16s
Doing AES-128-CBC for 3s on 8192 size blocks: 30701 AES-128-CBC's in 0.13s
Doing AES-128-CBC for 3s on 16384 size blocks: 18370 AES-128-CBC's in 0.05s
version: 3.0.13
built on: Thu Apr 11 13:32:37 2024 UTC
options: bn(64,64)
compiler: aarch64-openwrt-linux-musl-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O3 -pipe -mcpu=cortex-a53 -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,relro -O3 -DPIC -fPIC -pipe -mcpu=cortex-a53 -fno-caller-saves -fno-plt -fhonour-copts -ffunction-sections -fdata-sections -Wformat -Werror=format-security -fstack-protector -O3 -fPIC -fuse-ld=mold -zrelro -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -D_FORTIFY_SOURCE=1 -DPIC
CPUINFO: OPENSSL_armcap=0x3d
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
AES-128-CBC      10205.92k    27991.47k   113650.35k   366220.80k  1934635.32k  6019481.60k

I've been eyeing the MX8500 PR for sometime. Looks like it'll be merged shortly.

Seems to be the only IPQ807x out there with WiFI 6E too. And by the looks of it the reason for it being cheap is due to Linksys' crappy 2+ year old firmware.

I might end up revisiting all the 6E patches from QSDK...

I'm in the same boat as you. I attempted to migrate my k3s cluster (flannel based) over to cilium (eBPF) and royally failed. eBPF is definitely promising though. I've seen many large discussions online about switching the netfilter stack to eBPF over nftables/iptables.

1 Like

I am absolutely lost.
My configuration is very normal, a couple of vlans and nothing more. I haven't changed the settings in a long time so I don't understand what's going on.
The problem is more serious with openwrt without nss offloading because not only is the wlan--->wan performance, but also the wlan--->lan speed is reduced by almost 50%

I have tried adding kernel 6.6.26, the problem remains the same. I will reset and test again.

You've verified all required NSS modules are loaded and running?

Check the outputs of:

  1. nss_stats
  2. ecm_dump.sh
  3. lsmod | awk '$1~/nss|qca|ecm/'
  4. cat /sys/module/ath11k/parameters/nss_offload

There's been cases where ecm fails to load due to the way the symbol dependancies are handled. Features in ECM get enabled only if certain kmods are selected, but don't get updated if they're unselected (i.e. full kernel rebuild)

If this problem was only on NSS build i will look on those stuff.
The problem i have go beyond NSS.
it could be my config, my device. I really don't kown.

After trying NSS builds with wds issues, I installed snapshot 20240409. Now no more errors.

Snapshot performance speed is just as fast if not faster, free memory and cpu looks normal.

I thought NSS build are faster? Or is snapshot also nss enabled already?

How fast is your WAN connection? If its not that fast, like 150 Mbps or lower, then you won't see any gains from NSS. If its a fast WAN connection, like 1.0-1.5 Gbps, then the only way you're going to be able to make use of that kind of speed is with NSS -- the CPU is simply not up to it.

Oh so the key is WAN? Sorry my AX3600 is a wifi repeater setup with wds to another AP.

Speed is 430mbps max, but I think the Linksys AP my AX3600 is link to is an even older Wifi5 router.

Can you post your memory usage and cpu load with latest snapshot?

I'm running Adblock-Fast and https-dns-proxy on it.

Im not sure about those packages affecting memory..
But I'm with dimfish's NSS build, and the memory is about 47% free, 53% used - approximately...
And I'm getting around 900mbps on 5Ghz.
Redmi AX6 (ipq807x)

Which ones should be off?


This is my current selection. It has been this way since I've started using NSS builds.
Never experienced issues/reboots, at least I think so.
Maybe unnecessary and useless overkill (bloat).

I'm on the previous to last commits but don't see any speed regression.

1 Like

yep !!! over 1gbps wan and in particular fiber