yeah i tried rmandrad repo and got some errors lol
1 Like
you need to select the nss firmware and eip
suggest you use qosmio repo for the latest
Yeah I did
Oh ok cool
After using NSS builds for several months on QNAP QHora 301W I can say the following.
Builds based on @bitthief repo.
NSS offloading works as expected for devices connected with cables. Still no NSS WiFi offloading.
I add netfilter: optional tcp window check patch to my build too in order to resolve the SSL errors I experience without it.
With this build after sysupgrade (keeping settings) the DHCP is not working right after the reboot and devices cannot get IP addresses. I have to turn off the power and after the next reboot everything is OK. Same issue was mentioned from others too and is present currently as far as I know on @AgustinLorenzo builds too.
When packet steering is enabled LAN becomes unusable.
These two are the only glitches I have found. All other things work flawlessly on my device. Rebooting takes a little over 30 seconds.
Builds based on @qosmio NSS-WiFi offloading repo (currently I only use it for testing purposes on my QNAP).
I add latest ath11k firmware to this build.
Issues with WLAN are mentioned in the above several posts.
The DHCP is working from the first reboot.
Additionally NSS doesn't work right out of the box. I have to reboot the device and only after the second reboot the NSS offload works OK (zero CPU usage) including NSS wifi offloading.
Running a torrent client on a PC connected to the router with cable for some reason destroys the LAN network and I have difficulties accessing the router web interface and all other clients connected via cable experience connection drops too.
Several other questions.
@dimfish
I see that you still use the obsolete HACK patch by Ansuel to workaround failing sysupgrade. And it was stated that it doesn't work OK.
As @hnyman committed this sysupgrade: Add 2 sec sleep into process KILL loop, I think it is not needed anymore.
Maybe you can review the commits of your NSS-6.1 repo.
Does your repo contain all necessary settings for QNAP? I ask because I see that you don't compile builds for it.
Thanks to all for your great work on this.
Still got the same error weirdly
small update, just wanted to provide an uptime report and more wifili stats
________________________________________________________________________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< INTERNAL: UNKNOWN >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
________________________________________________________________________________
________________________________________________________________________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< PDEV 0 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
________________________________________________________________________________
#txrx
wifili[0]_rx_msdu_error = 0 error
wifili[0]_rx_inv_peer_rcv = 0 special
wifili[0]_rx_wds_srcport_exception = 0 exception
wifili[0]_rx_wds_srcport_exception_fail = 0 drop
wifili[0]_rx_deliverd = 59388413 special
wifili[0]_rx_deliver_drops = 0 drop
wifili[0]_rx_intra_bss_ucast = 0 special
wifili[0]_rx_intra_bss_ucast_fail = 0 drop
wifili[0]_rx_intra_bss_mcast = 0 special
wifili[0]_rx_intra_bss_mcast_fail = 0 drop
wifili[0]_rx_sg_rcv_send = 0 special
wifili[0]_rx_sg_rcv_fail = 0 drop
wifili[0]_rx_mcast_echo = 0 special
wifili[0]_rx_inv_tid = 0 special
wifili[0]_stats_rx_frag_inv_sc = 0 special
wifili[0]_stats_rx_frag_inv_fc = 0 special
wifili[0]_stats_rx_frag_non_frag = 0 special
wifili[0]_stats_rx_frag_retry = 0 special
wifili[0]_stats_rx_frag_ooo = 0 special
wifili[0]_stats_rx_frag_ooo_seq = 0 special
wifili[0]_stats_rx_frag_all_frag_rcv = 0 special
wifili[0]_stats_rx_frag_deliver = 0 special
wifili[0]_tx_enqueue = 311604588 special
wifili[0]_tx_enqueue_drop = 0 drop
wifili[0]_tx_dequeue = 311604494 special
wifili[0]_tx_hw_enqueue_fail = 0 drop
wifili[0]_tx_sent_count = 311604831 special
#tcl ring
wifili[0]_tcl_no_hw_desc = 0 special
wifili[0]_tcl_ring_full = 0 special
wifili[0]_tcl_ring_sent = 311602437 special
#tcl comp
wifili[0]_tx_desc_free_inv_bufsrc = 0 error
wifili[0]_tx_desc_free_inv_cookie = 0 special
wifili[0]_tx_desc_free_hw_ring_empty = 0 special
wifili[0]_tx_desc_free_reaped = 312331835 special
wifili[0]_tx_capture_enqueue_count = 0 special
wifili[0]_tx_capture_enqueue_fail_count = 0 error
#reo ring
wifili[0]_reo_error = 0 error
wifili[0]_reo_reaped = 59389374 special
wifili[0]_reo_inv_cookie = 0 special
wifili[0]_stats_reo_frag_rcv = 0 special
#tx sw pool
wifili[0]_tx_desc_in_use = 0 special
wifili[0]_tx_desc_alloc_fail = 0 special
wifili[0]_tx_desc_already_allocated = 0 special
wifili[0]_tx_desc_invalid_free = 0 special
wifili[0]_tx_desc_free_src_fw = 8696640 special
wifili[0]_tx_desc_free_completion = 302907279 special
wifili[0]_tx_desc_no_pb = 0 special
wifili[0]_tx_desc_queuelimit_drop = 0 drop
#tx ext sw pool
wifili[0]_ext_tx_desc_in_use = 0 special
wifili[0]_ext_tx_desc_alloc_fail = 0 special
wifili[0]_ext_tx_desc_already_allocated = 0 special
wifili[0]_ext_tx_desc_invalid_free = 0 special
#rxdma pool
wifili[0]_rx_desc_no_pb = 14793189 special
wifili[0]_rx_desc_alloc_fail = 0 special
wifili[0]_rx_desc_in_use = 4095 special
#rxdma ring
wifili[0]_rxdma_hw_desc_unavailable = 0 special
wifili[0]_rxdma_buf_replenished = 60121572 special
________________________________________________________________________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< PDEV 1 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
________________________________________________________________________________
#txrx
wifili[1]_rx_msdu_error = 0 error
wifili[1]_rx_inv_peer_rcv = 0 special
wifili[1]_rx_wds_srcport_exception = 0 exception
wifili[1]_rx_wds_srcport_exception_fail = 0 drop
wifili[1]_rx_deliverd = 5800291 special
wifili[1]_rx_deliver_drops = 0 drop
wifili[1]_rx_intra_bss_ucast = 0 special
wifili[1]_rx_intra_bss_ucast_fail = 0 drop
wifili[1]_rx_intra_bss_mcast = 0 special
wifili[1]_rx_intra_bss_mcast_fail = 0 drop
wifili[1]_rx_sg_rcv_send = 3 special
wifili[1]_rx_sg_rcv_fail = 0 drop
wifili[1]_rx_mcast_echo = 0 special
wifili[1]_rx_inv_tid = 0 special
wifili[1]_stats_rx_frag_inv_sc = 0 special
wifili[1]_stats_rx_frag_inv_fc = 0 special
wifili[1]_stats_rx_frag_non_frag = 0 special
wifili[1]_stats_rx_frag_retry = 0 special
wifili[1]_stats_rx_frag_ooo = 0 special
wifili[1]_stats_rx_frag_ooo_seq = 0 special
wifili[1]_stats_rx_frag_all_frag_rcv = 0 special
wifili[1]_stats_rx_frag_deliver = 0 special
wifili[1]_tx_enqueue = 14745283 special
wifili[1]_tx_enqueue_drop = 0 drop
wifili[1]_tx_dequeue = 14745304 special
wifili[1]_tx_hw_enqueue_fail = 0 drop
wifili[1]_tx_sent_count = 14745417 special
#tcl ring
wifili[1]_tcl_no_hw_desc = 0 special
wifili[1]_tcl_ring_full = 0 special
wifili[1]_tcl_ring_sent = 14745415 special
#tcl comp
wifili[1]_tx_desc_free_inv_bufsrc = 0 error
wifili[1]_tx_desc_free_inv_cookie = 0 special
wifili[1]_tx_desc_free_hw_ring_empty = 0 special
wifili[1]_tx_desc_free_reaped = 15050094 special
wifili[1]_tx_capture_enqueue_count = 0 special
wifili[1]_tx_capture_enqueue_fail_count = 0 error
#reo ring
wifili[1]_reo_error = 0 error
wifili[1]_reo_reaped = 5794282 special
wifili[1]_reo_inv_cookie = 0 special
wifili[1]_stats_reo_frag_rcv = 0 special
#tx sw pool
wifili[1]_tx_desc_in_use = 0 special
wifili[1]_tx_desc_alloc_fail = 0 special
wifili[1]_tx_desc_already_allocated = 0 special
wifili[1]_tx_desc_invalid_free = 0 special
wifili[1]_tx_desc_free_src_fw = 8654669 special
wifili[1]_tx_desc_free_completion = 6090685 special
wifili[1]_tx_desc_no_pb = 0 special
wifili[1]_tx_desc_queuelimit_drop = 0 drop
#tx ext sw pool
wifili[1]_ext_tx_desc_in_use = 0 special
wifili[1]_ext_tx_desc_alloc_fail = 0 special
wifili[1]_ext_tx_desc_already_allocated = 0 special
wifili[1]_ext_tx_desc_invalid_free = 0 special
#rxdma pool
wifili[1]_rx_desc_no_pb = 0 special
wifili[1]_rx_desc_alloc_fail = 0 special
wifili[1]_rx_desc_in_use = 0 special
#rxdma ring
wifili[1]_rxdma_hw_desc_unavailable = 0 special
wifili[1]_rxdma_buf_replenished = 6103133 special
________________________________________________________________________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< PDEV 2 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
________________________________________________________________________________
#txrx
wifili[2]_rx_msdu_error = 0 error
wifili[2]_rx_inv_peer_rcv = 0 special
wifili[2]_rx_wds_srcport_exception = 0 exception
wifili[2]_rx_wds_srcport_exception_fail = 0 drop
wifili[2]_rx_deliverd = 0 special
wifili[2]_rx_deliver_drops = 0 drop
wifili[2]_rx_intra_bss_ucast = 0 special
wifili[2]_rx_intra_bss_ucast_fail = 0 drop
wifili[2]_rx_intra_bss_mcast = 0 special
wifili[2]_rx_intra_bss_mcast_fail = 0 drop
wifili[2]_rx_sg_rcv_send = 0 special
wifili[2]_rx_sg_rcv_fail = 0 drop
wifili[2]_rx_mcast_echo = 0 special
wifili[2]_rx_inv_tid = 0 special
wifili[2]_stats_rx_frag_inv_sc = 0 special
wifili[2]_stats_rx_frag_inv_fc = 0 special
wifili[2]_stats_rx_frag_non_frag = 0 special
wifili[2]_stats_rx_frag_retry = 0 special
wifili[2]_stats_rx_frag_ooo = 0 special
wifili[2]_stats_rx_frag_ooo_seq = 0 special
wifili[2]_stats_rx_frag_all_frag_rcv = 0 special
wifili[2]_stats_rx_frag_deliver = 0 special
wifili[2]_tx_enqueue = 0 special
wifili[2]_tx_enqueue_drop = 0 drop
wifili[2]_tx_dequeue = 0 special
wifili[2]_tx_hw_enqueue_fail = 0 drop
wifili[2]_tx_sent_count = 0 special
#tcl ring
wifili[2]_tcl_no_hw_desc = 0 special
wifili[2]_tcl_ring_full = 0 special
wifili[2]_tcl_ring_sent = 0 special
#tcl comp
wifili[2]_tx_desc_free_inv_bufsrc = 0 error
wifili[2]_tx_desc_free_inv_cookie = 0 special
wifili[2]_tx_desc_free_hw_ring_empty = 0 special
wifili[2]_tx_desc_free_reaped = 0 special
wifili[2]_tx_capture_enqueue_count = 0 special
wifili[2]_tx_capture_enqueue_fail_count = 0 error
#reo ring
wifili[2]_reo_error = 0 error
wifili[2]_reo_reaped = 0 special
wifili[2]_reo_inv_cookie = 0 special
wifili[2]_stats_reo_frag_rcv = 0 special
#tx sw pool
wifili[2]_tx_desc_in_use = 0 special
wifili[2]_tx_desc_alloc_fail = 0 special
wifili[2]_tx_desc_already_allocated = 0 special
wifili[2]_tx_desc_invalid_free = 0 special
wifili[2]_tx_desc_free_src_fw = 0 special
wifili[2]_tx_desc_free_completion = 0 special
wifili[2]_tx_desc_no_pb = 0 special
wifili[2]_tx_desc_queuelimit_drop = 0 drop
#tx ext sw pool
wifili[2]_ext_tx_desc_in_use = 0 special
wifili[2]_ext_tx_desc_alloc_fail = 0 special
wifili[2]_ext_tx_desc_already_allocated = 0 special
wifili[2]_ext_tx_desc_invalid_free = 0 special
#rxdma pool
wifili[2]_rx_desc_no_pb = 9059145 special
wifili[2]_rx_desc_alloc_fail = 0 special
wifili[2]_rx_desc_in_use = 4095 special
#rxdma ring
wifili[2]_rxdma_hw_desc_unavailable = 0 special
wifili[2]_rxdma_buf_replenished = 0 special
#wbm ring
wifili_wbm_ie_local_alloc_fail = 0 error
wifili_wbm_src_dma = 772923 special
wifili_wbm_src_dma_code_inv = 0 special
wifili_wbm_src_reo = 259979 special
wifili_wbm_src_reo_code_nullq = 483 special
wifili_wbm_src_reo_code_inv = 246258 error
wifili_wbm_src_inv = 0 error
I noticed most of my devices that were failing to connect, have all reconnected (they were all 2G smart devices) over the last few days. Still haven't pinpointed what exactly is causing it.
These are the warnings I still get every couple hours (MAC addresses masked)
[Sat Dec 23 12:01:49 2023] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[Sat Dec 23 12:01:49 2023] ath11k c000000.wifi: failed to transmit frame -22
[Sat Dec 23 12:01:53 2023] ath11k c000000.wifi: peer (xx:xx:xx:xx:xx:xx) not found for nss peer delete
[Sat Dec 23 12:06:51 2023] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[Sat Dec 23 12:06:51 2023] ath11k c000000.wifi: failed to transmit frame -22
[Sat Dec 23 12:06:55 2023] ath11k c000000.wifi: peer (xx:xx:xx:xx:xx:xx) not found for nss peer delete
As others have mentioned, I'll try to revert just the nss peer delete related code, easier said than done, since it looks to be part of the patch that introduces NSS offload into ath11k...
6 Likes
Great news, and good luck on the peer delete code.
Maybe I'll ask Santa to solve this 
Best present there is!
1 Like
Am I understanding correctly that your build is, for the most part, fully functional except for some reconnect glitches on 2.4 GHz? Should be able to keep up with 1.5 Gbps fiber?
i am using the same build with no issues apart from iot devices ... it should keep up with 1.5 Gbps ... I have 1.2Gbps
1 Like
That's great to hear!
Now if I understand correctly, nobody is hosting builds, its a build it yourself deal, right? So I need to get set up for that. Any hints? I've seen you above mention some firmware selection, not sure how that works.
Thanks 
not necessarily I know that @AgustinLorenzo and @dimfish post their own build but of course you can build your own.
I suggest you use this forum and search/ask but first have a read @ https://openwrt.org/docs/guide-developer/start
I managed to get through the generic parts, but compile failed on parts that are very target-specific;
/home/adam/openwrt-ipq/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/qca-nss-drv-2023-08-06-1ab18403/nss_rps.c:289:24: error: 'nss_rps_ipv4_hash_bitmap_cfg' defined but not used [-Werror=unused-function]
289 | static nss_tx_status_t nss_rps_ipv4_hash_bitmap_cfg(struct nss_ctx_instance *nss_ctx, int hash_bitmap)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[5]: *** [scripts/Makefile.build:250: /home/adam/openwrt-ipq/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/qca-nss-drv-2023-08-06-1ab18403/nss_rps.o] Error 1
make[4]: *** [Makefile:2014: /home/adam/openwrt-ipq/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/qca-nss-drv-2023-08-06-1ab18403] Error 2
make[3]: *** [Makefile:279: /home/adam/openwrt-ipq/build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/qca-nss-drv-2023-08-06-1ab18403/.built] Error 2
make[3]: Leaving directory '/home/adam/openwrt-ipq/feeds/nss_packages/qca/qca-nss-drv'
time: package/feeds/nss_packages/qca-nss-drv/compile#8.00#2.11#10.06
ERROR: package/feeds/nss_packages/qca-nss-drv failed to build.
make[2]: *** [package/Makefile:128: package/feeds/nss_packages/qca-nss-drv/compile] Error 1
make[2]: Leaving directory '/home/adam/openwrt-ipq'
make[1]: *** [package/Makefile:122: /home/adam/openwrt-ipq/staging_dir/target-aarch64_cortex-a53_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/adam/openwrt-ipq'
make: *** [/home/adam/openwrt-ipq/include/toplevel.mk:232: world] Error 2
Got some time this weekend to work on the wifi offload patches some more, and can happily say I've been able to resolve almost all of the issues I was encountering
1.) Resolved encapsulation mismatch errors
[Sun Dec 24 12:03:21 2023] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[Sun Dec 24 12:03:21 2023] ath11k c000000.wifi: failed to transmit frame -22
[Sun Dec 24 12:08:25 2023] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[Sun Dec 24 12:08:25 2023] ath11k c000000.wifi: failed to transmit frame -22
2.) Properly handle nss peer deletion.
[Sun Dec 17 02:43:25 2023] ath11k c000000.wifi: peer not found for nss peer delete
[Sun Dec 17 02:43:28 2023] ath11k c000000.wifi: peer not found for nss peer delete
[Sun Dec 17 02:43:30 2023] ath11k c000000.wifi: peer not found for nss peer delete
3.) Clients should also be able to switch between 2Ghz and 5Ghz AP, as well as join using password (no longer required to boot with NSS off first, connect the client to acquire PSK, and reboot back with nss wifi offload enabled).
Uptime has been 5 hours and so far no issues.
Be sure to enable the symbol ATH11K_MEM_PROFILE_512M in menuconfig when building.
One step closer getting a fully offloaded NSS build. Merry Christmas 
13 Likes
Thanks for your hard works!
Can you please share your build .config?
Certainly. IPQ807x dynalink dl-wrx36 build.config
Correct (more or less). You should be able to get fully offloaded 1.2-1.5Gbps for WiFi 6 clients if your ISP provides it. I can't confirm more than 1Gbps, as that's the max for my ISP. But theoretically it should work.
How is the WIFI-LAN iperf3 speed on your build?
And CPU usage?
M1 Pro -> LAN Server
iPhone 15 Pro Max -> LAN Server
CPU usage is 0% for WLAN <-> LAN (iperf3 NOT on router)
iPhone 15 Pro Max -> Router
Just providing throughput example when running iperf3 on the router. CPU usage will obviously spike since iperf3 is CPU intensive, but I've pretty consistently been able to reach 1.3-1.5gbps when before, max would be ~900-1100mbps
10 Likes