IPQ807X NSS Build

I thought you said the patches from AU_LINUX_QSDK_NHSS.QSDK.12.4.5.R2_TARGET_ALL.12.4.502.099.019 was ready and working for latest version. And that you had compiled with the patches and build to see if it did work.

Anyways, thanks for the input tho.

1 Like

hmm i got nss build running but cant get higher than 600mbps

@rmandrad

Wed Dec 13 15:21:15 2023 kern.alert kernel: [   10.154156] nss_cryptoapi_init[2350]:ffffffc001083100: Failed to register nss_cryptoapi
Wed Dec 13 15:21:15 2023 kern.warn kernel: [   10.177158] nss_vxlanmgr_init_module[193]:nss-common not found.
Wed Dec 13 15:21:15 2023 user.err kernel: [   10.192989] kmodloader: 2 modules could not be probed
Wed Dec 13 15:21:15 2023 user.err kernel: [   10.193120] kmodloader: - qca-nss-cfi-cryptoapi - 0
Wed Dec 13 15:21:15 2023 kern.warn kernel: [   10.193338] ath11k c000000.wifi: qmi ignore invalid mem req type 3
Wed Dec 13 15:21:15 2023 user.err kernel: [   10.197088] kmodloader: - qca-nss-vxlanmgr - 0
Wed Dec 13 15:21:15 2023 kern.info kernel: [   10.207170] ath11k c000000.wifi: chip_id 0x0 chip_family 0x0 board_id 0xff soc_id 0xffffffff
Wed Dec 13 15:21:15 2023 kern.info kernel: [   10.212405] ath11k c000000.wifi: fw_version 0x290104a5 fw_build_timestamp 2023-08-02 20:32 fw_build_id WLAN.HK.2.9.0.1-01890-QCAHKSWPL_SILICONZ-1
Wed Dec 13 15:21:15 2023 kern.warn kernel: [   10.539237] ath11k c000000.wifi: htt event 48 not handled

you got any clue on this

also getting

Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-arptables = 1
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-iptables = 1
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-ip6tables = 1
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: sysctl: error: 'dev.nss/general/redirect' is an unknown key
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: Bridge filtering is enabled in the ECM config, this will cause issues with NAT loopback!
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-arptables = 1
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-iptables = 1
Wed Dec 13 17:04:10 2023 daemon.notice procd: /etc/rc.d/S19qca-nss-ecm: net.bridge.bridge-nf-call-ip6tables = 1
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nldyna_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) dynamic_interface init failed, error(-22)
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nlethrx_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) eth_rx init failed, error(-22)
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nlipv4_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) ipv4 init failed, error(-22)
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nlipv6_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) ipv6 init failed, error(-22)
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nllso_rx_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) lso_rx init failed, error(-22)
NSS_NLERROR(nss_nlsock_join_grp[262]):failed to resolve group(nss_nln2h_mc)
NSS_NLERROR(nss_nlmcast_sock_join_grp[97]):Unable to subscribe for mcast group, error(-                                                                                                                            22)
ERR(nssinfo_init:671) n2h init failed, error(-22)

which device are you using ? and which repo your forked from for your build?

Linksys MX4200 and im using bittheif repo

does the dts for your device has an include statement for the nss dts ?

something like this

#include "ipq8074.dtsi"
#include "ipq8074-hk-cpu.dtsi"
#include "ipq8074-ess.dtsi"
#include "ipq8074-nss.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
#include <dt-bindings/leds/common.h>

1 Like

ah no i didnt

yeah that kinda worked

Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.560621] qca-nss 39000000.nss: Direct firmware load for qca-nss0.bin failed with error -2
Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.560668] qca-nss 39000000.nss: Falling back to sysfs fallback for: qca-nss0.bin
Wed Dec 13 17:26:20 2023 user.info kernel: [    8.730285] urngd: v1.0.2 started.
Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.805970] qca-nss 39400000.nss: Direct firmware load for qca-nss1.bin failed with error -2
Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.806020] qca-nss 39400000.nss: Falling back to sysfs fallback for: qca-nss1.bin
Wed Dec 13 17:26:20 2023 kern.alert kernel: [    8.834375] NSS DDR size is 0x1000000
Wed Dec 13 17:26:20 2023 kern.alert kernel: [    8.834416] ffffffc000adaf40: nss core 0 booted successfully
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.882764] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
Wed Dec 13 17:26:20 2023 kern.alert kernel: [    8.883512] NSS DDR size is 0x1000000
Wed Dec 13 17:26:20 2023 kern.alert kernel: [    8.887526] ffffffc000ae2880: nss core 1 booted successfully
Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.895399] nat46: module (version 1182f30785e4274913f01a8c3d7e1b5437ae3819) loaded.
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.899277] gre: GRE over IPv4 demultiplexor driver
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.906187] ip_gre: GRE over IPv4 tunneling driver
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.914074] ip6_gre: GRE over IPv6 tunneling driver
Wed Dec 13 17:26:20 2023 kern.warn kernel: [    8.919892] QCA multicast snooping installed successfully
Wed Dec 13 17:26:20 2023 kern.notice kernel: [    8.921738] nss_vlan_mgr_init_module[1926]:Module (Build Build Id: 12/13/23, 17:54:33) loaded
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.929959] PPP generic driver version 2.4.2
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.945895] NET: Registered PF_PPPOX protocol family
Wed Dec 13 17:26:20 2023 kern.info kernel: [    8.949629] PPTP driver version 0.8.5
Wed Dec 13 17:26:20 2023 kern.notice kernel: [    8.963729] <NSS-CRYPTO>:module loaded Build_ID - 12/13/23, 17:54:14 SoC=ipq807x
1 Like

ah just modprobed nss_ifb

still doesnt seem to hit 1gbps 560mbps at max

hmmm

ERR(nssinfo_lso_rx_stats_display:46) (null) is not running on the NPU

To clarify @rmandrad comment, the qca-nss-drv package is what's being shown in the screenshot, specifically that's it's been compiled with the WiFi offload feature. You still need to apply patches to mac80211/ath11k that will hook into it.

Getting NSS WiFi offload to work has been a constant struggle for years, ever since ipq806x. @quarky, @Ansuel, @ACwifidude, @robimarko and myself have all tried retrofitting Qualcomm's (half baked) ath10/11k NSS patches but have not been successful. There's "some" acceleration support by using encapsulation mode (frame_mode=2) with NSS, but has been unstable/unusable on ipq807x.

FWIW, I recently got a DL-WRX36 myself and have been attempting to try again. Since cherry picking the NSS patches hasn't been fruitful, I figured the best approach would be to start with Qualcomm's fork of mac80211 and overlay upstream's patches on top. I've only just now got it to compile, but the patching is a mess... Haven't been able to boot it unfortunately.

I also suspect hostapd needs to be patched with Qualcomm's fork to properly get the wireless stack usable. But that too is a colossal mess of patches.

2 Likes

don't mix the 2 thing... ath10k had a very simple implementation with virtual interface. ath11k have correct offload. I used for a while offload patch (if correctly applied instead of randomly adapt stuff) but i'm really busy with other task and maintain that isn't really top of my priority now.

Honestly I don't suggest starting from mac80211 qsdk source as you will quickly notice how unusable it is.

3 Likes

i will stop what i was intending in doing (starting by the patches/subsys/199-00(1-3)-mac80211-add-nss-support) ... as you are much further ahead

for reference i was also looking at Ansuel's - https://github.com/Ansuel/openwrt/blob/ipq807x-5.15-pr-wifi-offload/package/kernel/mac80211/patches/ath11k_nss/ that took a similar approach ...

and yes I also suspect that hostapd needs patching ... but hey that's why openwrt is fun :grin:

2 Likes

You may even go further, and see past @Ansuel commit to start support ath11k offload on 5.15:
[WIP] mac80211: add ath11k nss wifi offload ยท Ansuel/openwrt@3ecaee7 (github.com) and ipq807x: add nss nodes for wifi offload as there you have some already pre-selected patches and code fixes

I did that and for now I get this...

[   15.644742] ath11k c000000.wifi: failed to transmit frame -22
[   15.749902] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[   15.749970] ath11k c000000.wifi: failed to transmit frame -22
[   16.757197] ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2
[   16.757282] ath11k c000000.wifi: failed to transmit frame -22
[   17.662106] ath11k c000000.wifi: wmi tlv parse failure of tag 663 at byte 4 (52 bytes is less than min length 60)
[   17.662166] ath11k c000000.wifi: failed to parse tlv: -22
[   17.671373] ath11k c000000.wifi: failed to extract pdev bss chan info event
[   20.727033] ath11k c000000.wifi: bss channel survey timed out

It's a start :smile:

Edit: Just did a quick test with patch:

113-ath11k-add-8023-undecap-support.patch

and that was the reason of all the messages...

ath11k c000000.wifi: encap mismatch in nss tx skb encap type 1vif encap type 2

However, I think we are talking and shaking hands with nss-drv, but hostapd/ath11k does not like what's going on... I added some messages so I could see what they are talking about.

# from boot
[    9.202901] ath11k c000000.wifi: ath11k_nss_setup: target_type is 24
[    9.203113] ath11k c000000.wifi: ath11k_nss_init: target_type is 24
[    9.209734] ath11k c000000.wifi: ath11k_nss_init: nss init soc if_num is 203
[    9.229219] ath11k c000000.wifi: ath11k_nss_init: NSS Init Message TX Success 000000000c1c8222 on if_num 203
[    9.235055] ath11k c000000.wifi: ath11k_nss_pdev_init: the response from pdev init is 0
[    9.238166] ath11k c000000.wifi: ath11k_nss_stats_cfg: nss stats 32 enable 0
[    9.245855] ath11k c000000.wifi: ath11k_nss_stats_cfg: status back from nss_wifili_tx_msg is 0
[    9.253167] ath11k c000000.wifi: ath11k_nss_stats_cfg: nss stats 17 enable 1
[    9.261588] ath11k c000000.wifi: ath11k_nss_stats_cfg: status back from nss_wifili_tx_msg is 0
[    9.274686] ath11k c000000.wifi: ath11k_nss_pdev_init: the response from pdev init is 0
[    9.277261] ath11k c000000.wifi: ath11k_nss_stats_cfg: nss stats 32 enable 0
[    9.285136] ath11k c000000.wifi: ath11k_nss_stats_cfg: status back from nss_wifili_tx_msg is 0
[    9.292425] ath11k c000000.wifi: ath11k_nss_stats_cfg: nss stats 17 enable 1
[    9.300826] ath11k c000000.wifi: ath11k_nss_stats_cfg: status back from nss_wifili_tx_msg is 0
[    9.308041] ath11k c000000.wifi: ath11k_nss_start: status response is 0
[    9.316444] ath11k c000000.wifi: ath11k_nss_start: nss start is a success 0
[    9.322954] ath11k c000000.wifi: ath11k_nss_setup: nss setup is complete 0

# tried the channel analysis tool (no wifi is enabled yet)
[   64.526223] ath11k c000000.wifi: ath11k_nss_vdev_register: nss registered with if_num 203 and status is 0
[   64.526367] ath11k c000000.wifi: ath11k_nss_vdev_configure: id is 0 ifnum 28 and opmode 3
[   64.534866] ath11k c000000.wifi: ath11k_nss_vdev_create: nss vdev interface created ctx 000000000c1c8222, ifnum 30
[   64.543047] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:1 val:2
[   64.553225] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:2 val:2
[   69.147077] ath11k c000000.wifi: ath11k_nss_vdev_unregister: nss unregistered with if_num 203

# enabled wifi in AP mode
[  170.073688] ath11k c000000.wifi: ath11k_nss_vdev_configure: id is 0 ifnum 28 and opmode 1
[  170.082192] ath11k c000000.wifi: ath11k_nss_vdev_create: nss vdev interface created ctx 000000000c1c8222, ifnum 30
[  170.090335] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:1 val:2
[  170.100542] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:2 val:2
[  170.109513] ath11k c000000.wifi: ath11k_nss_peer_create: nss peer_create msg success mac:00:03:... vdev:0 peer_id:2 hw_ast_idx:492 ast_hash:492
[  170.896433] ath11k c000000.wifi: ath11k_nss_vdev_up: nss vdev up tx msg success with status 0
[  170.916125] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:14 val:0
[  171.017284] ath11k c000000.wifi: ath11k_nss_vdev_set_cmd: nss vdev set cmd success cmd:15 val:6
[  171.017348] ath11k c000000.wifi: ath11k_nss_set_peer_sec_type: nss peer id 2 security cfg complete
[  171.024815] ath11k c000000.wifi: ath11k_nss_set_peer_authorize: nss peer id 2 auth cfg complete
[  176.525162] ath11k c000000.wifi: wmi tlv parse failure of tag 663 at byte 4 (52 bytes is less than min length 60)
[  176.525235] ath11k c000000.wifi: failed to parse tlv: -22
[  176.534426] ath11k c000000.wifi: failed to extract pdev bss chan info event
[  179.527042] ath11k c000000.wifi: bss channel survey timed out

and the last 4 messages is looping...

Lesson learned the hard way... After brute forcing 442 of qsdk patches, I got everything to compile and boot up... but shortly after it just crash dumps.

Was it stable for you on 5.15? I'm guessing this was on the Xiaomi AX3600?

1 Like

Interesting stuff going on here

@rmandrad @Ansuel

heres ECM log i get about 500mbps im not sure if its working i have a 1gbps package
https://katb.in/lolilulujak

According to this the NSS cores should be running at 1ghz for the Oak platform (ipq817x)

Some good news. I've got NSS wifi offload working on latest branch of main here: qualcommax-6.1-nss-wifi. Used @dimfish's qualcommax-6.1-nss branch as a starting point.

Everything seems to be working, but it doesn't seem stable. Boots with a ton of:

ath11k c000000.wifi: peer not found for nss peer delete

But it eventually seems to subside. So far it's been up and running for a few hours. Still needs a lot of work.

7 Likes