Qualcommax NSS Build

[cjayacopra]

Hi @qosmio!

Kudos for the hard work you have put on this project and to everyone involved.

I just want to raise a concern about a NSS build for Arcadyan AW1000.

I was able to successfully build one using these repos and branches:

qualcommax-6.x-nss-wifi default

NSS-12.5-K6.x default

I cloned the openwrt-ipq repo last 10/29/2024. I don't know how to address this, as I am not really a developer. Just teaching myself.

At first, I was not able to build a working one with luci-proto-quectel in rmnet. But was able to make it work by reverting this commit.

Now, I am running a build with working luci-proto-quectel with rmnet. The issue is it's not stable. The device sometimes reboots or reconnects to the internet. There are no logs about NSS in htop, there are still cpu spikes when doing speedtest even though there are NSS logs in kernel logs.

I have installed all the NSS_DRV and kmod_rmnet as well.

Below are my nss_diag and kernel logs.

root@Telstra5G:~# nss_diag
     MODEL: Arcadyan AW1000
   OPENWRT: v0.0.1
IPQ BRANCH: qualcommax-6.x-nss-wifi
IPQ COMMIT: 9e31cc9c9f
  IPQ DATE: 2024-10-25
    NSS FW: NSS.HK.11.4.0.5-6-R
  MAC80211: v6.11.2-0-g7aa21fec187b
 ATH11K FW: WLAN.HK.2.9.0.1-02146-QCAHKSWPL_SILICONZ-1
  GRO FRAG: br-lan      : off
            erspan0     : off
            gre0        : off
            gretap0     : off
            lan1        : off
            lan2        : off
            lan3        : off
            lan4        : off
            phy0-ap0    : off
            phy1-ap0    : off
            wan         : off
            wwan0       : off
            wwan0_1     : off
            wwan0_2     : off

  NSS PKGS: kmod-qca-nss-dp - 6.6.58.2024.04.16~5bf8b91e-r1
            kmod-qca-nss-drv - 6.6.58.11.4.0.5.2021.09.13~53e5863-r15
            kmod-qca-nss-drv-bridge-mgr - 6.6.58.11.4.0.5.2021.08.17~c4049d1-r7
            kmod-qca-nss-drv-igs - 6.6.58.11.4.0.5.2021.08.17~c4049d1-r7
            kmod-qca-nss-drv-qdisc - 6.6.58.11.4.0.5.2021.08.17~c4049d1-r7
            kmod-qca-nss-drv-vlan-mgr - 6.6.58.11.4.0.5.2021.08.17~c4049d1-r7
            kmod-qca-nss-drv-wifi-meshmgr - 6.6.58.11.4.0.5.2021.08.17~c4049d1-r7
            kmod-qca-nss-ecm - 6.6.58.12.5.5.2024.09.02~bd5057b-r3
            nss-firmware-default - 2024.08.04~794fe373-r1
            nss-firmware-ipq8074 - 2024.08.04~794fe373-r1

[    5.275431] **********************************************************
[    5.283478] * NSS Data Plane driver
[    5.289952] **********************************************************
[    5.325023] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.325084] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 1
[    5.329546] xhci-hcd xhci-hcd.1.auto: hcc params 0x0220fe65 hci version 0x110 quirks 0x0000008002000010
[    5.337087] xhci-hcd xhci-hcd.1.auto: irq 42, io mem 0x08a00000
[    5.346462] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.352244] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 2
[    5.357816] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0 SuperSpeed
[    5.365751] hub 1-0:1.0: USB hub found
[    5.371961] hub 1-0:1.0: 1 port detected
[    5.375944] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    5.380043] hub 2-0:1.0: USB hub found
[    5.388040] hub 2-0:1.0: 1 port detected
[    5.391759] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
[    5.395499] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus number 3
[    5.400893] xhci-hcd xhci-hcd.2.auto: hcc params 0x0220fe65 hci version 0x110 quirks 0x0000008002000010
[    5.408383] xhci-hcd xhci-hcd.2.auto: irq 43, io mem 0x08c00000
[    5.417815] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
[    5.423603] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus number 4
[    5.429158] xhci-hcd xhci-hcd.2.auto: Host supports USB 3.0 SuperSpeed
[    5.437099] hub 3-0:1.0: USB hub found
[    5.443331] hub 3-0:1.0: 1 port detected
[    5.447272] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
[    5.451404] hub 4-0:1.0: USB hub found
[    5.459262] hub 4-0:1.0: 1 port detected
[    5.465784] usbcore: registered new interface driver usb-storage
[    5.467657] usbcore: registered new interface driver uas
[    5.473123] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    5.481544] init: - preinit -
[    9.282713] nss-dp 3a001000.dp1 lan1: PHY Link up speed: 1000
...
[   11.043829] nss-dp 3a001000.dp1 lan1: PHY Link is down
[   11.049140] procd: - early -
[   11.049296] procd: - watchdog -
[   11.610964] procd: - watchdog -
[   11.613267] procd: - ubus -
[   11.765091] procd: - init -
[   12.272228] kmodloader: loading kernel modules from /etc/modules.d/*
[   12.369239] urngd: v1.0.2 started.
[   12.390274] tun: Universal TUN/TAP device driver, 1.6
[   12.399858] l2tp_core: L2TP core driver, V2.0
[   12.400908] l2tp_netlink: L2TP netlink interface
[   12.566357] qca-nss 39000000.nss: NSS FW Version: NSS.HK.11.4.0.5-6-R
[   12.566408] qca-nss 39000000.nss: fw of size 835960 bytes copied to addr: 40000000, nss_id: 0
[   12.573485] 
[   12.615762] qca-nss 39000000.nss: NSS core 0 booted successfully
[   12.727728] qca-nss 39400000.nss: fw of size 292296 bytes copied to addr: 40800000, nss_id: 1
[   12.728592] 
[   12.740151] qca-nss 39400000.nss: NSS core 1 booted successfully
[   12.742033] gre: GRE over IPv4 demultiplexor driver
[   12.746727] ip_gre: GRE over IPv4 tunneling driver
[   12.753818] QCA multicast snooping installed successfully
[   12.756290] usbcore: registered new interface driver cdc_wdm
[   12.761086] rmnet_nss_init(): initializing rmnet_nss
[   12.797183] qmi_wwan_q 4-1:1.4: cdc-wdm0: USB WDM device
[   12.797432] qmi_wwan_q 4-1:1.4: Quectel RG500Q-EA work on RawIP mode
[   12.803277] qmi_wwan_q 4-1:1.4: rx_urb_size = 31744
[   12.808538] qmi_wwan_q 4-1:1.4 wwan0: register 'qmi_wwan_q' at usb-xhci-hcd.2.auto-1, RMNET/USB device, a2:92:07:8f:b9:c4
[   12.986700] net wwan0 wwan0_1: NSS context created
[   12.986750] net wwan0: qmap_register_device wwan0_1
[   12.991755] net wwan0 wwan0_2: NSS context created
[   12.995232] net wwan0: qmap_register_device wwan0_2
[   13.000275] usbcore: registered new interface driver qmi_wwan_q
...
[   21.711117] ath11k c000000.wifi: nss init soc nss if_num 203 userpd_id 0
[   24.285312] ECM init: selected_front_end=1
[   24.285393] ECM database jhash random seed: 0xf97a2fec
[   24.412791] ECM init complete
[   24.502399] br-lan: port 1(lan1) entered blocking state
[   24.502484] br-lan: port 1(lan1) entered disabled state
[   24.506651] nss-dp 3a001000.dp1 lan1: entered allmulticast mode
[   24.512140] nss-dp 3a001000.dp1 lan1: entered promiscuous mode
[   24.525299] br-lan: port 2(lan2) entered blocking state
[   24.525355] br-lan: port 2(lan2) entered disabled state
[   24.529493] nss-dp 3a001200.dp2 lan2: entered allmulticast mode
[   24.535223] nss-dp 3a001200.dp2 lan2: entered promiscuous mode
[   24.545316] br-lan: port 3(lan3) entered blocking state
[   24.546372] br-lan: port 3(lan3) entered disabled state
[   24.551530] nss-dp 3a001400.dp3 lan3: entered allmulticast mode
[   24.557211] nss-dp 3a001400.dp3 lan3: entered promiscuous mode
[   24.568167] br-lan: port 4(lan4) entered blocking state
[   24.568505] br-lan: port 4(lan4) entered disabled state
[   24.573792] nss-dp 3a001600.dp4 lan4: entered allmulticast mode
[   24.579773] nss-dp 3a001600.dp4 lan4: entered promiscuous mode
[   25.553040] br-lan: port 5(phy0-ap0) entered blocking state
[   25.553096] br-lan: port 5(phy0-ap0) entered disabled state
[   25.557541] ath11k c000000.wifi phy0-ap0: entered allmulticast mode
[   25.563982] ath11k c000000.wifi phy0-ap0: entered promiscuous mode
[   25.569672] br-lan: port 5(phy0-ap0) entered blocking state
[   25.575580] br-lan: port 5(phy0-ap0) entered forwarding state
[   25.581362] br-lan: port 5(phy0-ap0) entered disabled state
[   26.370113] br-lan: port 5(phy0-ap0) entered blocking state
[   26.370169] br-lan: port 5(phy0-ap0) entered forwarding state
[   26.424924] qca-nss 39000000.nss: Configuring additional NSS pbufs
[   26.454605] qca-nss 39000000.nss: Additional pbufs of size 10002432 got added to NSS
[   26.686784] br-lan: port 6(phy1-ap0) entered blocking state
[   26.686840] br-lan: port 6(phy1-ap0) entered disabled state
[   26.691266] ath11k c000000.wifi phy1-ap0: entered allmulticast mode
[   26.697818] ath11k c000000.wifi phy1-ap0: entered promiscuous mode
[   26.752735] ath11k c000000.wifi phy0-ap0: left allmulticast mode
[   26.752800] ath11k c000000.wifi phy0-ap0: left promiscuous mode
[   26.757927] br-lan: port 5(phy0-ap0) entered disabled state
[   27.602702] nss-dp 3a001000.dp1 lan1: PHY Link up speed: 1000
...
 32.472601] l11: disabling
[   37.158874] ath11k c000000.wifi phy1-ap0: left allmulticast mode
[   37.158929] ath11k c000000.wifi phy1-ap0: left promiscuous mode
[   37.164067] br-lan: port 6(phy1-ap0) entered disabled state
[   37.892342] br-lan: port 6(phy1-ap0) entered blocking state
[   37.892393] br-lan: port 6(phy1-ap0) entered disabled state
[   37.896861] ath11k c000000.wifi phy1-ap0: entered allmulticast mode
[   37.902645] ath11k c000000.wifi phy1-ap0: entered promiscuous mode
[   42.910084] net wwan0: ul_data_aggregation_max_datagrams=11, ul_data_aggregation_max_size=4096, dl_minimum_padding=0
[   43.325800] net wwan0: link_state 0x0 -> 0x1
[   45.368439] Ignoring NSS change in VHT Operating Mode Notification from 70:bb:e9:21:48:d0 with invalid nss 2
[   51.534072] br-lan: port 6(phy1-ap0) entered blocking state
[   51.542987] br-lan: port 6(phy1-ap0) entered forwarding state

[qosmio]

@SaddFox

[  234.711316] CPU: 1 PID: 6350 Comm: igmpproxy Tainted: G           O       6.6.58 #0

igmpproxy breaks offloading as it's a software based routing daemon for multicast traffic. You would need to build without qca-mcs, which will tell ECM to disable building multicast offload. Although not sure how well that will hold up. But it's start.

@cjayacopra

At first, I was not able to build a working one with luci-proto-quectel in rmnet. But was able to make it work by reverting

Can you revert that file with the very first commit? I believe the last few commits broke something in the script.

[cjayacopra]

Will do just that. I'll let you know if there are changes.

UPDATE:

@qosmio I have used the very first commit but needed to change something in the Makefile. I updated all lines with quectel-CM-5G to quectel-cm as it was not detected by other programs that depends on it. In the end was able to compile a fw without dependency issues for quectel-cm but here's what I got.

luci-proto-quectel is missing even if its in the config.

I have updated my openwrt-ipq clone as well to using git pull and when i run nss_diag, here's what I got.

root@Telstra5G:~# nss_diag
/usr/bin/nss_diag: line 4: syntax error: unexpected redirection

[junky]

Just create a VLAN interface (wan is 10G-T port) and bridge, then add the interface and create AP/SSID from LuCI as usual.

My AP is running as a dumb AP (bridge), so not routing or NATing on this device. As I remember, VLANs on switch ports (LAN1-4) also work, but there may be limitations on other devices.

My device is a WXR-5950AX12 running firmware prebuilt images (NSS-WiFi with Mesh) 2024-05-15-1908. It's a slightly old firmware, so some changes might have been made in more recent builds, but I think this works.

Hope this helps.

[bgcngm]

So, your WAN port is connected to a router which is using DSA VLAN tagging?

My problem is that when I connect the managed switch or the dumb AP (both running OpenWrt and setup with DSA VLAN tagging) to the router running NSS build I don't have connectivity because I don't know how to tag those specific LAN ports on the router with id 1 and 10 using NSS.

[junky]

The WXR-5950AX12 has two independent AQR113Cs, which don’t appear to be using DSA. So, I brought back a Xiaomi AX3600 from storage and installed the prebuilt images (NSS-WiFi) 2024-10-20-0030. After running some tests, I confirmed that VLAN is working flawlessly on the latest firmware.
There are two methods to define VLANs. The first method is to create a VLAN on a physical interface, then create a bridge and add the VLAN interface to use as the base interface for the WiFi AP. The second method is to create a bridge and use VLAN filtering on the bridge. For example, you can create a bridge named wan-br, add a WAN interface, then enable VLAN filtering and add the VLAN tags you want to trunk.

After saving the configuration, the VLAN interfaces will be created automatically, like br-wan.1, br-wan.10, and so on. You can then use these interfaces as regular bridge interfaces.

Both methods worked on the AX3600.

Edit:
These methods are not specific to the NSS build but work well in my environment without any special setup. I tested VLAN switching (tagging VID 10 on WAN, untagging on LAN3), NAT, WiFi bridging, and more; everything worked as expected.

NSS is also functioning well, with the CPU usage remaining low (almost sleeping) while switching/routing 1 Gbps traffic. Pretty impressive!!!

[bgcngm]

I don't know what magic you're doing, but that certainly doesn't work for me and goes totally against what has been mentioned here lately that DSA VLAN tagging is incompatible with NSS. Let me show you the setup I have on AX3600 without NSS build:

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config interface 'iot'
	option proto 'static'
	option device 'br-lan.10'
	option ipaddr '192.168.10.1'
	option netmask '255.255.255.0'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:t'
	list ports 'lan2:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3'

As you can see lan1 and lan2 ports are trunks with vlan 1 and 10, to which I connect the a dumb AP and a managed switch. I need to redo this setup with NSS.

[junky]

I realized that after doing a configuration reset by holding the reset button, VLANs no longer work. Something is strange. I'll check it later.

[wolf359]

You have to create the VLAN and link it to a specific port which is the trunk to the rest of your network ie port LAN3 on the device

config device
        option type '8021q'
        option ifname 'lan3'
        option vid '30'
        option name 'lan3.30'
        option ipv6 '0'

then create a bridge for it

config device
        option type 'bridge'
        option name 'br-guest'
        list ports 'lan3.30'

then create an interface for the bridge which can then be associtaed to the wifi interface

config interface 'guest'
        option proto 'dhcp'
        option device 'br-guest'

[bgcngm]

What you're explaining there is clear, but you're simply tagging lan3 port with vlan id 30. How do you really make it a trunk for vlan id 1 and 10, for example? The device that shall be connected to that port is expecting this.

[wolf359]

I added additional vlan, device and interface for each vlan I want to support

[bgcngm]

That doesn't help. Do you have one lan port with more than one vlan? In your setup do you connect your router to some other switch or AP, configured with VLAN tagging?

[wolf359]

yes exaclty that, if you have a trunk which supports many vlan's then it needs to be tagged with them all.
if you wanted lan2 to ONLY be a specific VLAN then you add lan2 to the bridge for that vlan and remove it from the other bridges.
What is currently a mystery to me is how the PVID is assigned if nothing it tagged which was possible in the old way but just "seems to work" for me currently !

[junky]

I performed a fresh install of the 2024-10-20-0030 firmware (erasing the existing configuration) and reconfigured everything from scratch in the LuCI web UI. After that, VLANs started working again. I learned that enabling VLAN filtering on a bridge actually creates VLAN interfaces, so you can't assign an SSID (AP) to these VLAN interfaces (no radio icon).

SSIDs requires a bridge interface.

VLAN filtering on a bridge might only be for switching/routing purposes.

Here is my configuration:

root@AX3600:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd2c:bfe9:378b::/48'

config interface 'lan'
        option device 'lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'

config interface 'wan'
        option proto 'dhcp'
        option device 'wan.2002'

config interface 'wan6'
        option device 'wan.2002'
        option proto 'dhcpv6'
        option reqaddress 'try'
        option reqprefix 'auto'

config interface 'iot'
        option proto 'static'
        option device 'iot'
        option ipaddr '192.168.10.1'
        option netmask '255.255.255.0'

config device
        option type '8021q'
        option ifname 'lan1'
        option vid '1'
        option name 'lan1.1'

config device
        option type '8021q'
        option ifname 'lan2'
        option vid '1'
        option name 'lan2.1'

config device
        option type '8021q'
        option ifname 'lan1'
        option vid '10'
        option name 'lan1.10'

config device
        option type '8021q'
        option ifname 'lan2'
        option vid '10'
        option name 'lan2.10'

config device
        option type 'bridge'
        option name 'lan'
        list ports 'lan1.1'
        list ports 'lan2.1'

config device
        option type 'bridge'
        option name 'iot'
        list ports 'lan1.10'
        list ports 'lan2.10'
        list ports 'lan3'

config device
        option type '8021q'
        option ifname 'wan'
        option vid '2002'
        option name 'wan.2002'

I'm not sure why I had to do a fresh install after resetting the configuration, but it's working nicely again.

[teleport]

does anyone have a dumbAP nss build and have a working device discovery/connection using different vlans (like roku,speakers from their phone)?

[bgcngm]

@junky many thanks for your support, I have successfully recreated my old setup. Out of curiosity, why did you create vlan 2002 for wan? I have skipped that and ended up with:

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan1.1'
        list ports 'lan2.1'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option device 'wan'
        option proto 'dhcp'

config interface 'wan6'
        option device 'wan'
        option proto 'dhcpv6'

config device
        option type '8021q'
        option ifname 'lan1'
        option vid '1'
        option name 'lan1.1'

config device
        option type '8021q'
        option ifname 'lan2'
        option vid '1'
        option name 'lan2.1'

config device
        option type '8021q'
        option ifname 'lan1'
        option vid '10'
        option name 'lan1.10'

config device
        option type '8021q'
        option ifname 'lan2'
        option vid '10'
        option name 'lan2.10'

config device
        option type 'bridge'
        option name 'br-iot'
        list ports 'lan1.10'
        list ports 'lan2.10'
        list ports 'lan3'

config interface 'iot'
        option proto 'static'
        option device 'br-iot'
        option ipaddr '192.168.10.1'
        option netmask '255.255.255.0'

EDIT: I forgot to mention that attaching wireless networks to either "lan" or "iot" interfaces now works properly.

[junky]

Good to hear that. VLAN2002 is my public IPv6 VLAN, so please ignore it. I also tried the newly added IPv4-over-IPv6 tunnel, but it is not offloaded to NSS yet.

Anyway, I'm glad that my post helped you.

[sppmaster]

To all having QNAP 301w running NSS-wifi builds.
I'm glad to announce that loading the AQR 10G stock 5.4.C firmware for 10G-1 and 10G-2 ports on the 301w 0:ethphyfw1 and 0:ethphyfw2 partitions with the NSS build finally works from NVMEM.
There is another very interesting topic about the 301w eMMC here.

[M10]

What is the nat performance with hw nss offload on qhora on 10gig ports?

[bgcngm]

@qosmio I have created a pull request that documents this advanced NSS setup.

On a side note, not sure if anyone already noticed but the official firmware-selector page is showing qualcommax/ipq807x as possible platform for SNAPSHOT builds.