Qualcommax NSS Build

qosmio · 2024-10-01T02:05:39.547Z

All,

I wanted to let you know that I just pushed an update to the README, which has been a long time coming! While it’s still a work in progress and doesn’t cover all the nitty-gritty details, but hopefully should answer a few of the repeat questions we get here.

The updated sections include clearer explanations of the NSS integration, offloading options, and quickstart instructions to help you get started more easily.

I’ll do my best to update it whenever I can, so please bear with me. It's very time-consuming and my least favorite of part development

qosmio · 2024-10-01T02:09:44.384Z

If you cold start the router, do subsequent reboots still produce that error?

zxlhhyccc · 2024-10-01T03:10:55.275Z

@qosmio

github.com

openwrt-ipq/nss-setup at qualcommax-6.x-nss-wifi · qosmio/openwrt-ipq

This repository is a mirror of https://git.openwrt.org/openwrt/openwrt.git It is for reference only and is not active for check-ins. We will continue to accept Pull Requests here. They will be mer...

Is this an example for how to configure compilation?

georgem83 · 2024-10-01T03:47:52.418Z

@qosmio: what would it take to add NSS support for ipq50xx?
We’ve been working on this for a while at: IPQ5018: Support for Linksys MX2000 Atlas 6 & MX5500 Atlas 6 Pro - #266 by lytr

Apart from a few very minor tasks left (such as optimizing BDFs and scripts to set the MAC addresses), we should be done.

sppmaster · 2024-10-01T07:26:50.181Z

No, neither worm nor cold reboot produce any error.
Not exactly on the subject but I have the same issue (started recently) on R7800 NSS builds. After firmware flash it produces (another) error after first reboot.
But maybe the cold reboot revealed another interesting thing.
I had complaints of 80 MHz cap on Rx chain with ath11k 2.12.
Now after cold reboot I saw the same happened with ath11k 2.9 - it just stuck at 80 MHz (as seen in Luci) and after playing a bit I just disabled and then reenabled the 5GHz radio and it showed 160 MHz again.
But this "trick" doesn't work for me with ath11k 2.12.
It appears it's not only visual/cosmetic glitch because iper3 tests confirm it - speed from client to the router is cut in half.

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-30.06  sec   453 MBytes   127 Mbits/sec                  receiver
[  8]   0.00-30.06  sec   460 MBytes   128 Mbits/sec                  receiver
[ 10]   0.00-30.06  sec   453 MBytes   126 Mbits/sec                  receiver
[ 12]   0.00-30.06  sec   456 MBytes   127 Mbits/sec                  receiver
[SUM]   0.00-30.06  sec  1.78 GBytes   508 Mbits/sec                  receiver

With 2.9 I get 1.2-1.3 Gbit/sec in both directions.
Excellent start with the Readme by the way.

glassdoor · 2024-10-01T08:18:00.659Z

@qosmio

github.com/qosmio/openwrt-ipq

kernel: Fix UDP hanging in unbound

committed 06:27AM - 30 Sep 24 UTC

qosmio

+50 -0

Signed-off-by: Sean Khan <datapronix@protonmail.com> (cherry picked from commit …6b2ed85b2abe693f3b14bde7b12627cc272830cc)

will this patch apply to mainline openwrt as well?

zxlhhyccc · 2024-10-01T14:29:03.329Z

@qosmio

I didn't compile the NAPI branch, why do I still have napi content?

root@OpenWrt:~# ls /sys/fs/pstore/
dmesg-ramoops-0
root@OpenWrt:~# cat /sys/fs/pstore/*
Panic#1 Part1
<5>[    2.013814] ubi0: background thread "ubi_bgt0d" started, PID 558
<6>[    2.023678] block ubiblock0_1: created from ubi0:1(rootfs)
<6>[    2.029292] clk: Disabling unused clocks
<6>[    2.038191] VFS: Mounted root (squashfs filesystem) readonly on device 254:0.
<6>[    2.038653] Freeing unused kernel memory: 896K
<6>[    2.044416] Run /sbin/init as init process
<7>[    2.048662]   with arguments:
<7>[    2.048665]     /sbin/init
<7>[    2.048668]   with environment:
<7>[    2.048670]     HOME=/
<7>[    2.048673]     TERM=linux
<14>[    2.262769] init: Console is alive
<14>[    2.262911] init: - watchdog -
<5>[    2.995447] random: crng init done
<14>[    3.303565] kmodloader: loading kernel modules from /etc/modules-boot.d/*
<4>[    3.338239] gpio_button_hotplug: loading out-of-tree module taints kernel.
<6>[    3.343954] pstore: Using crash dump compression: deflate
<6>[    3.344016] pstore: Registered ramoops as persistent store backend
<6>[    3.349584] ramoops: using 0x100000@0x4e900000, ecc: 0
<4>[    3.399464] ssdk_dt_parse_interrupt[941]:INFO:intr-gpio does not exist
<4>[    3.821573] regi_init[2525]:INFO:Initializing HPPE Done!!
<4>[    3.821698] regi_init[2574]:INFO:qca-ssdk module init succeeded!
<6>[    3.828693] EDMA ver 1 hw init
<6>[    3.832263] EDMA HW Reset completed succesfully
<6>[    3.834933] Num rings - TxDesc:1 (23-23) TxCmpl:1 (7-7)
<6>[    3.839373] RxDesc:1 (15-15) RxFill:1 (7-7)
<6>[    3.845000] dp2: ppe offload disabled: 0 for macid 2
<6>[    3.848718] dp2: Switch attached to macid 2 status: 0
<6>[    4.046240] Qualcomm QCA8075 90000.mdio-1:01: attached PHY driver (mii_bus:phy_addr=90000.mdio-1:01, irq=POLL)
<6>[    4.047287] dp3: ppe offload disabled: 0 for macid 3
<6>[    4.055130] dp3: Switch attached to macid 3 status: 0
<6>[    4.145800] Qualcomm QCA8075 90000.mdio-1:02: attached PHY driver (mii_bus:phy_addr=90000.mdio-1:02, irq=POLL)
<6>[    4.146893] dp4: ppe offload disabled: 0 for macid 4
<6>[    4.154694] dp4: Switch attached to macid 4 status: 0
<6>[    4.235796] Qualcomm QCA8075 90000.mdio-1:03: attached PHY driver (mii_bus:phy_addr=90000.mdio-1:03, irq=POLL)
<6>[    4.236833] dp5: ppe offload disabled: 0 for macid 5
<6>[    4.244687] dp5: Switch attached to macid 5 status: 0
<6>[    4.325786] Qualcomm QCA8075 90000.mdio-1:04: attached PHY driver (mii_bus:phy_addr=90000.mdio-1:04, irq=POLL)
<6>[    4.326793] **********************************************************
<6>[    4.334679] * NSS Data Plane driver
<6>[    4.341208] **********************************************************
<14>[    4.347167] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<14>[    4.361270] init: - preinit -
<5>[    7.341163] UBIFS (ubi0:2): default file-system created
<5>[    7.341805] UBIFS (ubi0:2): Mounting in unauthenticated mode
<5>[    7.345322] UBIFS (ubi0:2): background thread "ubifs_bgt0_2" started, PID 728
<5>[    7.406451] UBIFS (ubi0:2): UBIFS: mounted UBI device 0, volume 2, name "rootfs_data"
<5>[    7.406487] UBIFS (ubi0:2): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
<5>[    7.413267] UBIFS (ubi0:2): FS size: 24887296 bytes (23 MiB, 196 LEBs), max 206 LEBs, journal size 1269760 bytes (1 MiB, 10 LEBs)
<5>[    7.423185] UBIFS (ubi0:2): reserved for root: 1175487 bytes (1147 KiB)
<5>[    7.434798] UBIFS (ubi0:2): media format: w5/r0 (latest is w5/r0), UUID D031711E-5FC7-4E13-AECA-8C7613491D7F, small LPT model
<14>[    7.443001] mount_root: overlay filesystem has not been fully initialized yet
<14>[    7.453015] mount_root: switching to ubifs overlay
<4>[    7.462498] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
<12>[    7.468969] urandom-seed: Seed file not found (/etc/urandom.seed)
<14>[    7.545789] procd: - early -
<14>[    7.545906] procd: - watchdog -
<14>[    8.126709] procd: - watchdog -
<14>[    8.154817] procd: - ubus -
<14>[    8.315789] procd: - init -
<14>[    8.880605] kmodloader: loading kernel modules from /etc/modules.d/*
<14>[    9.025363] urngd: v1.0.2 started.
<6>[    9.032802] NET: Registered PF_ALG protocol family
<6>[    9.042669] tun: Universal TUN/TAP device driver, 1.6
<6>[    9.053619] ipip: IPv4 and MPLS over IPv4 tunneling driver
<13>[    9.303871] hotplug: symlinking qca-nss0.bin to /lib/firmware/qca-nss0-retail.bin
<6>[    9.432347] qca-nss 39000000.nss: NSS FW Version: NSS.HK.11.4.0.5-6-R
<6>[    9.432400] qca-nss 39000000.nss: fw of size 835960 bytes copied to addr: 40000000, nss_id: 0
<1>[    9.439801]
<6>[    9.485441] qca-nss 39000000.nss: NSS core 0 booted successfully
<1>[    9.491776] NSS core 0 signal COREDUMP COMPLETE 4000
<1>[    9.491803] ffffffc07907f580: Starting NSS-FW logbuffer dump for core 0
<1>[    9.495809] ffffffc07907f580: Warn: trap[813]: Trap on CHIP ID 00050002
<1>[    9.502137] ffffffc07907f580: Warn: trap[620]: Trapped: TRAP_TD(00000200) DCAPT(3C000080)
<1>[    9.508741] ffffffc07907f580: Warn: trap[645]: Trapped: Thread: 9, reason: 00001000, PC: 3F0039DC, previous PC: 3F0039D8
<1>[    9.517082] ffffffc07907f580: Warn: trap[594]: A0_3: 3F012850 3F02DC4C 3F0127A8 00000000
<1>[    9.528013] ffffffc07907f580: Warn: trap[594]: A4_7: 3F02DB9C 3F0127E0 3F02DA4C 3F00D718
<1>[    9.536085] ffffffc07907f580: Warn: trap[599]: D0_3: 00000000 00000020 00000024 00003FFF
<1>[    9.544153] ffffffc07907f580: Warn: trap[599]: D4_7: 40000000 00000000 00001000 00000000
<1>[    9.552231] ffffffc07907f580: Warn: trap[599]: D8_11: 00000000 00000000 00000000 00000000
<1>[    9.560304] ffffffc07907f580: Warn: trap[599]: D12_15: 00000000 00000000 00000000 00000000
<1>[    9.568378] ffffffc07907f580: Warn: trap[649]: Thread_9 has non-recoverable trap
<0>[    9.576538] Kernel panic - not syncing: NSS FW coredump: bringing system down
<4>[    9.584084] CPU: 0 PID: 1052 Comm: napi/nss-13 Tainted: G           O       6.6.52 #0
<4>[    9.591115] Hardware name: Redmi AX6 (DT)
<4>[    9.598920] Call trace:
<4>[    9.602910]  dump_backtrace+0xa0/0xd8
<4>[    9.605168]  show_stack+0x18/0x24
<4>[    9.608988]  dump_stack_lvl+0x48/0x60
<4>[    9.612286]  dump_stack+0x18/0x24
<4>[    9.615932]  panic+0x2c8/0x314
<4>[    9.619229]  nss_fw_coredump_notify+0x1e4/0x3dc [qca_nss_drv]
<4>[    9.622187]  nss_core_handle_napi_emergency+0x38/0x4c [qca_nss_drv]
<4>[    9.628002]  __napi_poll+0x38/0x178
<4>[    9.634075]  napi_threaded_poll_loop+0x80/0x1bc
<4>[    9.637548]  napi_threaded_poll+0x78/0x84
<4>[    9.642061]  kthread+0x10c/0x110
<4>[    9.646226]  ret_from_fork+0x10/0x20
<2>[    9.649527] SMP: stopping secondary CPUs
<0>[    9.653091] Kernel Offset: disabled
<0>[    9.656991] CPU features: 0x0,00000000,00000000,0000400b
<0>[    9.660208] Memory Limit: none

innovara · 2024-10-01T14:49:01.223Z

@qosmio
I'm having constant and consistent reboots every 3 minutes or so on a MX4200v2 with the latest builds. The last build that works without any issue includes up to commit 5f8e5ea (Sat Sep 21 16:03:48 2024 -0400). Subsequent rebases have this problem. I am currently testing b32a80d (Mon Sep 30 23:38:59 2024 -0400).

I am trying to collect the trace with ramoops but I am facing another problem.

For the Linksys MX4200v2, ipq8174-mx4200.dtsi doesn't include ipq8074-512m.dtsi and neither does ipq8174-mx4200v2.dts. However, ipq8174-mx4200v1.dts does include ipq8074-512m.dtsi. So I reckoned for it to work on the MX4200v2, either #include "ipq8074-512m.dtsi" had to be promoted to ipq8174-mx4200.dtsi, or if that caused other problems ramoops_region should be added to ipq8174-mx4200v2.dts.

Problem? I've tried both options and the promotion doesn't boot at all and adding ramoops_region to ipq8174-mx4200v2.dts doesn't produce any output, even when I trigger the kernel panic with sh -c 'echo 10 > /proc/sys/kernel/panic; echo c > /proc/sysrq-trigger'. Is there anything you can suggest to get ramoops working?

EDIT: I've also tried to just add #include "ipq8074-512m.dtsi" to ipq8174-mx4200v2.dts and it boots but I still don't get anything even after triggering a kernel panic.

qosmio · 2024-10-01T17:21:32.530Z

innovara:

I'm having constant and consistent reboots every 3 minutes or so on a MX4200v2 with the latest builds. The last build that works without any issue includes up to commit 5f8e5ea (Sat Sep 21 16:03:48 2024 -0400). Subsequent rebases have this problem. I am currently testing b32a80d (Mon Sep 30 23:38:59 2024 -0400).

It's very strange to see. I've been up 19 hours on my dl-wrx36, mx5300, and mx4300.

image928×330 46.9 KB

Most of my commits in the last 24 hours have been patch refreshes. Can you try with reverting 3132334f0a - ath11k_nss: Remove unecessary ignore frags patch

innovara:

I am trying to collect the trace with ramoops but I am facing another problem.

If you build from qualcommax-6.x-nss-mx4300-6.11 branch, your device should populate the correct DTS entries.

I just verified the generated dtb

dtc -I dtb -O dts -o /dev/stdout build_dir/target-aarch64_cortex-a53_musl/linux-qualcommax_ipq807x/image-ipq8174-mx4200v2.dtb > /tmp/mage-ipq8174-mx4200v2.dts

Searching for ramoops in the dump I can see it.


		q6_etr_dump@50f00000 {
			no-map;
			reg = <0x00 0x50f00000 0x00 0x100000>;
		};

		m3_dump@51000000 {
			no-map;
			reg = <0x00 0x51000000 0x00 0x100000>;
		};

		ramoops@51200000 {
			compatible = "ramoops";
			reg = <0x00 0x51200000 0x00 0x100000>;
			no-map;
			record-size = <0x1000>;
		};		
	};

glassdoor:

will this patch apply to mainline openwrt as well?

@glassdoor, Yes, you can apply this to mainline as well. That's why I put that specific patch in "hack-6.6" vs. patches specific to "qualcommax". I'm assuming you had the same issue with UDP traffic hanging?

sppmaster:

But this "trick" doesn't work for me with ath11k 2.12.
It appears it's not only visual/cosmetic glitch because iper3 tests confirm it - speed from client to the router is cut in half.

Very strange to see. You said you were on NSS 12.2? Did you have issues with 11.4? I consistently get 160mhz on all my capable devices running NSS 11.4, and ath11k 2.9 (and 2.12).

georgem83:

@qosmio: what would it take to add NSS support for ipq50xx?
We’ve been working on this for a while at: IPQ5018: Support for Linksys MX2000 Atlas 6 & MX5500 Atlas 6 Pro - #266 by lytr

A hell of a lot tbh... Mostly because I don't have those devices, so it's a complete shot in the dark even if I tried. I know @BrainSlayer added support in DD-WRT, but NSS on IPQ5018 is still very unstable. Linksys has some weird version of NSS firmware they're using. None of the publicly available firmwares seem to load properly. It's also based on NSS 11.3, which doesn't support a lot of newer features like mesh.

sppmaster · 2024-10-01T17:58:10.541Z

qosmio:

Did you have issues with 11.4?

I've tried it only for a short time. To resume, I couldn't get Inactive Time to work on my bloated build at that time.
Now I've finally slimmed down my NSS build to minimally required NSS modules. And when I have free air at home I'll try again with 11.4.

asvio · 2024-10-01T18:08:31.095Z

Is this normal after reboot or sysupgrade?

Sin título1368×436 59.1 KB

If i restart radios stats goes normal.

lytr · 2024-10-01T18:23:35.489Z

qosmio:

Linksys has some weird version of NSS firmware they're using. None of the publicly available firmwares seem to load properly. It's also based on NSS 11.3, which doesn't support a lot of newer features like mesh.

I'm not sure, but I think that some IPQ5018 devices use the new NSS firmware:

MX6200 - NSS.FW.12.1-068-MP.R
GL-B3000 - NSS.FW.12.2-156-MP.R

asvio · 2024-10-01T18:30:25.967Z

qosmio:

I consistently get 160mhz on all my capable devices running NSS 11.4, and ath11k 2.9 (and 2.12)

I've a mediatek wifi7 smartphone that on 2.12 only get 1200Mbits/s with 160MHz
With 2.9 it get 2400Mbits/s
Sin título-1

innovara · 2024-10-01T18:32:13.587Z

I'm using the qualcommax-6.x-nss-wifi branch. The problem started with builds around 48h ago but it could be something that changed in OpenWRT's main.

The patch you suggest to revert is not in qualcommax-6.x-nss-wifi.

Re ramoops, I've checked with dtc and the output matches yours. Also dmesg shows what it's supposed to show in this regard.

I'll continue investigating and trying different things and get back to you when I have something more useful to share.

Buddy · 2024-10-01T20:53:18.005Z

If I currently have the release from arix00 installed for the MX4300 (https://github.com/arix00/openwrt-mx4300/releases), and I am trying to update to qosmio's latest commits (https://github.com/qosmio/openwrt-ipq), after I build the image, do I flash the "squashfs-sysupgrade.bin" or the "squashfs-factory.ubi"?

Just want to confirm since I'm not sure if there is anything I should look out for when updating openwrt from potentially different builds.

qosmio · 2024-10-01T21:02:24.827Z

lytr:

I'm not sure, but I think that some IPQ5018 devices use the new NSS firmware:

MX6200 - NSS.FW.12.1-068-MP.R

GL-B3000 - NSS.FW.12.2-156-MP.R

Sorry, I meant specifically for Linksys MX5500/MR5500.

NSS.MP.11.3-12-004-R is the highest and only known working NSS FW I believe. @BrainSlayer's that's still correct?

asvio:

Is this normal after reboot or sysupgrade?

Sin título1368×436 59.1 KB

If i restart radios stats goes normal.

A lot folks mentioned this too. @asvio,@sppmaster,@anom3

I remember seeing it a while back too, but have not been able to repeat this. I have some uncommitted changes in qca-nss-dp that might be why.

Could you guys try removing the "1" in the autoload section? This makes it so the module is loaded later in the boot sequence rather than on boot.

❯ git diff package/kernel/qca-nss-dp/Makefile
diff --git a/package/kernel/qca-nss-dp/Makefile b/package/kernel/qca-nss-dp/Makefile
index a917f0518c..4f63456d8a 100644
--- a/package/kernel/qca-nss-dp/Makefile
+++ b/package/kernel/qca-nss-dp/Makefile
@@ -22,7 +22,7 @@ define KernelPackage/qca-nss-dp
   DEPENDS:=@TARGET_qualcommax +kmod-qca-ssdk
   TITLE:=Qualcom NSS dataplane ethernet driver
   FILES:=$(PKG_BUILD_DIR)/qca-nss-dp.ko
-  AUTOLOAD:=$(call AutoLoad,31,qca-nss-dp,1)
+  AUTOLOAD:=$(call AutoLoad,31,qca-nss-dp)
 endef

Also, make sure you're using as much of the defaults as possible. So no manually selecting the NSS firmware, or MEM_PROFILE.

In my case the only things I changed related to nss/wifi:

enabled:

mesh

disabled:

pppoe
ath11k thermal
extended ath11k stats

You can see yours with

scripts/diffconfig.sh | grep -E "ATH11K|NSS_|MAC80211|IPQ|-nss"

Which outputs these changes

# CONFIG_ATH11K_DEBUGFS_HTT_STATS is not set
# CONFIG_ATH11K_DEBUGFS_STA is not set
CONFIG_ATH11K_NSS_MESH_SUPPORT=y
# CONFIG_ATH11K_THERMAL is not set
# CONFIG_NSS_DRV_PPPOE_ENABLE is not set
CONFIG_NSS_DRV_WIFI_MESH_ENABLE=y
CONFIG_PACKAGE_kmod-qca-nss-drv-wifi-meshmgr=y

Ideally, your diffconfig should have minimal output.

qosmio · 2024-10-01T21:05:34.335Z

lytr:

I think that sysupgrade -n and sysupgrade image is enough.

~~You'll need to flash squashfs-factory.ubi~~
That works to!

innovara:

Re ramoops, I've checked with dtc and the output matches yours. Also dmesg shows what it's supposed to show in this regard.

It could just be that the memory region needs to be shifted. It's a hit or miss really. FWIW, I have not been able to get ramoops working on my DL-WRX36, no matter how much a shift the regions. But MX4300/MX5300 and a lot of other boards seem to work. Your best bet is to use the UART console.

That's guaranteed to always show you all the output. I leave one connected to my dynalink and a RPi that's always logged in. It captures everything outputted to the terminal.

lytr · 2024-10-01T21:12:57.141Z

I think that sysupgrade -n and sysupgrade image is enough.

BramWrt · 2024-10-01T21:18:21.583Z

Hey qosmio, I tried forever to use your build but I have no idea how to do it, because I couldn't find a sysupgrade image on your branch. Is there anything I can do to get an NSS build that has netem, wireguard client, adblock and, sqm?

Thanks

qosmio · 2024-10-01T21:32:26.868Z

I don’t provide pre-built images because it's time-consuming and adds another layer of support, which I can’t commit to while actively developing and debugging the NSS fork and packages.

I pushed an updated README with a Quickstart guide that should help you build your own image. You could also check out @AgustinLorenzo’s builds if your device is supported.