Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

@ACwifidude @ninjanoir78 Thanks for the advice.

1 Like

Yes, I’ve been building for chromium targets. I’d try the 23.05 version in post #1 above. Let me know how it works.

Awesome, i'll test it after friday, and i want to play with it's integrated speaker, leds and zigbee module. Will it handle home assistant? Can't wait yo play with it :smile_cat:.

Would it be safe to upgrade from 22.03 (Stable) 20230607 (r20168+33-171b515192) to 23.05 (Stable) 20240115 while keeping settings?

I can't remember if I installed ath10k or ath10k-ct .... :pensive: ... how to find out?

check installed softwares in luci.

I did before asking and I saw both :wink:

Taking a second look I may look like -ct:

ath10k-board-qca9984
ath10k-firmware-qca9984-ct
kmod-ath10k-ct

But not 100% sure about it.

my router is non-ct, like this:

ath10k-board-qca9984	        20230804-1
ath10k-firmware-qca9984	        20230804-1
kmod-ath10k                  	5.15.137+6.1.24-3
1 Like

Thanks, that confirms it. Why did you choose for the non-ct?
When I read (now) there are issues with it I would probably do the same... What I know is that non-ct is slower, according to post #1.

Is it safe to move from 22 to 23 with keeping settings? And would it be safe making a switch from ct to non-ct with current config?

I tested on my mobile phone many times, non-ct is faster.

I do not have special softwares installed, so upgrade to 23 or switch non-ct/ct no issues.

1 Like

I'm new to using openwrt, I installed nss build to take advantage of the gigabit connection on my xr500 when I'm not gaming. I would be really grateful if anyone had some settings for prioritizing traffic to call of duty ports and settings in general.

Hi there, I have the archer c2600 and I'm intent on using it as an access point in my apartment. I've been reading up on AQL and I wanted to know if all current versions of this build have it enabled and which one to use? Right now my wifi is the bottle neck and I'd like to improve latency on my wifi connections. Putting sqm on the uplink to the modem on my raspberry pi only works well when I'm a couple of feet away from the access point, it works great on wired connections though, and going into another room my wifi latency spikes. As I understand it, using AQL drivers improves this, so I'd like to know which one of these builds v22? or v23? would be better for my use case? I also have an old archer C7v2 sitting in the closet, does default openwrt firmware have AQL for that router in its drivers, or are there similar builds for that router too? And finally, is there a better choice in routers or access points that have some sort of sqm in their wifi drivers that helps with latency spikes on wifi connections? I've tried a couple of cheap wifi6 routers as access points (Huawei and Tenda) and they don't help when moving away from the access points. So I'm looking for an openwrt solution that isn't too expensive and something that I can repurpose my old routers for.

finaly managed to install openwrt on this naughy thing, so , there is it , works quite well, need to test more :smiley: THANK YOU !

After upgrading to a 1 Gbit fiber connection (coming from a 200 Mbit cable ISP) I just installed your NSS Build on my TP Link C2600. I am impressed by performance increase compared to standard Openwrt Stable v23.05.2. I performed some testing by Speedtest.net (down/up) between the ath10k, ath10k-ct and openwrt standard builds. Later I added iperf3 tests as comparison with almost the same results as speedtest.net. I chose the ath10k-ct driver because it’s more open as mentioned on the C2600 device page.

NSS builds:

ath10k-ct: 5 GHz Wifi: 503 / 743 Mbps
ath10k: 5 GHz Wifi: 546 / 742 Mbps
Cable: 938 / 942 Mbps

Compared to standard Openwrt stable build (None NSS) – EDIT: New test on clean install:

v23.05.2 stable: 5 Ghz Wifi: 379 / 694 Mbps
Cable: 939 / 939 Mbps
v23.05.2 stable ct-driver/non-ct firmware: 5 Ghz Wifi: 460-540 / 740 Mbps
Cable: 934/ 934 Mbps

Your NSS builds (ath10k-CT) just improved my Wifi 5 GHz performance by approx. 30% more download speed and on the cable the performance remained the same compared to standard openwrt Stable v23.05.2. Nice. Thanks!

Edit: Units update to Mbps AND iperf3 test for ath10k-ct included; first by cable connected and then 5 GHz Wifi test
Edit2: Performed new v23.05.2 stable speedtest. My original test was not representative. Now cable connected speed is the same by NSS and none-NSS build and Wifi 5 Ghz speeds are more closer together. Also included a new iperf3 tests.
Edit3: Added results for special build by firmware selector, experimented with package steering (on/off) and software flow offloading).

iperf3 test ath10k-ct
ath10k-ct iperf3 testing from Windows Laptop 

Cable connection:
C:\temp>iperf3.exe -c 192.168.2.39 -R -t 10
Connecting to host 192.168.2.39, port 5201
Reverse mode, remote host 192.168.2.39 is sending
[  4] local 192.168.2.2 port 50622 connected to 192.168.2.39 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   113 MBytes   947 Mbits/sec
[  4]   1.00-2.00   sec   113 MBytes   945 Mbits/sec
[  4]   2.00-3.00   sec   113 MBytes   946 Mbits/sec
[  4]   3.00-4.00   sec   113 MBytes   946 Mbits/sec
[  4]   4.00-5.00   sec   113 MBytes   947 Mbits/sec
[  4]   5.00-6.00   sec   113 MBytes   947 Mbits/sec
[  4]   6.00-7.00   sec   113 MBytes   947 Mbits/sec
[  4]   7.00-8.00   sec   113 MBytes   946 Mbits/sec
[  4]   8.00-9.00   sec   113 MBytes   946 Mbits/sec
[  4]   9.00-10.00  sec   113 MBytes   946 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   947 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.10 GBytes   946 Mbits/sec                  receiver

iperf Done.

Ath10k-ct Wifi 5 GHz (from windows laptop Intel AX210, Wifi 80 Mhz channel): 
C:\temp>iperf3 -c 192.168.2.39 -t 10
Connecting to host 192.168.2.39, port 5201
[  4] local 192.168.2.4 port 52118 connected to 192.168.2.39 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  57.5 MBytes   482 Mbits/sec
[  4]   1.00-2.00   sec  59.4 MBytes   498 Mbits/sec
[  4]   2.00-3.00   sec  59.8 MBytes   501 Mbits/sec
[  4]   3.00-4.00   sec  59.9 MBytes   502 Mbits/sec
[  4]   4.00-5.00   sec  60.0 MBytes   503 Mbits/sec
[  4]   5.00-6.00   sec  59.9 MBytes   502 Mbits/sec
[  4]   6.00-7.00   sec  59.9 MBytes   503 Mbits/sec
[  4]   7.00-8.00   sec  60.2 MBytes   505 Mbits/sec
[  4]   8.00-9.00   sec  59.6 MBytes   500 Mbits/sec
[  4]   9.00-10.00  sec  60.0 MBytes   503 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   596 MBytes   500 Mbits/sec                  sender
[  4]   0.00-10.00  sec   596 MBytes   500 Mbits/sec                  receiver

iperf Done.
iperf3 test v23.05.2 stable (none-NSS)
v23.05.2 stable (none-NSS):
cable connected
C:\temp>iperf3.exe -c 192.168.2.39 -R -t 10
Connecting to host 192.168.2.39, port 5201
Reverse mode, remote host 192.168.2.39 is sending
[  4] local 192.168.2.2 port 49416 connected to 192.168.2.39 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   113 MBytes   945 Mbits/sec
[  4]   1.00-2.00   sec   113 MBytes   945 Mbits/sec
[  4]   2.00-3.00   sec   113 MBytes   946 Mbits/sec
[  4]   3.00-4.00   sec   113 MBytes   947 Mbits/sec
[  4]   4.00-5.00   sec   113 MBytes   946 Mbits/sec
[  4]   5.00-6.00   sec   113 MBytes   947 Mbits/sec
[  4]   6.00-7.00   sec   113 MBytes   946 Mbits/sec
[  4]   7.00-8.00   sec   113 MBytes   945 Mbits/sec
[  4]   8.00-9.00   sec   113 MBytes   946 Mbits/sec
[  4]   9.00-10.00  sec   113 MBytes   945 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   948 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.10 GBytes   948 Mbits/sec                  receiver

iperf Done.

v23.05.2 stable (none-nss) Wifi 5 GHz (from windows laptop Intel AX210, Wifi 80 Mhz channel): 
C:\temp>iperf3.exe -c 192.168.2.39 -R -t 10
Connecting to host 192.168.2.39, port 5201
Reverse mode, remote host 192.168.2.39 is sending
[  4] local 192.168.2.4 port 64464 connected to 192.168.2.39 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  46.7 MBytes   391 Mbits/sec
[  4]   1.00-2.00   sec  47.3 MBytes   397 Mbits/sec
[  4]   2.00-3.00   sec  46.3 MBytes   388 Mbits/sec
[  4]   3.00-4.00   sec  46.4 MBytes   389 Mbits/sec
[  4]   4.00-5.00   sec  46.3 MBytes   389 Mbits/sec
[  4]   5.00-6.00   sec  45.1 MBytes   378 Mbits/sec
[  4]   6.00-7.00   sec  45.7 MBytes   384 Mbits/sec
[  4]   7.00-8.00   sec  45.6 MBytes   383 Mbits/sec
[  4]   8.00-9.00   sec  45.0 MBytes   377 Mbits/sec
[  4]   9.00-10.00  sec  45.7 MBytes   383 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   461 MBytes   387 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   461 MBytes   386 Mbits/sec                  receiver

iperf3, speedtest v23.05.2 ct-driver/non-ct firmware)
Special build by firmware selector, experimented with package steering (on/off) and software flow offloading). 
This increased the v23.05.2 openwrt stable build from around 370-380 Mbps to around 460-540 Mbps download speeds. The iperf3 speeds were quite lower as the results with speedtest.net

settings	   down [Mbps]       up [Mbps]       Iperf3 [Mbits/s]
PS (off), SFO (off)	   409-538	         601-722            388
PS (ON), SFO (off)	   475-509	         723-736            387
PS (ON), SFO (ON)    512-542	         722-737            397
PS (off), SFO (ON)	    459-552	         723-741            386


3 Likes

Xiaomi R3D (ipq8064), OpenWRT 23.05.2 (w/o NSS), ath10k-CT, client Win10 + Intel AX211:

iperf3 test
c:\_system\nperf3>iperf3.exe -c 192.168.2.4 -t 10
Connecting to host 192.168.2.4, port 5201
[  5] local 192.168.2.77 port 52128 connected to 192.168.2.4 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  27.8 MBytes   232 Mbits/sec
[  5]   1.00-2.00   sec  49.9 MBytes   420 Mbits/sec
[  5]   2.00-3.01   sec  51.6 MBytes   431 Mbits/sec
[  5]   3.01-4.00   sec  49.2 MBytes   415 Mbits/sec
[  5]   4.00-5.00   sec  48.8 MBytes   407 Mbits/sec
[  5]   5.00-6.00   sec  50.9 MBytes   427 Mbits/sec
[  5]   6.00-7.00   sec  51.1 MBytes   429 Mbits/sec
[  5]   7.00-8.00   sec  49.9 MBytes   419 Mbits/sec
[  5]   8.00-9.00   sec  44.9 MBytes   376 Mbits/sec
[  5]   9.00-10.01  sec  48.0 MBytes   401 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec   472 MBytes   396 Mbits/sec                  sender
[  5]   0.00-10.03  sec   472 MBytes   394 Mbits/sec                  receiver

iperf Done.

c:\_system\nperf3>iperf3.exe -c 192.168.2.4 -R -t 10
Connecting to host 192.168.2.4, port 5201
Reverse mode, remote host 192.168.2.4 is sending
[  5] local 192.168.2.77 port 52135 connected to 192.168.2.4 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  45.0 MBytes   377 Mbits/sec
[  5]   1.00-2.00   sec  56.6 MBytes   475 Mbits/sec
[  5]   2.00-3.00   sec  50.4 MBytes   422 Mbits/sec
[  5]   3.00-4.00   sec  41.5 MBytes   349 Mbits/sec
[  5]   4.00-5.00   sec  41.8 MBytes   351 Mbits/sec
[  5]   5.00-6.00   sec  43.7 MBytes   366 Mbits/sec
[  5]   6.00-7.00   sec  47.6 MBytes   399 Mbits/sec
[  5]   7.00-8.00   sec  39.6 MBytes   332 Mbits/sec
[  5]   8.00-9.00   sec  45.3 MBytes   381 Mbits/sec
[  5]   9.00-10.00  sec  40.6 MBytes   340 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.02  sec   455 MBytes   381 Mbits/sec  128             sender
[  5]   0.00-10.00  sec   452 MBytes   379 Mbits/sec                  receiver

iperf Done.

WiFi 5G: 396 / 380 Mbps

Xiaomi R3D (ipq8064), OpenWRT 23.05.0-rc2 (with NSS), ath10k-CT:
WiFi 5G: 680 / 363 Mbps

2 Likes

I've just detected a router crash - R7800 with 23.05 NSS snapshot self compiled build (ACwifidude repo).
If someone wants to take a look. It was after 20 days run.
ramoops-0 file from pstore.

cd xhci-hcd.1.auto: xHCI Host Controller
[    6.615166] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 4
[    6.620724] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0 SuperSpeed
[    6.628585] hub 3-0:1.0: USB hub found
[    6.634904] hub 3-0:1.0: 1 port detected
[    6.638766] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
[    6.643141] hub 4-0:1.0: USB hub found
[    6.650799] hub 4-0:1.0: 1 port detected
[    6.657434] usbcore: registered new interface driver usb-storage
[    6.659236] usbcore: registered new interface driver uas
[    6.664616] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    6.680251] init: - preinit -
[    8.396919] ipq8064-mdio 37000000.mdio eth1: nss_gmac_start_up: Force link up
[    8.396957] ipq8064-mdio 37000000.mdio eth1: 1000 Mbps Full Duplex
[    8.403107] ipq8064-mdio 37000000.mdio eth1: nss_gmac_dev_set_speed:NSS_ETH_CLK_DIV0(0xc) - 0x100
[    8.409120] ipq8064-mdio 37000000.mdio eth1: nss_gmac_dev_set_speed: qsgmii_base(0xde8c0000) + PCS_MODE_CTL(0x68): 0x80008082
[    8.418120] ipq8064-mdio 37000000.mdio eth1: nss_gmac_reset: eth1 resetting...
[    9.536418] ipq8064-mdio 37000000.mdio eth1: GMAC reset completed in 111 jiffies; dma_bus_mode - 0x20100
[    9.536695] ipq8064-mdio 37000000.mdio eth1: nss_gmac_change_mtu: Enabling Normal Frame MTU (Requested MTU [1500])
[    9.545037] ipq8064-mdio 37000000.mdio eth1: nss_gmac_change_mtu: Enabling Jumbo Frame MTU for eth1 (Requested MTU [1500])
[    9.555189] ipq8064-mdio 37000000.mdio eth1: Set addr b0:7f:b9:3e:4f:ad
[    9.566216] ipq8064-mdio 37000000.mdio eth1: nss_gmac_ipc_offload_init: enable Rx checksum
[    9.572713] ipq8064-mdio 37000000.mdio eth1: nss_gmac_config_flow_control:
[    9.581053] ipq8064-mdio 37000000.mdio eth1: nss_gmac_rx_pause_disable: disable Rx flow control
[    9.587852] ipq8064-mdio 37000000.mdio eth1: nss_gmac_tx_pause_disable: disable Tx flow control
[    9.596956] ipq8064-mdio 37000000.mdio eth1: nss_gmac_linkup: data plane open command successfully issued
[    9.606051] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[    9.615220] IPv6: ADDRCONF(NETDEV_CHANGE): eth1.1: link becomes ready
[   14.719048] Starting lvm2 during preinit
[   15.482613] mount_root: loading kmods from internal overlay
[   15.504553] kmodloader: loading kernel modules from //etc/modules-boot.d/*
[   15.509140] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
[   15.569052] UBIFS (ubi0:1): Mounting in unauthenticated mode
[   15.570714] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 225
[   15.686775] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
[   15.686809] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[   15.693697] UBIFS (ubi0:1): FS size: 80375808 bytes (76 MiB, 633 LEBs), max 644 LEBs, journal size 4063232 bytes (3 MiB, 32 LEBs)
[   15.703574] UBIFS (ubi0:1): reserved for root: 3796347 bytes (3707 KiB)
[   15.715224] UBIFS (ubi0:1): media format: w5/r0 (latest is w5/r0), UUID 414EA552-354E-45A6-8E62-5819C050EDC6, small LPT model
[   15.722513] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
[   15.733288] block: unable to load configuration (fstab: Entry not found)
[   15.739855] block: attempting to load /tmp/ubifs_cfg/etc/config/fstab
[   15.746874] block: unable to load configuration (fstab: Entry not found)
[   15.753189] block: attempting to load /etc/config/fstab
[   15.761275] block: unable to load configuration (fstab: Entry not found)
[   15.764841] block: no usable configuration
[   15.774317] UBIFS (ubi0:1): un-mount UBI device 0
[   15.775683] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" stops
[   15.784091] UBIFS (ubi0:1): Mounting in unauthenticated mode
[   15.786575] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 226
[   15.903270] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
[   15.903303] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[   15.910113] UBIFS (ubi0:1): FS size: 80375808 bytes (76 MiB, 633 LEBs), max 644 LEBs, journal size 4063232 bytes (3 MiB, 32 LEBs)
[   15.920091] UBIFS (ubi0:1): reserved for root: 3796347 bytes (3707 KiB)
[   15.931724] UBIFS (ubi0:1): media format: w5/r0 (latest is w5/r0), UUID 414EA552-354E-45A6-8E62-5819C050EDC6, small LPT model
[   15.960918] block: attempting to load /tmp/ubifs_cfg/upper/etc/config/fstab
[   15.961027] block: unable to load configuration (fstab: Entry not found)
[   15.966737] block: attempting to load /tmp/ubifs_cfg/etc/config/fstab
[   15.973811] block: unable to load configuration (fstab: Entry not found)
[   15.980016] block: attempting to load /etc/config/fstab
[   15.986879] block: unable to load configuration (fstab: Entry not found)
[   15.991836] block: no usable configuration
[   15.999684] mount_root: overlay filesystem has not been fully initialized yet
[   16.003411] mount_root: switching to ubifs overlay
[   16.331584] urandom-seed: Seed file not found (/etc/urandom.seed)
[   16.433704] procd: - early -
[   16.433800] procd: - watchdog -
[   16.435715] procd: Watchdog has previously reset the system
[   16.968183] procd: - watchdog -
[   16.968263] procd: Watchdog has previously reset the system
[   16.970331] procd: - ubus -
[   17.031074] procd: - init -
[   17.639272] kmodloader: loading kernel modules from /etc/modules.d/*
[   17.650176] NET: Registered PF_ALG protocol family
[   17.666165] Key type dns_resolver registered
[   17.694376] RPC: Registered named UNIX socket transport module.
[   17.694410] RPC: Registered udp transport module.
[   17.699098] RPC: Registered tcp transport module.
[   17.704054] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   17.712982] tun: Universal TUN/TAP device driver, 1.6
[   17.721073] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[   17.725108] gre: GRE over IPv4 demultiplexor driver
[   17.726799] ip_gre: GRE over IPv4 tunneling driver
[   17.762065] NFS: Registering the id_resolver key type
[   17.762116] Key type id_resolver registered
[   17.766094] Key type id_legacy registered
[   17.772095] cryptodev: driver 1.12 loaded.
[   17.777194] PPP generic driver version 2.4.2
[   17.778863] NET: Registered PF_PPPOX protocol family
[   17.786211] PPTP driver version 0.8.5
[   17.791453] <NSS-CRYPTO>:module loaded (platform - IPQ806x, build - Build_ID - 01/31/24, 03:14:44)
[   17.791657] <NSS-CRYPTO>:Register with NSS driver-
[   17.803573] GACT probability on
[   17.805521] Mirror/redirect action on
[   17.814197] u32 classifier
[   17.814224]     input device check on
[   17.815791]     Actions configured
[   17.833005] fuse: init (API version 7.34)
[   17.858582] urngd: v1.0.2 started.
[   17.861203] Loading modules backported from Linux version v6.1.24-0-g0102425ac76b
[   17.861234] Backport generated by backports.git v5.15.92-1-44-gd6ea70fafd36
[   17.900764] <NSS-CRYPTO>:NSS Firmware initialized
[   17.900968] <NSS-CRYPTO>:probing engine - 0
[   17.904448] <NSS-CRYPTO>:Device Tree node found
[   17.908554] <NSS-CRYPTO>:init completed for Pipe Pair[0]
[   17.913053] <NSS-CRYPTO>:init completed for Pipe Pair[1]
[   17.930392] xt_time: kernel timezone is -0000
[   17.942959] <NSS-CRYPTO>:probing engine - 1
[   17.942988] <NSS-CRYPTO>:Device Tree node found
[   17.946012] <NSS-CRYPTO>:init completed for Pipe Pair[0]
[   17.950471] <NSS-CRYPTO>:init completed for Pipe Pair[1]
[   17.980856] <NSS-CRYPTO>:probing engine - 2
[   17.980884] <NSS-CRYPTO>:Device Tree node found
[   17.983904] <NSS-CRYPTO>:init completed for Pipe Pair[0]
[   17.988360] <NSS-CRYPTO>:init completed for Pipe Pair[1]
[   18.003576] <NSS-CRYPTO>:probing engine - 3
[   18.003605] <NSS-CRYPTO>:Device Tree node found
[   18.006628] <NSS-CRYPTO>:init completed for Pipe Pair[0]
[   18.011213] <NSS-CRYPTO>:init completed for Pipe Pair[1]
[   18.085242] wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information.
[   18.085276] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
[   18.106488] ath10k_pci 0000:01:00.0: enabling device (0140 -> 0142)
[   18.107052] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[   21.414438] ath10k_pci 0000:01:00.0: qca9984/qca9994 hw1.0 target 0x01000000 chip_id 0x00000000 sub 168c:cafe
[   21.414485] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   21.425298] ath10k_pci 0000:01:00.0: firmware ver 10.4-3.9.0.2-00157 api 5 features no-p2p,mfp,peer-flow-ctrl,btcoex-param,allows-mesh-bcast,no-ps,peer-fixed-rate,iram-recovery crc32 6cdc6ff9
[   23.741148] ath10k_pci 0000:01:00.0: board_file api 2 bmi_id 0:1 crc32 0bfe5c35
[   27.534770] ath10k_pci 0000:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-nvmem max-sta 512 raw 0 hwcrypto 1
[   27.626124] wlan0: Created a NSS virtual interface
[   27.628131] ath10k_pci 0001:01:00.0: enabling device (0140 -> 0142)
[   27.630435] ath10k_pci 0001:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[   28.149204] ath10k_pci 0001:01:00.0: qca9984/qca9994 hw1.0 target 0x01000000 chip_id 0x00000000 sub 168c:cafe
[   28.149248] ath10k_pci 0001:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   28.160137] ath10k_pci 0001:01:00.0: firmware ver 10.4-3.9.0.2-00157 api 5 features no-p2p,mfp,peer-flow-ctrl,btcoex-param,allows-mesh-bcast,no-ps,peer-fixed-rate,iram-recovery crc32 6cdc6ff9
[   30.441140] ath10k_pci 0001:01:00.0: board_file api 2 bmi_id 0:2 crc32 0bfe5c35
[   34.220338] ath10k_pci 0001:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-nvmem max-sta 512 raw 0 hwcrypto 1
[   34.305365] debugfs: File 'virt_if' in directory 'stats' already present!
[   34.305858] wlan1: Created a NSS virtual interface
[   34.314489] kmodloader: done loading kernel modules from /etc/modules.d/*
[   37.683929] ECM init ( QSDK 12.1.r5_CS1 2022-11-25 1c401a1 )
[   37.683992] ECM database jhash random seed: 0x5a0a38b4
[   37.689730] ECM init complete
[   40.094561] ipq8064-mdio 37000000.mdio eth1: nss_gmac_tx_pause_disable: disable Tx flow control
[   40.095215] ipq8064-mdio 37000000.mdio eth1: nss_gmac_rx_pause_disable: disable Rx flow control
[   40.105074] ipq8064-mdio 37000000.mdio eth1: nss_gmac_start_up: Force link up
[   40.110820] ipq8064-mdio 37000000.mdio eth1: 1000 Mbps Full Duplex
[   40.118055] ipq8064-mdio 37000000.mdio eth1: nss_gmac_dev_set_speed:NSS_ETH_CLK_DIV0(0xc) - 0x100
[   40.124210] ipq8064-mdio 37000000.mdio eth1: nss_gmac_dev_set_speed: qsgmii_base(0xde8c0000) + PCS_MODE_CTL(0x68): 0x80008082
[   40.133155] ipq8064-mdio 37000000.mdio eth1: Set addr b0:7f:b9:3e:4f:ad
[   40.144434] ipq8064-mdio 37000000.mdio eth1: nss_gmac_ipc_offload_init: enable Rx checksum
[   40.150832] ipq8064-mdio 37000000.mdio eth1: nss_gmac_config_flow_control:
[   40.159098] ipq8064-mdio 37000000.mdio eth1: nss_gmac_rx_pause_disable: disable Rx flow control
[   40.166050] ipq8064-mdio 37000000.mdio eth1: nss_gmac_tx_pause_disable: disable Tx flow control
[   40.174816] ipq8064-mdio 37000000.mdio eth1: nss_gmac_linkup: data plane open command successfully issued
[   40.186231] br-lan: port 1(eth1.1) entered blocking state
[   40.193050] br-lan: port 1(eth1.1) entered disabled state
[   40.198531] device eth1.1 entered promiscuous mode
[   40.203813] device eth1 entered promiscuous mode
[   40.211531] br-lan: port 1(eth1.1) entered blocking state
[   40.213189] br-lan: port 1(eth1.1) entered forwarding state
[   40.229446] ipq8064-mdio 37000000.mdio eth0: nss_gmac_tx_pause_disable: disable Tx flow control
[   40.229541] ipq8064-mdio 37000000.mdio eth0: nss_gmac_rx_pause_disable: disable Rx flow control
[   40.237470] ipq8064-mdio 37000000.mdio eth0: nss_gmac_start_up: Force link up
[   40.245720] ipq8064-mdio 37000000.mdio eth0: 1000 Mbps Full Duplex
[   40.253007] ipq8064-mdio 37000000.mdio eth0: nss_gmac_dev_set_speed:NSS_ETH_CLK_DIV0(0xc) - 0x100
[   40.259009] ipq8064-mdio 37000000.mdio eth0: nss_gmac_reset: eth0 resetting...
[   41.370797] ipq8064-mdio 37000000.mdio eth0: GMAC reset completed in 111 jiffies; dma_bus_mode - 0x20100
[   41.371006] ipq8064-mdio 37000000.mdio eth0: nss_gmac_change_mtu: Enabling Normal Frame MTU (Requested MTU [1500])
[   41.379360] ipq8064-mdio 37000000.mdio eth0: Set addr b0:7f:b9:3e:4f:ae
[   41.389642] ipq8064-mdio 37000000.mdio eth0: nss_gmac_ipc_offload_init: enable Rx checksum
[   41.396098] ipq8064-mdio 37000000.mdio eth0: nss_gmac_config_flow_control:
[   41.404432] ipq8064-mdio 37000000.mdio eth0: nss_gmac_rx_pause_disable: disable Rx flow control
[   41.411288] ipq8064-mdio 37000000.mdio eth0: nss_gmac_tx_pause_disable: disable Tx flow control
[   41.420159] ipq8064-mdio 37000000.mdio eth0: nss_gmac_linkup: data plane open command successfully issued
[   41.430056] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   41.439899] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   41.446889] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   42.005423] ath: EEPROM regdomain: 0x8064
[   42.005455] ath: EEPROM indicates we should expect a country code
[   42.008443] ath: doing EEPROM country->regdmn map search
[   42.014578] ath: country maps to regdmn code: 0x37
[   42.019878] ath: Country alpha2 being used: BG
[   42.024544] ath: Regpair used: 0x37
[   42.028896] ath: regdomain 0x8064 dynamically updated by user
[   42.032376] ath: EEPROM regdomain: 0x8064
[   42.038187] ath: EEPROM indicates we should expect a country code
[   42.042256] ath: doing EEPROM country->regdmn map search
[   42.048268] ath: country maps to regdmn code: 0x37
[   42.053689] ath: Country alpha2 being used: BG
[   42.058240] ath: Regpair used: 0x37
[   42.062720] ath: regdomain 0x8064 dynamically updated by user
[   43.164042] wlan0: Destroyed NSS virtual interface
[   43.313003] wlan1: Destroyed NSS virtual interface
[   44.002653] debugfs: File 'virt_if' in directory 'stats' already present!
[   44.002969] phy0-ap0: Created a NSS virtual interface
[   50.344704] ath10k_pci 0000:01:00.0: Unknown eventid: 36933
[   50.348817] br-lan: port 2(phy0-ap0) entered blocking state
[   50.349107] br-lan: port 2(phy0-ap0) entered disabled state
[   50.354904] device phy0-ap0 entered promiscuous mode
[   50.360284] br-lan: port 2(phy0-ap0) entered blocking state
[   50.365469] br-lan: port 2(phy0-ap0) entered forwarding state
[   50.411039] br-lan: port 2(phy0-ap0) entered disabled state
[   50.755003] IPv6: ADDRCONF(NETDEV_CHANGE): phy0-ap0: link becomes ready
[   50.755164] br-lan: port 2(phy0-ap0) entered blocking state
[   50.760432] br-lan: port 2(phy0-ap0) entered forwarding state
[   51.127521] debugfs: File 'virt_if' in directory 'stats' already present!
[   51.127843] phy1-ap0: Created a NSS virtual interface
[   57.626182] ath10k_pci 0001:01:00.0: Unknown eventid: 36933
[   57.629185] br-lan: port 3(phy1-ap0) entered blocking state
[   57.630578] br-lan: port 3(phy1-ap0) entered disabled state
[   57.636376] device phy1-ap0 entered promiscuous mode
[   57.641821] br-lan: port 3(phy1-ap0) entered blocking state
[   57.646884] br-lan: port 3(phy1-ap0) entered forwarding state
[   57.683102] br-lan: port 3(phy1-ap0) entered disabled state
[   58.288953] IPv6: ADDRCONF(NETDEV_CHANGE): phy1-ap0: link becomes ready
[   58.289120] br-lan: port 3(phy1-ap0) entered blocking state
[   58.294434] br-lan: port 3(phy1-ap0) entered forwarding state
[51953.301253] phy1-ap0: NSS TX failed with error: NSS_TX_FAILURE_TOO_SHORT
[51981.181416] phy1-ap0: NSS TX failed with error: NSS_TX_FAILURE_TOO_SHORT
[602310.220109] ath10k_pci 0000:01:00.0: failed to flush transmit queue (skip 0 ar-state 1): 0
[776236.487520] ath10k_pci 0000:01:00.0: failed to flush transmit queue (skip 0 ar-state 1): 0
[1382086.530111] ath10k_pci 0001:01:00.0: failed to flush transmit queue (skip 0 ar-state 1): 0
[1382086.530528] ath10k_pci 0001:01:00.0: Invalid peer id 79 peer stats buffer
[1769471.822669] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[1769471.822706] rcu: 	1-...!: (1 GPs behind) idle=f7b/0/0x1 softirq=42970089/42970090 fqs=1 
[1769471.827746] 	(detected by 0, t=2102 jiffies, g=84375901, q=14)
[1769471.835903] Sending NMI from CPU 0 to CPUs 1:

I've had one of these as well (rcu stall) - note the stall duration: 2102 jiffies, or 21.02 seconds (assuming 100 jiffies/second - I'm going from memory so be kind if I'm wrong).

20 seconds is the default linux rcu stall timeout limit which (my .00002) is a bit crazy given the realtime nature of the device's functions.

If you want to see more of these more often, cut that time down to a second or 2 - entirely reasonable given the activity of an openwrt router. It takes rebuilding the kernel after changing the define for that default tmout.

I tried to chase this some time ago, but got little interest, and needed more openwrt and NSS expertise than i possess.

In broad terms, the core handling the idle task is stuck for 20 seconds (maybe hung in an isr) - a timer fires and the core is rebooted. Which is pretty cool, but renders meaningless uptime as an indicator of build quality.

And the router hangs for 20+ seconds.

The length of this timeout masks/hides significant stalls of < 20 seconds - the only indication is a pause in traffic, for whole seconds sometimes.

Mmmv,
M.

1 Like

And continuing from a higher-level perspective - those shorter pauses will surely confound work in the sqm/net-responsiveness area. No point in going there if something is hanging the router intermittently for varying lengths of time.

1 Like

Those wifi/cabled speeds seem a bit slow for the standard build. Can you redo them with sfe and packet steering enabled (and maybe different combinationa with/without irqbalance package installed and enabled.

1 Like

Perhaps I've missed it, but in both this thread & the "exploration" threads I can't find more info on CPU clocking. There's the scaling_max, but no option to go beyond that? Aka overclocking? Is it baked into the image or hardware?

Also any ideas how to change the kernel command line options? Default it ignores the bootloader options, and uboot doesn't seem to pass anything via bootiqp2.

Agree, I performed some new speedtests on a clean install. Now the none-NSS builds and NSS build are closer together. I edited my post with updated results.