IPQ807X NSS Build

I experienced the same issue, and it seems always happen when a new wireless client connects. Reverted to 07/09 seems help so far. Do you or anyone happen to know what could be the cause?

Hi sppmaster, thanks for your suggestion.
I've tried it:

  • changes to /etc/config/ecm
  • one liner "ethtool..." in /etc/rc.local
  • followed by a reboot

...but 5G LAN > WAN still only on 47 Mbps (U/D, server's log):

Have also tried di_Niko's (thank you too) suggestion with same results.

On the other hand, connected my Pixel 7 on 2.4G (Ch 1, 20MHz, GB) and I get better results (U/D, server's log):

Anyhow, am not too fussed if NSS build doesn't work as it should at the moment. At least there are other choices if the need arises: 2.4G / stable / snapshot.

Am still open to suggestions though.

Here is WLAN <-> LAN.
Iperf3 server is actually Pixel 7 (on Dynalink sppmaster's bits are in, Ch 36, 80 MHz, GB) and client is Laptop. Couldn't find iperf3 for Windows which has --bidir implemented so used WSL2 (Ubuntu) for this.

Thank you. That's a lot better than with no NSS. :+1:

Only adding this line will not resolve the issue.

Are you sure that the netfilter_optional_tcp_window_check.patc.patch was properly applied to the NSS build you are running before compiling.

You can get the latest iperf3 for Windows here.

You make sure that you test on https://www.waveform.com/tools/bufferbloat too.

Hi Agustin,

Is there a way I can add this to a customized build for CR1000A router?

Here is the @a_guy 's branch for CR1000A: https://github.com/tsg2k2/openwrt

Your first concern should be getting the CR1000A supported by plain OpenWrt (without NSS) in master first, only then it's reasonably possible to get it into these NSS based community builds.

Hi, thanks for the follow up.
To clear things up this is how it all played out at my end:

  • Flash @AgustinLorenzo's pre-built image for Dynalink
  • Iperf3 WLAN > WAN issue noticed
  • Flash Dynalink image built from @dimfish's repos. Used dimfish's build configuration file and just changed target to Dynalink
  • Iperf3 WLAN > WAN issue remains
  • Re-Flash Agustin's image
  • Verify issue persists and ask for help here
  • Try suggested fix (on Agustin's image): issue remains

Worth noting all repos involved in above have that '' patch + NSS setting in them.

TL;DR
HOWEVER, your follow up reminded me that I haven't tried the fix on that, what I consider, @dimfish's build for Dynalink.
And...the fix works there.

Here is bufferbloat you requested from Pixel 7. Also note that, if it matters, Dynalink is behind home's main router + switch.

Thank you again for pointing me in the right direction!

A little follow up:
Been cloning Codelinaro QCA NSS repos on wired laptop.

Side-track (apologies if this has been posted before): apparently branches to use (as have been most tested and are active) for NSS packages are NHSS.QSDK.12.4. Current NSS build (mostly) uses 12.4.r1 which are 6+ mts out of date in some instances and they feel abandoned.

Anyhow...starting up VSCode in one cloned repo (in WSL2) I got, what it looks to me, an SSL error frequently mentioned in this thread. Retrying the command worked OK. If I notice more of these I will report.

1 Like

Probably @bitthief and others who adapt the QCA NSS firmware to the OpenWrt code can comment on this.
I'm glad that you resolved your issue.

Thanks for the build.

I have AX3600. I flashed via sysupgrade from some july build with configuration reset and my clients can't connect to wireless networks. DHCP didn't give them addresses. When I'm typing address on clients manually it works. How I can fix it?

cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdca:485c:9a4b::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        list dns '1.1.1.1'
        list dns '1.0.0.1'

config interface 'wan'
        option device 'wan'
        option proto 'pppoe'
        option username '********'
        option password '********'
        option ipv6 'auto'
        option type 'bridge'

config interface 'wan6'
        option device 'wan'
        option proto 'dhcpv6'

cat /etc/config/dhcp

config dnsmasq                      
        option domainneeded '1'     
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'        
        option local '/lan/'                                   
        option domain 'lan'                
        option expandhosts '1'                                 
        option cachesize '1000'                                
        option authoritative '1'                               
        option readethers '1'                                  
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option localservice '1'
        option ednspacket_max '1232'
                              
config dhcp 'lan'             
        option interface 'lan'        
        option start '100'          
        option limit '150'            
        option leasetime '12h'        
        option dhcpv4 'server'        
        option dhcpv6 'server'        
        option ra 'server'          
        list ra_flags 'managed-config'
        list ra_flags 'other-config'
                                            
config dhcp 'wan'                                    
        option interface 'wan'              
        option ignore '1'
                                    
config odhcpd 'odhcpd'                
        option maindhcp '0'         
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'                 

Hi @t0ssox

You need to do the steps indicated in point 1 of this link: https://github.com/AgustinLorenzo/openwrt/issues/5

Regards, Agustín

1 Like

Now it works. Thank you

Hi @nickmid

Apart from that we should consider going to the NHSS.QSDK.12.4 version

What is the final fix for including it and fixing it for everyone?

Regards, Agustín

Hi @t0ssox

Sorry, it's a strange bug between LUCI and UCI, if you check in LUCI the DHCP is "enabled", but if you check in UCI is disabled, but it only happens once.

If someone can try to debug it, that would be nice.

Regards, Agustín

1 Like

Hi @AgustinLorenzo , I was able to compile a version for CR1000A based on your code, everything works great and I got hardware offload tested fine, iperf was reaching 5Gigs from PC->router.(Should be able to reach 10G with pc->pc, but I don't have second 10G pc for testing). There is only one thing I noticed that when I have WAN connected, every time I start running speedtest, the router will reboot, but with WAN disconnected, it runs fine on all test. Any help would be appreciated.

Attached panic log when it reboots:


[    0.067422] TCP established hash table entries: 16384 (order: 5, 131072 bytes                                                                              , linear)
[    0.067583] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linea                                                                              r)
[    0.068114] TCP: Hash tables configured (established 16384 bind 16384)
[    0.068227] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.068291] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.068529] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.068565] PCI: CLS 0 bytes, default 64
[    0.069882] workingset: timestamp_bits=46 max_order=19 bucket_order=0
[    0.074700] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.074712] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORIT                                                                              Y) (c) 2001-2006 Red Hat, Inc.
[    0.079409] qcom-qmp-usb-phy 58000.phy: supply vdda-phy not found, using dumm                                                                              y regulator
[    0.079612] qcom-qmp-usb-phy 58000.phy: supply vdda-pll not found, using dumm                                                                              y regulator
[    0.080388] qcom-qmp-usb-phy 78000.phy: supply vdda-phy not found, using dumm                                                                              y regulator
[    0.080505] qcom-qmp-usb-phy 78000.phy: supply vdda-pll not found, using dumm                                                                              y regulator
[    0.081653] qcom-qusb2-phy 59000.phy: supply vdd not found, using dummy regul                                                                              ator
[    0.081771] qcom-qusb2-phy 59000.phy: supply vdda-pll not found, using dummy                                                                               regulator
[    0.081855] qcom-qusb2-phy 59000.phy: supply vdda-phy-dpdm not found, using d                                                                              ummy regulator
[    0.082039] qcom-qusb2-phy 59000.phy: Registered Qcom-QUSB2 phy
[    0.082228] qcom-qusb2-phy 79000.phy: supply vdd not found, using dummy regul                                                                              ator
[    0.082334] qcom-qusb2-phy 79000.phy: supply vdda-pll not found, using dummy                                                                               regulator
[    0.082374] qcom-qusb2-phy 79000.phy: supply vdda-phy-dpdm not found, using d                                                                              ummy regulator
[    0.082524] qcom-qusb2-phy 79000.phy: Registered Qcom-QUSB2 phy
[    0.085117] qcom-pcie 10000000.pci: host bridge /soc/pci@10000000 ranges:
[    0.085168] qcom-pcie 10000000.pci:       IO 0x0010200000..0x001020ffff -> 0x                                                                              0000000000
[    0.085194] qcom-pcie 10000000.pci:      MEM 0x0010220000..0x001fffffff -> 0x                                                                              0010220000
[    0.144431] qcom-pcie 10000000.pci: Read DBI address failed
[    0.255465] qcom-pcie 10000000.pci: iATU unroll: disabled
[    0.255474] qcom-pcie 10000000.pci: iATU regions: 32 ob, 8 ib, align 4K, limi                                                                              t 4G
[    1.255698] qcom-pcie 10000000.pci: Phy link never came up
[    1.255891] qcom-pcie 10000000.pci: PCI host bridge to bus 0001:00
[    1.255903] pci_bus 0001:00: root bus resource [bus 00-ff]
[    1.255915] pci_bus 0001:00: root bus resource [io  0x0000-0xffff]
[    1.255924] pci_bus 0001:00: root bus resource [mem 0x10220000-0x1fffffff]
[    1.255962] pci 0001:00:00.0: [17cb:0302] type 01 class 0x060400
[    1.255985] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    1.256059] pci 0001:00:00.0: PME# supported from D0 D3hot D3cold
[    1.259467] pci 0001:00:00.0: BAR 0: assigned [mem 0x10220000-0x10220fff]
[    1.259488] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    1.261662] pcieport 0001:00:00.0: PME: Signaling with IRQ 16
[    1.261944] pcieport 0001:00:00.0: AER: enabled with IRQ 16
[    1.262707] qcom-pcie 20000000.pci: host bridge /soc/pci@20000000 ranges:
[    1.262754] qcom-pcie 20000000.pci:       IO 0x0020200000..0x002020ffff -> 0x                                                                              0000000000
[    1.262779] qcom-pcie 20000000.pci:      MEM 0x0020220000..0x002fffffff -> 0x                                                                              0020220000
[    1.385438] qcom-pcie 20000000.pci: iATU unroll: enabled
[    1.385449] qcom-pcie 20000000.pci: iATU regions: 8 ob, 8 ib, align 4K, limit                                                                               1024G
[    1.485512] qcom-pcie 20000000.pci: PCIe Gen.3 x1 link up
[    1.485646] qcom-pcie 20000000.pci: PCI host bridge to bus 0000:00
[    1.485658] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.485671] pci_bus 0000:00: root bus resource [io  0x10000-0x1ffff] (bus add                                                                              ress [0x0000-0xffff])
[    1.485680] pci_bus 0000:00: root bus resource [mem 0x20220000-0x2fffffff]
[    1.485716] pci 0000:00:00.0: [17cb:1002] type 01 class 0x060400
[    1.485738] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    1.485806] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    1.489177] pci 0000:01:00.0: [17cb:1104] type 00 class 0x028000
[    1.489276] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    1.489790] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    1.489943] pci 0000:01:00.0: 7.876 Gb/s available PCIe bandwidth, limited by                                                                               8.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 15.752 Gb/s with 8.0 GT/s PCI                                                                              e x2 link)
[    1.490375] pci 0000:00:00.0: BAR 8: assigned [mem 0x20400000-0x205fffff]
[    1.490390] pci 0000:00:00.0: BAR 9: assigned [mem 0x20600000-0x207fffff 64bi                                                                              t pref]
[    1.490402] pci 0000:00:00.0: BAR 0: assigned [mem 0x20220000-0x20220fff]
[    1.490414] pci 0000:00:00.0: BAR 7: assigned [io  0x10000-0x10fff]
[    1.490427] pci 0000:01:00.0: BAR 0: assigned [mem 0x20400000-0x205fffff 64bi                                                                              t]
[    1.490479] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    1.490488] pci 0000:00:00.0:   bridge window [io  0x10000-0x10fff]
[    1.490498] pci 0000:00:00.0:   bridge window [mem 0x20400000-0x205fffff]
[    1.490508] pci 0000:00:00.0:   bridge window [mem 0x20600000-0x207fffff 64bi                                                                              t pref]
[    1.492023] pcieport 0000:00:00.0: PME: Signaling with IRQ 17
[    1.492298] pcieport 0000:00:00.0: AER: enabled with IRQ 17
[    1.494195] bam-dma-engine 704000.dma-controller: num-channels unspecified in                                                                               dt
[    1.494210] bam-dma-engine 704000.dma-controller: num-ees unspecified in dt
[    1.498708] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
[    1.499379] msm_serial 78b3000.serial: msm_serial: detected port #0
[    1.499416] msm_serial 78b3000.serial: uartclk = 3686400
[    1.499509] 78b3000.serial: ttyMSM0 at MMIO 0x78b3000 (irq = 22, base_baud =                                                                               230400) is a MSM
[    1.499537] msm_serial: console setup on port #0
[    2.516715] printk: console [ttyMSM0] enabled
[    2.521868] msm_serial: driver initialized
[    2.530614] loop: module loaded
[    2.531745] spi_qup 78b5000.spi: IN:block:16, fifo:64, OUT:block:16, fifo:64
[    2.533362] spi-nor spi0.0: unrecognized JEDEC id bytes: ff ff ff ff ff ff
[    2.540559] spi_qup 78b9000.spi: IN:block:16, fifo:64, OUT:block:16, fifo:64
[    2.547716] spmi spmi-0: PMIC arbiter version v2 (0x20010000)
[    2.572375] mdio_bus 90000.mdio-1: MDIO device at address 4 is missing.
[    2.581594] i2c_dev: i2c /dev entries driver
[    2.588762] sdhci: Secure Digital Host Controller Interface driver
[    2.588805] sdhci: Copyright(c) Pierre Ossman
[    2.593840] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.600358] remoteproc remoteproc0: releasing cd00000.q6v5_wcss
[    2.607073] NET: Registered PF_INET6 protocol family
[    2.610732] Segment Routing with IPv6
[    2.614939] In-situ OAM (IOAM) with IPv6
[    2.618483] NET: Registered PF_PACKET protocol family
[    2.622451] Bridge firewalling registered
[    2.627376] l2tp_core: L2TP core driver, V2.0
[    2.631063] mmc0: SDHCI controller on 7824900.mmc [7824900.mmc] using ADMA 64                                                                              -bit
[    2.631339] l2tp_netlink: L2TP netlink interface
[    2.631374] 8021q: 802.1Q VLAN Support v1.8
[    2.670020] qcom,cpr4-apss-regulator b018000.cpr4-ctrl: CPR valid fuse count:                                                                               4
[    2.670380] cpr4_ipq807x_apss_read_fuse_data: apc_corner: speed bin = 0
[    2.676173] cpr4_ipq807x_apss_read_fuse_data: apc_corner: CPR fusing revision                                                                               = 1
[    2.682744] cpr4_ipq807x_apss_read_fuse_data: apc_corner: CPR misc fuse value                                                                               = 0
[    2.690415] cpr4_ipq807x_apss_read_fuse_data: apc_corner: Voltage boost fuse                                                                               config = 0 boost = disable
[    2.697931] cpr3_mem_acc_init: apc: not using memory accelerator regulator
[    2.707063] cpr4_ipq807x_apss_calculate_open_loop_voltages: apc_corner: fused                                                                                    SVS: open-loop= 704000 uV
[    2.714006] cpr4_ipq807x_apss_calculate_open_loop_voltages: apc_corner: fused                                                                                    NOM: open-loop= 808000 uV
[    2.723993] cpr4_ipq807x_apss_calculate_open_loop_voltages: apc_corner: fused                                                                                  TURBO: open-loop= 864000 uV
[    2.733789] cpr4_ipq807x_apss_calculate_open_loop_voltages: apc_corner: fused                                                                                 STURBO: open-loop= 960000 uV
[    2.743659] cpr4_ipq807x_apss_calculate_target_quotients: apc_corner: fused                                                                                    SVS: quot[ 7]= 729, quot_offset[ 7]=   0
[    2.753417] cpr4_ipq807x_apss_calculate_target_quotients: apc_corner: fused                                                                                    NOM: quot[ 7]= 931, quot_offset[ 7]= 200
[    2.764268] cpr4_ipq807x_apss_calculate_target_quotients: apc_corner: fused                                                                                  TURBO: quot[ 7]=1027, quot_offset[ 7]=  95
[    2.775205] cpr4_ipq807x_apss_calculate_target_quotients: apc_corner: fused                                                                                 STURBO: quot[ 7]=1204, quot_offset[ 7]= 175
[    2.786332] cpr3_regulator_init_ctrl: apc: Default CPR mode = closed-loop
[    2.789356] cpufreq: cpufreq_online: CPU0: Running at unlisted initial freque                                                                              ▒[    2.815556] Waiting for root device /dev/mmcblk0p20...
[    2.836972] mmc0: new HS400 MMC card at address 0001
[    2.837441] mmcblk0: mmc0:0001 004GA0 3.69 GiB
[    2.841909] Alternate GPT is invalid, using primary GPT.
[    2.845332]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16                                                                               p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29
[    2.852339] mmcblk0boot0: mmc0:0001 004GA0 2.00 MiB
[    2.862848] mmcblk0boot1: mmc0:0001 004GA0 2.00 MiB
[    2.867530] mmcblk0rpmb: mmc0:0001 004GA0 512 KiB, chardev (247:0)
[    2.875852] VFS: Mounted root (squashfs filesystem) readonly on device 179:20                                                                              .
[    2.878595] Freeing unused kernel memory: 1280K
[    2.933981] Run /sbin/init as init process
[    3.011927] init: Console is alive
[    3.012045] init: - watchdog -
[    3.489200] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    3.530552] Key type encrypted registered
[    3.535710] hwmon hwmon0: temp1_input not attached to any thermal zone
[    3.541755] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-d                                                                              evel@redhat.com
[    3.656078] aquantia_phy_api_ops_init[2244]:INFO:qca probe aquantia phy drive                                                                              r succeeded!
[    3.662801] sfp_phy_init[412]:INFO:qca probe sfp phy driver succeeded!
[    5.323922] regi_init[3989]:INFO:Initializing HPPE Done!!
[    5.324029] regi_init[4049]:INFO:qca-ssdk module init succeeded!
[    5.330021] EDMA ver 1 hw init
[    5.334616] EDMA HW Reset completed succesfully
[    5.337268] Num rings - TxDesc:1 (23-23) TxCmpl:1 (7-7)
[    5.341666] RxDesc:1 (15-15) RxFill:1 (7-7)
[    5.347176] GMAC5(ffffff800311e900) Invalid MAC@ - using d2:45:61:a8:af:57
[    5.351952] GMAC6(ffffff8003fe5900) Invalid MAC@ - using 8a:d8:f9:c5:36:16
[    5.359086] Aquantia AQR113C 90000.mdio-1:08: FW 5.4, Build 15, Provisioning                                                                               1
[    5.373305] **********************************************************
[    5.373335] * NSS Data Plane driver
[    5.378732] **********************************************************
[    5.527573] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.527632] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus nu                                                                              mber 1
[    5.532080] xhci-hcd xhci-hcd.1.auto: hcc params 0x0220fe65 hci version 0x110                                                                               quirks 0x0000000002010010
[    5.539584] xhci-hcd xhci-hcd.1.auto: irq 43, io mem 0x08a00000
[    5.548989] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[    5.554792] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus nu                                                                              mber 2
[    5.560346] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0 SuperSpeed
[    5.568249] hub 1-0:1.0: USB hub found
[    5.574519] hub 1-0:1.0: 1 port detected
[    5.578438] usb usb2: We don't know the algorithms for LPM for this host, dis                                                                              abling LPM.
[    5.582557] hub 2-0:1.0: USB hub found
[    5.590398] hub 2-0:1.0: 1 port detected
[    5.594244] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
[    5.598021] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus nu                                                                              mber 3
[    5.603434] xhci-hcd xhci-hcd.2.auto: hcc params 0x0220fe65 hci version 0x110                                                                               quirks 0x0000000002010010
[    5.610948] xhci-hcd xhci-hcd.2.auto: irq 45, io mem 0x08c00000
[    5.620343] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
[    5.626146] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus nu                                                                              mber 4
[    5.631701] xhci-hcd xhci-hcd.2.auto: Host supports USB 3.0 SuperSpeed
[    5.639549] hub 3-0:1.0: USB hub found
[    5.645885] hub 3-0:1.0: 1 port detected
[    5.649786] usb usb4: We don't know the algorithms for LPM for this host, dis                                                                              abling LPM.
[    5.653913] hub 4-0:1.0: USB hub found
[    5.661744] hub 4-0:1.0: 1 port detected
[    5.669117] usbcore: registered new interface driver usb-storage
[    5.670092] usbcore: registered new interface driver uas
[    5.675623] kmodloader: done loading kernel modules from /etc/modules-boot.d/                                                                              *
[    5.681078] init: - preinit -
[   15.443851] random: crng init done
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
[   17.588268] Starting lvm2 during preinit
File descriptor 3 (/dev/watchdog) leaked on lvm invocation. Parent PID 658: /bin/sh
File descriptor 3 (/dev/watchdog) leaked on lvm invocation. Parent PID 658: /bin/sh
[   18.040421] mount_root: loading kmods from internal overlay
[   18.049156] kmodloader: loading kernel modules from //etc/modules-boot.d/*
[   18.051212] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
[   18.167069] block: attempting to load /etc/config/fstab
[   18.167881] block: unable to load configuration (fstab: Entry not found)
[   18.171112] block: no usable configuration
[   18.178445] loop0: detected capacity change from 0 to 307500
[   18.233859] loop0: detected capacity change from 307500 to 274732
[   18.289049] F2FS-fs (loop0): Mounted with checkpoint version = 6eb89bc7
[   18.289397] mount_root: loading kmods from internal overlay
[   18.303522] kmodloader: loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
[   18.303799] kmodloader: done loading kernel modules from /tmp/overlay/upper/etc/modules-boot.d/*
[   18.404885] block: attempting to load /etc/config/fstab
[   18.404946] block: unable to load configuration (fstab: Entry not found)
[   18.408928] block: no usable configuration
[   18.416432] mount_root: switching to f2fs overlay
[   18.421268] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
[   18.425683] urandom-seed: Seeding with /etc/urandom.seed
[   18.452120] procd: - early -
[   18.452197] procd: - watchdog -
[   18.980536] procd: - watchdog -
[   19.002946] procd: - ubus -
[   19.155842] procd: - init -
Please press Enter to activate this console.
[   19.338961] kmodloader: loading kernel modules from /etc/modules.d/*
[   19.388357] NET: Registered PF_ALG protocol family
[   19.450171] nss_driver - fw of size 827584  bytes copied to load addr: 40000000, nss_id : 0
[   19.451622] Supported Frequencies -
[   19.451632] 187.2 MHz
[   19.457394] 748.8 MHz
[   19.461145] 1.6896 GHz
[   19.463315]
[   19.468059] ffffffc000c1d600: set sdma ffffff8005dda200
[   19.469810] ffffffc000c1d600: meminfo init succeed
[   19.477580] ffffffc000c1d600: All resources initialized and nss core0 has been brought out of reset
[   19.491373] nss_driver - fw of size 295196  bytes copied to load addr: 40800000, nss_id : 1
[   19.499956] Supported Frequencies -
[   19.499966] 187.2 MHz
[   19.502290] urngd: v1.0.2 started.
[   19.507620] node size 2 # items 4
[   19.516937] memory: 40000000 2147483648 (avl 1999642624) items 4 active_cores 2
[   19.520478] NSS DDR size is 0x1000000
[   19.527446] ffffffc000c1d600: nss core 0 booted successfully
[   19.531365] 748.8 MHz
[   19.531367] 1.6896 GHz
[   19.536986]
[   19.541569] ffffffc000c24f40: set sdma ffffff8005c93600
[   19.543292] ffffffc000c24f40: meminfo init succeed
[   19.548425] ffffffc000c24f40: All resources initialized and nss core1 has been brought out of reset
[   19.550625] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[   19.552591] node size 2 # items 4
[   19.567898] memory: 40000000 2147483648 (avl 1999642624) items 4 active_cores 2
[   19.568052] gre: GRE over IPv4 demultiplexor driver
[   19.571518] NSS DDR size is 0x1000000
[   19.583356] ffffffc000c24f40: nss core 1 booted successfully
[   19.583503] ip_gre: GRE over IPv4 tunneling driver
[   19.594861] ip6_gre: GRE over IPv6 tunneling driver
[   19.601004] QCA multicast snooping installed successfully
[   19.605173] <NSS-CRYPTO>:module loaded Build_ID - 11/07/23, 01:33:43 SoC=ipq807x
[   19.610925] GACT probability on
[   19.615896] Mirror/redirect action on
[   19.621425] u32 classifier
[   19.621968]     input device check on
[   19.624589]     Actions configured
[   19.636082] Loading modules backported from Linux version v6.5-0-g2dde18cd1d8f
[   19.636122] Backport generated by backports.git v5.15.92-1-56-g5fb2ccb6b9e8
[   19.650519] NET: Registered PF_QIPCRTR protocol family
[   19.657848] xt_time: kernel timezone is -0000
[   19.676896] nf_conntrack_rtsp v0.7 loading
[   19.679915] nf_nat_rtsp v0.7 loading
[   19.694690] PPP generic driver version 2.4.2
[   19.695655] NET: Registered PF_PPPOX protocol family
[   19.699196] PPTP driver version 0.8.5
[   19.704681] wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information.
[   19.706544] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
[   19.727071] ath11k c000000.wifi: ipq8074 hw2.0
[   19.727103] ath11k c000000.wifi: FW memory mode: 0
[   19.730657] remoteproc remoteproc0: powering up cd00000.q6v5_wcss
[   19.739097] remoteproc remoteproc0: Booting fw image IPQ8074/q6_fw.mdt, size 668
[   20.466722] remoteproc remoteproc0: remote processor cd00000.q6v5_wcss is now up
[   20.469104] ath11k_pci 0000:01:00.0: BAR 0: assigned [mem 0x20400000-0x205fffff 64bit]
[   20.473317] ath11k_pci 0000:01:00.0: enabling device (0000 -> 0002)
[   20.481356] ath11k_pci 0000:01:00.0: MSI vectors: 16
[   20.487113] ath11k_pci 0000:01:00.0: qcn9074 hw1.0
[   20.492307] ath11k_pci 0000:01:00.0: FW memory mode: 2
[   20.538350] ath11k c000000.wifi: qmi ignore invalid mem req type 3
[   20.544796] ath11k c000000.wifi: chip_id 0x0 chip_family 0x0 board_id 0xff soc_id 0xffffffff
[   20.544824] ath11k c000000.wifi: fw_version 0x290c84a5 fw_build_timestamp 2023-03-25 07:34 fw_build_id WLAN.HK.2.9.0.1-01385-QCAHKSWPL_SILICONZ-1
[   20.654215] mhi mhi0: Requested to power ON
[   20.654273] mhi mhi0: Power on setup success
[   20.845385] ath11k c000000.wifi: htt event 48 not handled
[   21.043917] mhi mhi0: Wait for device to enter SBL or Mission mode
[   21.098062] kmodloader: done loading kernel modules from /etc/modules.d/*
[   21.110450] ath11k_pci 0000:01:00.0: chip_id 0x0 chip_family 0x0 board_id 0xff soc_id 0xffffffff
[   21.110497] ath11k_pci 0000:01:00.0: fw_version 0x290c8569 fw_build_timestamp 2023-03-25 06:50 fw_build_id
[   21.554459] ath11k_pci 0000:01:00.0: htt event 48 not handled
[   22.844944] ECM init
[   22.845011] ECM database jhash random seed: 0x6b30153f
[   22.984070] ECM init complete
[   23.717924] br-lan: port 1(lan) entered blocking state
[   23.717965] br-lan: port 1(lan) entered disabled state
[   23.722270] device lan entered promiscuous mode
[   24.111067] br-lan: port 2(phy1-ap0) entered blocking state
[   24.111107] br-lan: port 2(phy1-ap0) entered disabled state
[   24.115832] device phy1-ap0 entered promiscuous mode
[   24.139860] br-lan: port 2(phy1-ap0) entered blocking state
[   24.139913] br-lan: port 2(phy1-ap0) entered forwarding state
[   24.144595] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   24.315445] IPv6: ADDRCONF(NETDEV_CHANGE): phy1-ap0: link becomes ready
[   24.362509] br-lan: port 3(phy2-ap0) entered blocking state
[   24.362545] br-lan: port 3(phy2-ap0) entered disabled state
[   24.367165] device phy2-ap0 entered promiscuous mode
[   29.358169] nss-dp 3a007000.dp6-syn wan: PHY Link up speed: 10000
[   29.358241] IPv6: ADDRCONF(NETDEV_CHANGE): wan: link becomes ready
[   29.993909] nss-dp 3a003000.dp5-syn lan: PHY Link up speed: 10000
[   29.994025] br-lan: port 1(lan) entered blocking state
[   29.998990] br-lan: port 1(lan) entered forwarding state
[   33.113942] l11: disabling
[  100.063581] nss-dp 3a007000.dp6-syn wan: PHY Link is down
[  104.936176] Unable to handle kernel read from unreadable memory at virtual address 0000000000000028
[  104.936218] Mem abort info:
[  104.944035]   ESR = 0x0000000096000005
[  104.946801]   EC = 0x25: DABT (current EL), IL = 32 bits
[  104.950623]   SET = 0, FnV = 0
[  104.956093]   EA = 0, S1PTW = 0
[  104.958953]   FSC = 0x05: level 1 translation fault
[  104.961994] Data abort info:
[  104.966855]   ISV = 0, ISS = 0x00000005
[  104.969978]   CM = 0, WnR = 0
[  104.973540] user pgtable: 4k pages, 39-bit VAs, pgdp=0000000059f78000
[  104.976667] [0000000000000028] pgd=0800000059dd5003, p4d=0800000059dd5003, pud=0800000059dd5003, pmd=0000000000000000
[  104.983106] Internal error: Oops: 0000000096000005 [#1] SMP
[  104.993679] Modules linked in: ecm xt_connlimit pppoe ppp_async nft_fib_inet nf_flow_table_inet nf_conncount iptable_nat ath11k_pci ath11k_ahb ath11k xt_state xt_nat xt_helper xt_conntrack xt_connmark xt_connbytes xt_REDIRECT xt_MASQUERADE wireguard pptp pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat_rtsp nf_nat nf_flow_table nf_conntrack_rtsp nf_conntrack mac80211 libchacha20poly1305 iptable_mangle iptable_filter ipt_REJECT ip_tables chacha_neon cfg80211 xt_time xt_tcpudp xt_recent xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG x_tables tcp_bbr spidev slhc qrtr_smd qrtr_mhi qrtr qmi_helpers poly1305_neon nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 mhi mdio_netlink libcurve25519_generic libcrc32c libchacha crc_ccitt compat
[  104.993813]  sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact qca_nss_cfi_cryptoapi qca_nss_crypto qca_mcs bonding ip6_gre ip_gre gre ifb sit qca_nss_drv ip6_tunnel oid_registry tunnel6 tunnel4 vxlan crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_generic seqiv jitterentropy_rng drbg michael_mic hmac cmac uas usb_storage leds_fan5646 leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom qca_nss_dp qca_ssdk gpio_button_hotplug f2fs ext4 mbcache jbd2 dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax aquantia hwmon tpm rng_core crc32c_generic crc32_generic encrypted_keys trusted
[  105.124745] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.1.61 #0
[  105.146979] Hardware name: Verizon CR1000A (DT)
[  105.152618] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  105.157137] pc : nss_crypto_hdr_free+0x5c/0x144 [qca_nss_crypto]
[  105.164080] lr : nss_cryptoapi_skcipher_done+0x40/0xa4 [qca_nss_cfi_cryptoapi]
[  105.170332] sp : ffffffc008c83d20
[  105.177355] x29: ffffffc008c83d20 x28: ffffffc000c25000 x27: ffffff8005b3c100
[  105.180748] x26: ffffffc000c24f40 x25: 000000000000007f x24: ffffff8007a6107c
[  105.187866] x23: ffffff8007a61040 x22: 0000000000000000 x21: ffffffc000d2ccc0
[  105.194984] x20: ffffff80079f3400 x19: 0000000000000000 x18: 0000000000360000
[  105.202103] x17: ffffffc077162000 x16: ffffffc008c80000 x15: 0000000030f97cda
[  105.209220] x14: 0000000000000377 x13: 0000000000000001 x12: 0000000000000000
[  105.216339] x11: 0000000000000040 x10: ffffffc008bd74e8 x9 : ffffffc008bd74e0
[  105.223457] x8 : ffffff807fbdb028 x7 : 0000000000000000 x6 : 9582b054b48279b0
[  105.230575] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000001
[  105.237693] x2 : 0000000000000000 x1 : ffffff8002098000 x0 : 0000000000000028
[  105.244812] Call trace:
[  105.251921]  nss_crypto_hdr_free+0x5c/0x144 [qca_nss_crypto]
[  105.254183]  nss_cryptoapi_skcipher_done+0x40/0xa4 [qca_nss_cfi_cryptoapi]
[  105.260089]  nss_crypto_transform_done+0x9c/0x170 [qca_nss_crypto]
[  105.266771]  nss_core_send_buffer+0x1964/0x1ff0 [qca_nss_drv]
[  105.272933]  nss_core_handle_napi_queue+0x30/0x80 [qca_nss_drv]
[  105.278749]  __napi_poll+0x38/0x190
[  105.284474]  net_rx_action+0x160/0x320
[  105.287947]  _stext+0x10c/0x290
[  105.291765]  ____do_softirq+0x10/0x20
[  105.294805]  call_on_irq_stack+0x24/0x50
[  105.298624]  do_softirq_own_stack+0x1c/0x30
[  105.302617]  __irq_exit_rcu+0xc4/0xf0
[  105.306523]  irq_exit_rcu+0x10/0x20
[  105.310341]  el1_interrupt+0x38/0x5c
[  105.313640]  el1h_64_irq_handler+0x18/0x2c
[  105.317461]  el1h_64_irq+0x68/0x6c
[  105.321365]  arch_cpu_idle+0x18/0x2c
[  105.324752]  default_idle_call+0x30/0x74
[  105.328486]  do_idle+0x228/0x250
[  105.332390]  cpu_startup_entry+0x38/0x40
[  105.335603]  arch_show_interrupts+0x0/0x16c
[  105.339510]  __secondary_switched+0xb0/0xb4
[  105.343421] Code: d2800002 9100a260 52800023 f9800011 (885ffc01)
[  105.347589] ---[ end trace 0000000000000000 ]---
[  105.353835] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[  105.358527] SMP: stopping secondary CPUs
[  105.365208] Kernel Offset: disabled
[  105.369283] CPU features: 0x00000,00000000,0000400b
[  105.372498] Memory Limit: none
[  106.377363] Rebooting in 3 seconds..

Hi. I managed to build a basic nss version for my dumb AP. I can tell there's a difference in performance, but I believe we need the wifi offload too. Anyone been working on wifi offload btw? Care to share some progress? :slight_smile:

To be honest, there was not too many changes to make it work from all the job @robimarko did. And thanks to all the others that did all the work for nss-packages patches etc.

Anyway, I noticed something in the file

we have changed this, but what caught my attention is the profile we are using.
I think the medium profile is for 512mb version?

DNSS_MEM_PROFILE_MEDIUM
#elif defined (NSS_MEM_PROFILE_MEDIUM)
#define NSS_DEFAULT_NUM_CONN			    2048		/* Default number of connections for IPv4 and IPv6 each, for medium memory profile */
#define NSS_MAX_TOTAL_NUM_CONN_IPV4_IPV6	4096		/* MAX Connection shared between IPv4 and IPv6 for medium memory profile */

My AP's have 1GB of ram, so we could go to the default profile that is

#define NSS_DEFAULT_NUM_CONN			    4096		/* Default number of connections for each IPv4 and IPv6 */
#define NSS_MAX_TOTAL_NUM_CONN_IPV4_IPV6	8192		/* MAX Connection shared between IPv4 and IPv6 */

maybe that could benefit some devices.

My last question is, how can one verify what profile is used, like is there any file that I can view the default number of connections NSS use?

For those who interested in the basic version, this is my changes for openwrt.

And this is my basic-nss-packages I'm using for my dumb AP build.

Again, thanks to all for all the work. :+1:

Kernel 6.1.61
OpenWrt SNAPSHOT, r24315-ae500e62e2

Great work, do you mind sharing your build for xiaomi AX3600?

BTW, do you experience this issue IPQ807X NSS Build - #816 by jklasjk?

I'm running wlan.hk.2.7 because of those issues, and I've read some users with ax3600 does not work with this wlan.hk version.

But from testing with wlan.hk.2.7 + latest updates I only get one high ping. But I don't get that Destination Host Unreachable. With wlan.hk.2.7 my lan printer works too compared to latest version.

You can read more about my feedback here https://github.com/openwrt/openwrt/issues/9555#issuecomment-1792872687

But my focus is trying to figure out why my iperf3 --bidir performance is so bad. I'm trying different wlan.hk firmware, arrange IRQ and change settings. But nope, it just don't work on my AP's.
So that is why I'm now testing NSS and see if (when available) wlan offload fixes my issues.

Ah, forgot to ask last time. Anyone else have this "layout" for your nss queues?
What is that second queue for?

root@wax620:~# cat /proc/interrupts | grep nss
 34:    1152621          0          0          0 GIC-0 409 Edge      nss_empty_buf_sos
 35:     269591          0          0          0 GIC-0 410 Edge      nss_empty_buf_queue
 36:          0          0          0          0 GIC-0 411 Edge      nss-tx-unblock
 37:   62777189          0          0          0 GIC-0 412 Edge      nss_queue0
 38:          0     329903          0          0 GIC-0 413 Edge      nss_queue1
 39:          0          0    1111892          0 GIC-0 414 Edge      nss_queue2
 40:          0          0          0     212157 GIC-0 415 Edge      nss_queue3
 41:          0          0          0          0 GIC-0 416 Edge      nss_coredump_complete
 42:          0          0          0          0 GIC-0 417 Edge      nss_paged_empty_buf_sos
 43:          0          0          0          0 GIC-0 418 Edge      nss_profile_dma
 44:     586059          0          0          0 GIC-0 422 Edge      nss_empty_buf_sos
 45:          0          0          0          0 GIC-0 423 Edge      nss_empty_buf_queue
 46:          0          0          0          0 GIC-0 424 Edge      nss-tx-unblock
 47:   61085278          0          0          0 GIC-0 425 Edge      nss_queue0
 48:          0          0          0          0 GIC-0 426 Edge      nss_queue1
 49:          0          0          0          0 GIC-0 427 Edge      nss_queue2
 50:          0          0          0          0 GIC-0 428 Edge      nss_queue3
 51:          0          0          0          0 GIC-0 429 Edge      nss_coredump_complete
 53:          0          0          0          0 GIC-0 430 Edge      nss_paged_empty_buf_sos

Thanks for sharing. Do you mind to share your .config? I would like to build a AP only firmware for my xiaomi AX3600 as well. Or is there any easy way to downgrade to wlan.hk.2.7 without building Openwrt?