Are you able to grab kernel panic logs?
Sorry im a little beginner. May I know how to get it
Just curious, on your non-NSS build are able to get stats to show up for you wlan interfaces? I have a patch, but wondering if it's the same on vanilla builds.
phy0-ap0 Link encap:Ethernet HWaddr
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:841 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
phy1-ap0 Link encap:Ethernet HWaddr
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
1 Like
Check and see if you have any ramoops records in pstore.
ls /sys/fs/pstore/
Then cat to see what it captured.
cat /sys/fs/pstore/*
Good morning @qosmio,
I am going to prepare a new build, do you need me to prove something special?
Regards, Agustin
1 Like
Y, this is empty. There are no phy stats collected. But AFAIR it was never there in the past at any time resp. it was discussed in the past already (I think there is sth. about within the AX3600 thread). It should be shown within cat /proc/net/dev. But there are only LAN/WAN ports filled. I was always wondering how LuCI is getting the realtime stats. But I've observed that LuCI is loosing this ability if the nss firmware changes e. g. for 11.4 LuCI is showing realtime stats while for 12.5 it is not showing anything. I didn't dig further.
1 Like
I've make those iperf3 tests after almost every update of my 301w firmware.
My setup is 301w (in router mode running iperf3 server) and Moto Fusion Smartphone (160MHz 5GHz WLAN client).
I get these results.
Server listening on 5201 (test #3)
-----------------------------------------------------------
Accepted connection from 192.168.1.6, port 44110
[ 5] local 192.168.1.1 port 5201 connected to 192.168.1.6 port 44118
[ 8] local 192.168.1.1 port 5201 connected to 192.168.1.6 port 44122
[ 10] local 192.168.1.1 port 5201 connected to 192.168.1.6 port 44124
[ 12] local 192.168.1.1 port 5201 connected to 192.168.1.6 port 44138
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 37.6 MBytes 315 Mbits/sec 0 1.43 MBytes
[ 8] 0.00-1.00 sec 11.6 MBytes 97.4 Mbits/sec 0 283 KBytes
[ 10] 0.00-1.00 sec 18.9 MBytes 158 Mbits/sec 0 520 KBytes
[ 12] 0.00-1.00 sec 21.2 MBytes 178 Mbits/sec 0 553 KBytes
[SUM] 0.00-1.00 sec 89.4 MBytes 749 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 37.5 MBytes 314 Mbits/sec 0 1.43 MBytes
[ 8] 1.00-2.00 sec 26.6 MBytes 223 Mbits/sec 0 635 KBytes
[ 10] 1.00-2.00 sec 29.8 MBytes 249 Mbits/sec 0 720 KBytes
[ 12] 1.00-2.00 sec 31.2 MBytes 261 Mbits/sec 0 706 KBytes
[SUM] 1.00-2.00 sec 125 MBytes 1.05 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 26.00-27.00 sec 35.0 MBytes 294 Mbits/sec 0 2.29 MBytes
[ 8] 26.00-27.00 sec 33.4 MBytes 280 Mbits/sec 0 2.10 MBytes
[ 10] 26.00-27.00 sec 40.6 MBytes 341 Mbits/sec 0 2.44 MBytes
[ 12] 26.00-27.00 sec 37.6 MBytes 316 Mbits/sec 0 2.36 MBytes
[SUM] 26.00-27.00 sec 147 MBytes 1.23 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 27.00-28.00 sec 34.8 MBytes 292 Mbits/sec 0 2.29 MBytes
[ 8] 27.00-28.00 sec 32.4 MBytes 272 Mbits/sec 0 2.10 MBytes
[ 10] 27.00-28.00 sec 37.0 MBytes 310 Mbits/sec 0 2.44 MBytes
[ 12] 27.00-28.00 sec 36.6 MBytes 307 Mbits/sec 0 2.36 MBytes
[SUM] 27.00-28.00 sec 141 MBytes 1.18 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 28.00-29.00 sec 36.6 MBytes 307 Mbits/sec 0 2.29 MBytes
[ 8] 28.00-29.00 sec 36.9 MBytes 309 Mbits/sec 0 2.10 MBytes
[ 10] 28.00-29.00 sec 37.6 MBytes 315 Mbits/sec 0 2.44 MBytes
[ 12] 28.00-29.00 sec 38.2 MBytes 321 Mbits/sec 0 2.36 MBytes
[SUM] 28.00-29.00 sec 149 MBytes 1.25 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 29.00-30.00 sec 36.4 MBytes 305 Mbits/sec 0 2.29 MBytes
[ 8] 29.00-30.00 sec 33.8 MBytes 283 Mbits/sec 0 2.10 MBytes
[ 10] 29.00-30.00 sec 37.4 MBytes 314 Mbits/sec 0 2.44 MBytes
[ 12] 29.00-30.00 sec 39.9 MBytes 335 Mbits/sec 0 2.36 MBytes
[SUM] 29.00-30.00 sec 147 MBytes 1.24 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.01 sec 1.06 GBytes 304 Mbits/sec 0 sender
[ 8] 0.00-30.01 sec 953 MBytes 266 Mbits/sec 0 sender
[ 10] 0.00-30.01 sec 930 MBytes 260 Mbits/sec 0 sender
[ 12] 0.00-30.01 sec 956 MBytes 267 Mbits/sec 0 sender
[SUM] 0.00-30.01 sec 3.84 GBytes 1.10 Gbits/sec 0 sender
I get the same results in both directions - sending from server (-R) and from client. Currently I'm on NSS 12.2 + ath11k 2.9.
I've never been able to get past 1.3-1.4 Gbits/sec with the Moto but it's the only 160MHz capable device I have in my hands.
Do you use latest iperf3 on Windows. I remember that there was a bug in the older Windows iperf3 versions that gave wrong (low) results in some cases.
Currently I use this build.
PS E:\iperf3-win64> .\iperf3 -v
iperf 3.17.1 (cJSON 1.7.15)
CYGWIN_NT-10.0-22631 s-100915 3.5.3-1.x86_64 2024-04-03 17:25 UTC x86_64
Optional features available: CPU affinity setting, support IPv4 don't fragment, POSIX threads
Later I will try iprf3 from my Win 11 Laptop (2.5G cable)<-->(10G LAN port) 301w (WLAN 5G)<-->Moto (5G 160MHz) and I will update the post.
Now I have another job caused by @qosmio's latest commits 
and the kid has even more important job with the Laptop 
.
@asvio @anom3 @rmandrad
Probably you'll (and all having nbg7815 or 301w) be interested in this important PR.
1 Like
Hey All,
Currently have 5 active branches, the only key difference being backports version, ath11k firmware, and the still pending PR for MX4300/LN301.
Devices supported upstream
| branch | backports | ath11k |
|---|---|---|
| qualcommax-6.x-nss-wifi | 6.9.9 | 2.9 |
| qualcommax-6.x-nss-wifi-6.11 | 6.11 | 2.9 |
| qualcommax-6.x-nss-ath-2.12 | 6.9.9 | 2.12 |
Pending Devices (MX4300/LN301)
| branch | backports | ath11k |
|---|---|---|
| qualcommax-6.x-nss-mx4300 | 6.9.9 | 2.12 |
| qualcommax-6.x-nss-mx4300-6.11 | 6.11 | 2.12 |
I pushed quite a few updates earlier, but some of the highlight commits to look for.
| date | commit | desc |
|---|---|---|
| 2024-09-30 | f8e3b55066 | kernel: Fix UDP hanging in unbound |
| 2024-09-30 | 0e215ddc6a | ath11k_nss: Fix iface stats showing up as 0 |
| 2024-09-29 | 857fe848f6 | ath11k_nss: fix WDS instability by disabling NAWDS mode |
| 2024-09-29 | 1261cf826c | ath11k_nss: Fix kernel panic on free_netdev during reboot |
| 2024-09-27 | e1cac247f3 | ath11k_nss: fix peer ast not found warnings |
I know the number of branches looks a bit crazy, but wanted to ensure I separate them out to make rebasing with upstream easier. Eventually everything will be merged back in to qualcommax-6.x-nss-wifi once I can confirm 6.11 is stable for everyone. For know keeping qualcommax-6.x-nss-wifi on 6.9.9.
Long story short:
Build off branch: mx4300-6.11
3 Likes
My latest commits should now show proper stats in ifconfig and /proc/net/dev.
Running backports 6.11 + NSS 11.4 + ATH11K 2.12 on (dl-wrx36, mx5300, and mx4300)
4 Likes
Hey,
Can't comment for everyone, but I've been shuffling different builds (with different nss and ath11k fw versions for testing) based on @asvio repos in a last few days and weeks frequently - config was always preserved and haven't encountered any issues. My build is a bit different than asvio's default - I've replaced dnsmasq with full odhcpd, installed unbound with TLS and added adblocknto it, as well as changed wpad to openssl-full and added some random packages to taste. No issues with config preservation when reflashing back and forth.
Oh, and I'm on nbg7815.
1 Like
Maybe it's better to comment on the issue on the PR page just to not spam here but as this is really important for our devices I put a notice here.
Reading the PR I understand that the issue itself isn't that the settings are not preserved during sysupgrade rather than they are written in the wrong place of the emmc.
1 Like
root@OpenWrt:~# ls /sys/fs/pstore/
dmesg-ramoops-0
root@OpenWrt:~# cat /sys/fs/pstore/*
Panic#1 Part1
root@OpenWrt:~# cat /sys/fs/pstore/*
Panic#1 Part1
<5>[ 2.013814] ubi0: background thread "ubi_bgt0d" started, PID 558
<6>[ 2.023678] block ubiblock0_1: created from ubi0:1(rootfs)
<6>[ 2.029292] clk: Disabling unused clocks
<6>[ 2.038191] VFS: Mounted root (squashfs filesystem) readonly on device 254 :0.
<6>[ 2.038653] Freeing unused kernel memory: 896K
<6>[ 2.044416] Run /sbin/init as init process
<7>[ 2.048662] with arguments:
<7>[ 2.048665] /sbin/init
<7>[ 2.048668] with environment:
<7>[ 2.048670] HOME=/
<7>[ 2.048673] TERM=linux
<14>[ 2.262769] init: Console is alive
<14>[ 2.262911] init: - watchdog -
<5>[ 2.995447] random: crng init done
<14>[ 3.303565] kmodloader: loading kernel modules from /etc/modules-boot.d/*
<4>[ 3.338239] gpio_button_hotplug: loading out-of-tree module taints kernel.
<6>[ 3.343954] pstore: Using crash dump compression: deflate
<6>[ 3.344016] pstore: Registered ramoops as persistent store backend
<6>[ 3.349584] ramoops: using 0x100000@0x4e900000, ecc: 0
<4>[ 3.399464] ssdk_dt_parse_interrupt[941]:INFO:intr-gpio does not exist
<4>[ 3.821573] regi_init[2525]:INFO:Initializing HPPE Done!!
<4>[ 3.821698] regi_init[2574]:INFO:qca-ssdk module init succeeded!
<6>[ 3.828693] EDMA ver 1 hw init
<6>[ 3.832263] EDMA HW Reset completed succesfully
<6>[ 3.834933] Num rings - TxDesc:1 (23-23) TxCmpl:1 (7-7)
<6>[ 3.839373] RxDesc:1 (15-15) RxFill:1 (7-7)
<6>[ 3.845000] dp2: ppe offload disabled: 0 for macid 2
<6>[ 3.848718] dp2: Switch attached to macid 2 status: 0
<6>[ 4.046240] Qualcomm QCA8075 90000.mdio-1:01: attached PHY driver (mii_bus :phy_addr=90000.mdio-1:01, irq=POLL)
<6>[ 4.047287] dp3: ppe offload disabled: 0 for macid 3
<6>[ 4.055130] dp3: Switch attached to macid 3 status: 0
<6>[ 4.145800] Qualcomm QCA8075 90000.mdio-1:02: attached PHY driver (mii_bus :phy_addr=90000.mdio-1:02, irq=POLL)
<6>[ 4.146893] dp4: ppe offload disabled: 0 for macid 4
<6>[ 4.154694] dp4: Switch attached to macid 4 status: 0
<6>[ 4.235796] Qualcomm QCA8075 90000.mdio-1:03: attached PHY driver (mii_bus :phy_addr=90000.mdio-1:03, irq=POLL)
<6>[ 4.236833] dp5: ppe offload disabled: 0 for macid 5
<6>[ 4.244687] dp5: Switch attached to macid 5 status: 0
<6>[ 4.325786] Qualcomm QCA8075 90000.mdio-1:04: attached PHY driver (mii_bus :phy_addr=90000.mdio-1:04, irq=POLL)
<6>[ 4.326793] **********************************************************
<6>[ 4.334679] * NSS Data Plane driver
<6>[ 4.341208] **********************************************************
<14>[ 4.347167] kmodloader: done loading kernel modules from /etc/modules-boo t.d/*
<14>[ 4.361270] init: - preinit -
<5>[ 7.341163] UBIFS (ubi0:2): default file-system created
<5>[ 7.341805] UBIFS (ubi0:2): Mounting in unauthenticated mode
<5>[ 7.345322] UBIFS (ubi0:2): background thread "ubifs_bgt0_2" started, PID 728
<5>[ 7.406451] UBIFS (ubi0:2): UBIFS: mounted UBI device 0, volume 2, name "r ootfs_data"
<5>[ 7.406487] UBIFS (ubi0:2): LEB size: 126976 bytes (124 KiB), min./max. I/ O unit sizes: 2048 bytes/2048 bytes
<5>[ 7.413267] UBIFS (ubi0:2): FS size: 24887296 bytes (23 MiB, 196 LEBs), ma x 206 LEBs, journal size 1269760 bytes (1 MiB, 10 LEBs)
<5>[ 7.423185] UBIFS (ubi0:2): reserved for root: 1175487 bytes (1147 KiB)
<5>[ 7.434798] UBIFS (ubi0:2): media format: w5/r0 (latest is w5/r0), UUID D0 31711E-5FC7-4E13-AECA-8C7613491D7F, small LPT model
<14>[ 7.443001] mount_root: overlay filesystem has not been fully initialized yet
<14>[ 7.453015] mount_root: switching to ubifs overlay
<4>[ 7.462498] overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
<12>[ 7.468969] urandom-seed: Seed file not found (/etc/urandom.seed)
<14>[ 7.545789] procd: - early -
<14>[ 7.545906] procd: - watchdog -
<14>[ 8.126709] procd: - watchdog -
<14>[ 8.154817] procd: - ubus -
<14>[ 8.315789] procd: - init -
<14>[ 8.880605] kmodloader: loading kernel modules from /etc/modules.d/*
<14>[ 9.025363] urngd: v1.0.2 started.
<6>[ 9.032802] NET: Registered PF_ALG protocol family
<6>[ 9.042669] tun: Universal TUN/TAP device driver, 1.6
<6>[ 9.053619] ipip: IPv4 and MPLS over IPv4 tunneling driver
<13>[ 9.303871] hotplug: symlinking qca-nss0.bin to /lib/firmware/qca-nss0-re tail.bin
<6>[ 9.432347] qca-nss 39000000.nss: NSS FW Version: NSS.HK.11.4.0.5-6-R
<6>[ 9.432400] qca-nss 39000000.nss: fw of size 835960 bytes copied to addr: 40000000, nss_id: 0
<1>[ 9.439801]
<6>[ 9.485441] qca-nss 39000000.nss: NSS core 0 booted successfully
<1>[ 9.491776] NSS core 0 signal COREDUMP COMPLETE 4000
<1>[ 9.491803] ffffffc07907f580: Starting NSS-FW logbuffer dump for core 0
<1>[ 9.495809] ffffffc07907f580: Warn: trap[813]: Trap on CHIP ID 00050002
<1>[ 9.502137] ffffffc07907f580: Warn: trap[620]: Trapped: TRAP_TD(00000200) DCAPT(3C000080)
<1>[ 9.508741] ffffffc07907f580: Warn: trap[645]: Trapped: Thread: 9, reason: 00001000, PC: 3F0039DC, previous PC: 3F0039D8
<1>[ 9.517082] ffffffc07907f580: Warn: trap[594]: A0_3: 3F012850 3F02DC4C 3F0 127A8 00000000
<1>[ 9.528013] ffffffc07907f580: Warn: trap[594]: A4_7: 3F02DB9C 3F0127E0 3F0 2DA4C 3F00D718
<1>[ 9.536085] ffffffc07907f580: Warn: trap[599]: D0_3: 00000000 00000020 000 00024 00003FFF
<1>[ 9.544153] ffffffc07907f580: Warn: trap[599]: D4_7: 40000000 00000000 000 01000 00000000
<1>[ 9.552231] ffffffc07907f580: Warn: trap[599]: D8_11: 00000000 00000000 00 000000 00000000
<1>[ 9.560304] ffffffc07907f580: Warn: trap[599]: D12_15: 00000000 00000000 0 0000000 00000000
<1>[ 9.568378] ffffffc07907f580: Warn: trap[649]: Thread_9 has non-recoverabl e trap
<0>[ 9.576538] Kernel panic - not syncing: NSS FW coredump: bringing system d own
<4>[ 9.584084] CPU: 0 PID: 1052 Comm: napi/nss-13 Tainted: G O 6.6.52 #0
<4>[ 9.591115] Hardware name: Redmi AX6 (DT)
<4>[ 9.598920] Call trace:
<4>[ 9.602910] dump_backtrace+0xa0/0xd8
<4>[ 9.605168] show_stack+0x18/0x24
<4>[ 9.608988] dump_stack_lvl+0x48/0x60
<4>[ 9.612286] dump_stack+0x18/0x24
<4>[ 9.615932] panic+0x2c8/0x314
<4>[ 9.619229] nss_fw_coredump_notify+0x1e4/0x3dc [qca_nss_drv]
<4>[ 9.622187] nss_core_handle_napi_emergency+0x38/0x4c [qca_nss_drv]
<4>[ 9.628002] __napi_poll+0x38/0x178
<4>[ 9.634075] napi_threaded_poll_loop+0x80/0x1bc
<4>[ 9.637548] napi_threaded_poll+0x78/0x84
<4>[ 9.642061] kthread+0x10c/0x110
<4>[ 9.646226] ret_from_fork+0x10/0x20
<2>[ 9.649527] SMP: stopping secondary CPUs
<0>[ 9.653091] Kernel Offset: disabled
<0>[ 9.656991] CPU features: 0x0,00000000,00000000,0000400b
<0>[ 9.660208] Memory Limit: none
root@OpenWrt:~#
I don't remember having any problems updating nbg7815 while keeping the setting.
Anyway, I implemented it on one of my branches without nss support to check if there was any difference. I didn't really notice the difference.
I will wait for the PR to be approved or in case I notice any problems with the sysupgrade.
1 Like
@asvio
I haven't noticed any issues either and wrote a comment on the PR.
@qosmio
I am testing a build with backports 6.11 + NSS 12.2 + ATH11K 2.9 based on your qualcommax-6.x-nss-wifi-6.11 branch.
You are not environmentally friendly with all those branches . I have to compile twice now or maybe I just don't know how to use what you recommended. And I expect sometime in the future two MX4300/LN1301 from USA.
I have two dmesg-ramoops files after reboot.
Oops#1 Part1
<6>[ 4.906710] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
<6>[ 4.910444] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus number 3
<6>[ 4.915865] xhci-hcd xhci-hcd.2.auto: hcc params 0x0220fe65 hci version 0x110 quirks 0x0000008002000010
<6>[ 4.923379] xhci-hcd xhci-hcd.2.auto: irq 43, io mem 0x08c00000
<6>[ 4.932762] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
<6>[ 4.938549] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus number 4
<6>[ 4.944113] xhci-hcd xhci-hcd.2.auto: Host supports USB 3.0 SuperSpeed
<6>[ 4.951957] hub 3-0:1.0: USB hub found
<6>[ 4.958259] hub 3-0:1.0: 1 port detected
<6>[ 4.962206] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
<6>[ 4.966300] hub 4-0:1.0: USB hub found
<6>[ 4.974153] hub 4-0:1.0: 1 port detected
<6>[ 4.982102] usbcore: registered new interface driver usb-storage
<6>[ 4.983122] usbcore: registered new interface driver uas
<14>[ 4.988021] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<14>[ 5.003471] init: - preinit -
<6>[ 6.008907] usb 2-1: new SuperSpeed USB device number 2 using xhci-hcd
<6>[ 6.043513] scsi host0: uas
<5>[ 7.113824] scsi 0:0:0:0: Direct-Access SanDisk Extreme 55AE 3008 PQ: 0 ANSI: 6
<5>[ 7.116766] scsi 0:0:0:1: Enclosure SanDisk SES Device 3008 PQ: 0 ANSI: 6
<5>[ 7.117139] sd 0:0:0:0: [sda] 3906963617 512-byte logical blocks: (2.00 TB/1.82 TiB)
<5>[ 7.129403] sd 0:0:0:0: [sda] Write Protect is off
<7>[ 7.136781] sd 0:0:0:0: [sda] Mode Sense: 37 00 10 00
<5>[ 7.137001] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
<6>[ 7.259033] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
<6>[ 7.259055] sd 0:0:0:0: [sda] Optimal transfer size 1048576 bytes
<6>[ 7.268192] sda: sda1
<5>[ 7.270735] sd 0:0:0:0: [sda] Attached SCSI disk
<5>[ 13.128869] random: crng init done
<14>[ 15.602134] mount_root: loading kmods from internal overlay
<14>[ 15.614204] kmodloader: loading kernel modules from //etc/modules-boot.d/*
<14>[ 15.615516] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
<14>[ 16.036894] block: attempting to load /tmp/overlay/upper/etc/config/fstab
<11>[ 16.037033] block: unable to load configuration (fstab: Entry not found)
<14>[ 16.042801] block: attempting to load /tmp/overlay/etc/config/fstab
<11>[ 16.049532] block: unable to load configuration (fstab: Entry not found)
<14>[ 16.055490] block: attempting to load /etc/config/fstab
<11>[ 16.065707] block: unable to load configuration (fstab: Entry not found)
<11>[ 16.067389] block: no usable configuration
<14>[ 16.074592] block: attempting to load /etc/config/fstab
<11>[ 16.078248] block: unable to load configuration (fstab: Entry not found)
<11>[ 16.083408] block: no usable configuration
<6>[ 16.091221] loop0: detected capacity change from 0 to 1048576
<6>[ 16.128883] loop0: detected capacity change from 1048576 to 1009920
<6>[ 16.305783] EXT4-fs (loop0): recovery complete
<6>[ 16.305946] EXT4-fs (loop0): mounted filesystem cexxxxa-40xx-5xx0-xxx-58xxxxxcaf r/w with ordered data mode. Quota mode: disabled.
<6>[ 16.489066] nss-dp 3a001600.dp4 lan1: PHY Link up speed: 1000
<1>[ 16.517642] Unable to handle kernel read from unreadable memory at virtual address 0000000000000090
<1>[ 16.517689] Mem abort info:
<1>[ 16.525507] ESR = 0x0000000096000005
<1>[ 16.528273] EC = 0x25: DABT (current EL), IL = 32 bits
<1>[ 16.532118] SET = 0, FnV = 0
<1>[ 16.537560] EA = 0, S1PTW = 0
<1>[ 16.540433] FSC = 0x05: level 1 translation fault
<1>[ 16.543466] Data abort info:
<1>[ 16.548323] ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000
<1>[ 16.551463] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
<1>[ 16.556756] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
<1>[ 16.561879] user pgtable: 4k pages, 39-bit VAs, pgdp=0000000042fe4000
<1>[ 16.567254] [0000000000000090] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000
<0>[ 16.573605] Internal error: Oops: 0000000096000005 [#1] PREEMPT SMP
<4>[ 16.582098] Modules linked in: uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom qca_nss_dp(O) qca_ssdk(O) ramoops reed_solomon pstore gpio_button_hotplug(O) vfat fat f2fs ext4 mbcache jbd2 aquantia hwmon crc_ccitt crc32c_generic crc32_generic
<4>[ 16.589723] CPU: 0 PID: 954 Comm: block Tainted: G O 6.6.52 #0
<4>[ 16.611959] Hardware name: QNAP 301w (DT)
<4>[ 16.619159] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<4>[ 16.623244] pc : edma_napi+0x2c0/0x4c4 [qca_nss_dp]
<4>[ 16.630011] lr : edma_napi+0x290/0x4c4 [qca_nss_dp]
<4>[ 16.634871] sp : ffffffc080003e00
<4>[ 16.639729] x29: ffffffc080003e00 x28: 0000000000000040 x27: ffffff800287c000
<4>[ 16.643211] x26: ffffffc080003f50 x25: ffffff803fd9e480 x24: ffffffc080b7d1c0
<4>[ 16.650327] x23: ffffffc080003f40 x22: ffffffc078e27ee0 x21: 0000000000000001
<4>[ 16.657446] x20: ffffff8003088900 x19: 0000000000000001 x18: 0000000000000014
<4>[ 16.664564] x17: 000000005c83cc57 x16: 00000000ccaae286 x15: 00000000027925ee
<4>[ 16.671683] x14: 00000000198e17db x13: 00000000310db0ac x12: 0000000067d52c77
<4>[ 16.678801] x11: 00000000818466c0 x10: 000000007a248d2f x9 : 0000000000000000
<4>[ 16.685918] x8 : ffffff80028cb6b4 x7 : 0000000000000000 x6 : 0000000000000000
<4>[ 16.693036] x5 : fffffffe00000000 x4 : ffffffc078e28098 x3 : 0000000000000000
<4>[ 16.700154] x2 : 0000000000000000 x1 : ffffff80046a6000 x0 : ffffffc078e28088
<4>[ 16.707274] Call trace:
<4>[ 16.714382] edma_napi+0x2c0/0x4c4 [qca_nss_dp]
<4>[ 16.716643] __napi_poll+0x38/0x178
<4>[ 16.721153] net_rx_action+0x134/0x2b0
<4>[ 16.724627] handle_softirqs+0xf8/0x22c
<4>[ 16.728447] __do_softirq+0x14/0x20
<4>[ 16.732177] ____do_softirq+0x10/0x1c
<4>[ 16.735651] call_on_irq_stack+0x24/0x4c
<4>[ 16.739470] do_softirq_own_stack+0x1c/0x44
<4>[ 16.743464] irq_exit_rcu+0x90/0xc8
<4>[ 16.747369] el1_interrupt+0x38/0x68
<4>[ 16.750842] el1h_64_irq_handler+0x18/0x24
<4>[ 16.754662] el1h_64_irq+0x68/0x6c
<4>[ 16.758567] mas_store_gfp+0x0/0x8c
<4>[ 16.761953] do_vmi_munmap+0x80/0xd8
<4>[ 16.765339] __vm_munmap+0x84/0x100
<4>[ 16.769156] __arm64_sys_munmap+0x20/0x30
<4>[ 16.772370] invoke_syscall.constprop.0+0x5c/0x100
<4>[ 16.776539] do_el0_svc+0x40/0xcc
<4>[ 16.781223] el0_svc+0x30/0xb8
<4>[ 16.784608] el0t_64_sync_handler+0x120/0x12c
<4>[ 16.787563] el0t_64_sync+0x178/0x17c
<0>[ 16.791992] Code: 54000240 f9400001 b4ffff81 f941e022 (f9404843)
<4>[ 16.795639] ---[ end trace 0000000000000000 ]---
Panic#2 Part1
<6>[ 4.932762] xhci-hcd xhci-hcd.2.auto: xHCI Host Controller
<6>[ 4.938549] xhci-hcd xhci-hcd.2.auto: new USB bus registered, assigned bus number 4
<6>[ 4.944113] xhci-hcd xhci-hcd.2.auto: Host supports USB 3.0 SuperSpeed
<6>[ 4.951957] hub 3-0:1.0: USB hub found
<6>[ 4.958259] hub 3-0:1.0: 1 port detected
<6>[ 4.962206] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
<6>[ 4.966300] hub 4-0:1.0: USB hub found
<6>[ 4.974153] hub 4-0:1.0: 1 port detected
<6>[ 4.982102] usbcore: registered new interface driver usb-storage
<6>[ 4.983122] usbcore: registered new interface driver uas
<14>[ 4.988021] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<14>[ 5.003471] init: - preinit -
<6>[ 6.008907] usb 2-1: new SuperSpeed USB device number 2 using xhci-hcd
<6>[ 6.043513] scsi host0: uas
<5>[ 7.113824] scsi 0:0:0:0: Direct-Access SanDisk Extreme 55AE 3008 PQ: 0 ANSI: 6
<5>[ 7.116766] scsi 0:0:0:1: Enclosure SanDisk SES Device 3008 PQ: 0 ANSI: 6
<5>[ 7.117139] sd 0:0:0:0: [sda] 3906963617 512-byte logical blocks: (2.00 TB/1.82 TiB)
<5>[ 7.129403] sd 0:0:0:0: [sda] Write Protect is off
<7>[ 7.136781] sd 0:0:0:0: [sda] Mode Sense: 37 00 10 00
<5>[ 7.137001] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
<6>[ 7.259033] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
<6>[ 7.259055] sd 0:0:0:0: [sda] Optimal transfer size 1048576 bytes
<6>[ 7.268192] sda: sda1
<5>[ 7.270735] sd 0:0:0:0: [sda] Attached SCSI disk
<5>[ 13.128869] random: crng init done
<14>[ 15.602134] mount_root: loading kmods from internal overlay
<14>[ 15.614204] kmodloader: loading kernel modules from //etc/modules-boot.d/*
<14>[ 15.615516] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
<14>[ 16.036894] block: attempting to load /tmp/overlay/upper/etc/config/fstab
<11>[ 16.037033] block: unable to load configuration (fstab: Entry not found)
<14>[ 16.042801] block: attempting to load /tmp/overlay/etc/config/fstab
<11>[ 16.049532] block: unable to load configuration (fstab: Entry not found)
<14>[ 16.055490] block: attempting to load /etc/config/fstab
<11>[ 16.065707] block: unable to load configuration (fstab: Entry not found)
<11>[ 16.067389] block: no usable configuration
<14>[ 16.074592] block: attempting to load /etc/config/fstab
<11>[ 16.078248] block: unable to load configuration (fstab: Entry not found)
<11>[ 16.083408] block: no usable configuration
<6>[ 16.091221] loop0: detected capacity change from 0 to 1048576
<6>[ 16.128883] loop0: detected capacity change from 1048576 to 1009920
<6>[ 16.305783] EXT4-fs (loop0): recovery complete
<6>[ 16.305946] EXT4-fs (loop0): mounted filesystem xxxx-xxxxx-xxxx-xx r/w with ordered data mode. Quota mode: disabled.
<6>[ 16.489066] nss-dp 3a001600.dp4 lan1: PHY Link up speed: 1000
<1>[ 16.517642] Unable to handle kernel read from unreadable memory at virtual address 0000000000000090
<1>[ 16.517689] Mem abort info:
<1>[ 16.525507] ESR = 0x0000000096000005
<1>[ 16.528273] EC = 0x25: DABT (current EL), IL = 32 bits
<1>[ 16.532118] SET = 0, FnV = 0
<1>[ 16.537560] EA = 0, S1PTW = 0
<1>[ 16.540433] FSC = 0x05: level 1 translation fault
<1>[ 16.543466] Data abort info:
<1>[ 16.548323] ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000
<1>[ 16.551463] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
<1>[ 16.556756] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
<1>[ 16.561879] user pgtable: 4k pages, 39-bit VAs, pgdp=0000000042fe4000
<1>[ 16.567254] [0000000000000090] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000
<0>[ 16.573605] Internal error: Oops: 0000000096000005 [#1] PREEMPT SMP
<4>[ 16.582098] Modules linked in: uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom qca_nss_dp(O) qca_ssdk(O) ramoops reed_solomon pstore gpio_button_hotplug(O) vfat fat f2fs ext4 mbcache jbd2 aquantia hwmon crc_ccitt crc32c_generic crc32_generic
<4>[ 16.589723] CPU: 0 PID: 954 Comm: block Tainted: G O 6.6.52 #0
<4>[ 16.611959] Hardware name: QNAP 301w (DT)
<4>[ 16.619159] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<4>[ 16.623244] pc : edma_napi+0x2c0/0x4c4 [qca_nss_dp]
<4>[ 16.630011] lr : edma_napi+0x290/0x4c4 [qca_nss_dp]
<4>[ 16.634871] sp : ffffffc080003e00
<4>[ 16.639729] x29: ffffffc080003e00 x28: 0000000000000040 x27: ffffff800287c000
<4>[ 16.643211] x26: ffffffc080003f50 x25: ffffff803fd9e480 x24: ffffffc080b7d1c0
<4>[ 16.650327] x23: ffffffc080003f40 x22: ffffffc078e27ee0 x21: 0000000000000001
<4>[ 16.657446] x20: ffffff8003088900 x19: 0000000000000001 x18: 0000000000000014
<4>[ 16.664564] x17: 000000005c83cc57 x16: 00000000ccaae286 x15: 00000000027925ee
<4>[ 16.671683] x14: 00000000198e17db x13: 00000000310db0ac x12: 0000000067d52c77
<4>[ 16.678801] x11: 00000000818466c0 x10: 000000007a248d2f x9 : 0000000000000000
<4>[ 16.685918] x8 : ffffff80028cb6b4 x7 : 0000000000000000 x6 : 0000000000000000
<4>[ 16.693036] x5 : fffffffe00000000 x4 : ffffffc078e28098 x3 : 0000000000000000
<4>[ 16.700154] x2 : 0000000000000000 x1 : ffffff80046a6000 x0 : ffffffc078e28088
<4>[ 16.707274] Call trace:
<4>[ 16.714382] edma_napi+0x2c0/0x4c4 [qca_nss_dp]
<4>[ 16.716643] __napi_poll+0x38/0x178
<4>[ 16.721153] net_rx_action+0x134/0x2b0
<4>[ 16.724627] handle_softirqs+0xf8/0x22c
<4>[ 16.728447] __do_softirq+0x14/0x20
<4>[ 16.732177] ____do_softirq+0x10/0x1c
<4>[ 16.735651] call_on_irq_stack+0x24/0x4c
<4>[ 16.739470] do_softirq_own_stack+0x1c/0x44
<4>[ 16.743464] irq_exit_rcu+0x90/0xc8
<4>[ 16.747369] el1_interrupt+0x38/0x68
<4>[ 16.750842] el1h_64_irq_handler+0x18/0x24
<4>[ 16.754662] el1h_64_irq+0x68/0x6c
<4>[ 16.758567] mas_store_gfp+0x0/0x8c
<4>[ 16.761953] do_vmi_munmap+0x80/0xd8
<4>[ 16.765339] __vm_munmap+0x84/0x100
<4>[ 16.769156] __arm64_sys_munmap+0x20/0x30
<4>[ 16.772370] invoke_syscall.constprop.0+0x5c/0x100
<4>[ 16.776539] do_el0_svc+0x40/0xcc
<4>[ 16.781223] el0_svc+0x30/0xb8
<4>[ 16.784608] el0t_64_sync_handler+0x120/0x12c
<4>[ 16.787563] el0t_64_sync+0x178/0x17c
<0>[ 16.791992] Code: 54000240 f9400001 b4ffff81 f941e022 (f9404843)
<4>[ 16.795639] ---[ end trace 0000000000000000 ]---
<3>[ 16.807748] pstore: backend (ramoops) writing error (-28)
<0>[ 16.807772] Kernel panic - not syncing: Oops: Fatal exception in interrupt
<2>[ 16.812119] SMP: stopping secondary CPUs
<0>[ 16.818891] Kernel Offset: disabled
<0>[ 16.822961] CPU features: 0x0,00000000,00000000,0000400b
<0>[ 16.826178] Memory Limit: none
But seems this error after reboot affects other devices too as noted by lots of other users.
Maybe there is a solution already by @robimarko and PR.
1 Like
@sppmaster , @zxlhhyccc These particular crashes are related to the NSS-12.5-K6.x-NAPI branch of nss-packages. Due to the buggy nature of those patches, with no real performance gains, I've removed it altogether.
@sppmaster the PR is unrelated to this particular crash. You would see something like this:
[ 327.592248] Call trace:
[ 327.599355] free_netdev+0x164/0x1a0
[ 327.601615] 0xffffffc0798183a8
[ 327.605433] 0xffffffc079818478
[ 327.608298] platform_shutdown+0x24/0x34
[ 327.611424] device_shutdown+0x160/0x268
[ 327.615591] kernel_restart+0x40/0xc0
[ 327.619497] __do_sys_reboot+0x104/0x220
[ 327.623057] __arm64_sys_reboot+0x24/0x30
[ 327.627049] invoke_syscall.constprop.0+0x5c/0x100
[ 327.630956] do_el0_svc+0x40/0xc4
[ 327.635642] el0_svc+0x30/0xb8
[ 327.639026] el0t_64_sync_handler+0x120/0x12c
[ 327.641981] el0t_64_sync+0x178/0x17c
Key difference here it's free_netdev+0x164/0x1a0. If you see anything with nss and napi, very likely you might be using the NAPI branch.
1 Like
I'm a little ashamed because of the noise I made with my last post. Maybe in a hurry reading for that crash on reboot I overlooked the dmesg-ramoops files were simply left from the time I've tried NAPI threaded build. I've deleted them manually. And I really didn't cold reboot the router for a really long time (I can't even remember).
Really sorry and as always thanks for your invaluable help.
1 Like
No worries! Just glad to hear it wasn't another new NSS bug to chase down.
This is a good habit. I tend to manually move over ramoops dumps to a USB with timestamps so I know when exactly it occurred.
1 Like
may i ask a question
i prefer my builds with gro disabled also i delete 600-net-enable-fraglist-GRO-by-default.patch
this hack does negatifly impact or overcome disable_gro option
I didn't encounter any noticeable problems but I'm still not sure.
1 Like
You have the option to have it disabled before ECM NSS offloading comes up.
uci set ecm.general.disable_gro=1
uci commit ecm
You can verify with the following
for i in br-lan /sys/class/net/br-lan/brif/*; do
i=${i##*\/}
ethtool -k ${i} | \
awk -v i=$i '/generic-receive-offload/{printf "%8s %s\n", i,$2}'
done
Before
br-lan on
lan1 on
lan2 on
lan3 on
lan4 on
phy0-ap0 on
phy1-ap0 on
After
$ service qca-nss-ecm restart
br-lan off
lan1 off
lan2 off
lan3 off
lan4 off
phy0-ap0 off
phy1-ap0 off
2 Likes
Sorry to bother you again. I've compiled a build with latest commits.
Flashed it and after reboot this time I have dmesg-ramoops files with current date.
<4>[ 44.840940] CPU: 3 PID: 1627 Comm: napi/ath11k_ahb Tainted: G O 6.6.52 #0
<4>[ 44.863179] Hardware name: QNAP 301w (DT)
<4>[ 44.871504] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<4>[ 44.875502] pc : ath11k_dp_rx_process_mon_status+0xcc4/0x14c4 [ath11k]
<4>[ 44.882273] lr : ath11k_dp_rx_process_mon_status+0xcac/0x14c4 [ath11k]
<4>[ 44.888868] sp : ffffffc082dc3be0
<4>[ 44.895373] x29: ffffffc082dc3be0 x28: ffffffc082dc3cb0 x27: 0000000000000000
<4>[ 44.898767] x26: ffffff80232f2540 x25: 0000000000000001 x24: ffffff80082bbf48
<4>[ 44.905885] x23: ffffff802170a000 x22: ffffff80232f2000 x21: ffffff80232f2438
<4>[ 44.913003] x20: ffffff8026121800 x19: ffffff80082b0000 x18: 0000000000000000
<4>[ 44.920121] x17: 0000000000000002 x16: 0000000000000000 x15: 0000000000000000
<4>[ 44.927238] x14: 0000000000000060 x13: 0000000000000140 x12: 0000000000000002
<4>[ 44.934357] x11: 00000000000000c9 x10: 0000000000000000 x9 : 0000000000000002
<4>[ 44.941475] x8 : 0000000000000081 x7 : 0000000000000001 x6 : 0000000000000000
<4>[ 44.948593] x5 : 0000000001020091 x4 : ffffff80232f2438 x3 : ffffff80082b0000
<4>[ 44.955710] x2 : 0000000000000fd3 x1 : 0000000000000000 x0 : 0000000000000001
<4>[ 44.962830] Call trace:
<4>[ 44.969938] ath11k_dp_rx_process_mon_status+0xcc4/0x14c4 [ath11k]
<4>[ 44.972203] ath11k_dp_rx_process_mon_rings+0x2c/0x38 [ath11k]
<4>[ 44.978452] ath11k_dp_service_srng+0xb8/0x530 [ath11k]
<4>[ 44.984266] 0xffffffc07987f288
<4>[ 44.989383] __napi_poll+0x38/0x178
<4>[ 44.992508] napi_threaded_poll_loop+0x80/0x1b0
<4>[ 44.995983] napi_threaded_poll+0x78/0x84
<4>[ 45.000496] kthread+0x10c/0x110
<4>[ 45.004662] ret_from_fork+0x10/0x20
<0>[ 45.007964] Code: 91401060 39718000 b5ffe921 36001040 (f9400822)
<4>[ 45.011527] ---[ end trace 0000000000000000 ]---
<4>[ 44.840940] CPU: 3 PID: 1627 Comm: napi/ath11k_ahb Tainted: G O 6.6.52 #0
<4>[ 44.863179] Hardware name: QNAP 301w (DT)
<4>[ 44.871504] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<4>[ 44.875502] pc : ath11k_dp_rx_process_mon_status+0xcc4/0x14c4 [ath11k]
<4>[ 44.882273] lr : ath11k_dp_rx_process_mon_status+0xcac/0x14c4 [ath11k]
<4>[ 44.888868] sp : ffffffc082dc3be0
<4>[ 44.895373] x29: ffffffc082dc3be0 x28: ffffffc082dc3cb0 x27: 0000000000000000
<4>[ 44.898767] x26: ffffff80232f2540 x25: 0000000000000001 x24: ffffff80082bbf48
<4>[ 44.905885] x23: ffffff802170a000 x22: ffffff80232f2000 x21: ffffff80232f2438
<4>[ 44.913003] x20: ffffff8026121800 x19: ffffff80082b0000 x18: 0000000000000000
<4>[ 44.920121] x17: 0000000000000002 x16: 0000000000000000 x15: 0000000000000000
<4>[ 44.927238] x14: 0000000000000060 x13: 0000000000000140 x12: 0000000000000002
<4>[ 44.934357] x11: 00000000000000c9 x10: 0000000000000000 x9 : 0000000000000002
<4>[ 44.941475] x8 : 0000000000000081 x7 : 0000000000000001 x6 : 0000000000000000
<4>[ 44.948593] x5 : 0000000001020091 x4 : ffffff80232f2438 x3 : ffffff80082b0000
<4>[ 44.955710] x2 : 0000000000000fd3 x1 : 0000000000000000 x0 : 0000000000000001
<4>[ 44.962830] Call trace:
<4>[ 44.969938] ath11k_dp_rx_process_mon_status+0xcc4/0x14c4 [ath11k]
<4>[ 44.972203] ath11k_dp_rx_process_mon_rings+0x2c/0x38 [ath11k]
<4>[ 44.978452] ath11k_dp_service_srng+0xb8/0x530 [ath11k]
<4>[ 44.984266] 0xffffffc07987f288
<4>[ 44.989383] __napi_poll+0x38/0x178
<4>[ 44.992508] napi_threaded_poll_loop+0x80/0x1b0
<4>[ 44.995983] napi_threaded_poll+0x78/0x84
<4>[ 45.000496] kthread+0x10c/0x110
<4>[ 45.004662] ret_from_fork+0x10/0x20
<0>[ 45.007964] Code: 91401060 39718000 b5ffe921 36001040 (f9400822)
<4>[ 45.011527] ---[ end trace 0000000000000000 ]---
<3>[ 45.021831] pstore: backend (ramoops) writing error (-28)
<0>[ 45.022202] Kernel panic - not syncing: Oops: Fatal exception in interrupt
<2>[ 45.027500] SMP: stopping secondary CPUs
<0>[ 45.034269] Kernel Offset: disabled
<0>[ 45.038341] CPU features: 0x0,00000000,00000000,0000400b
<0>[ 45.041558] Memory Limit: none
I wonder how could be that possible I may use the NAPI branch.
In feeds.conf.default I have NSS-12.5-K6.x. Am I right or I have to check something else?
src-git packages https://git.openwrt.org/feed/packages.git
src-git luci https://git.openwrt.org/project/luci.git
src-git routing https://git.openwrt.org/feed/routing.git
src-git telephony https://git.openwrt.org/feed/telephony.git
src-git nss_packages https://github.com/qosmio/nss-packages.git;NSS-12.5-K6.x
src-git sqm_scripts_nss https://github.com/qosmio/sqm-scripts-nss.git
#src-git video https://github.com/openwrt/video.git
#src-git targets https://github.com/openwrt/targets.git
#src-git oldpackages http://git.openwrt.org/packages.git
#src-link custom /usr/src/openwrt/custom-feed