Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

apccv · February 10, 2023, 8:28pm

Sorry to hear you are having issues.

I have not tried using VPN with that build yet. Have plenty of old ath79 routers that are good enough for that. We were not road warriors to begin with and COVID didn't make a case for it either. We use VPN while on the go mostly to deal with unsecure WiFi and block ads... there is little on our network that we have exposed to the outside. Lately I find we are getting better results for securing roaming devices by using a cloud hosted VM. Oracle's free tier works well for it. Just noticed that they now allow a 4CPU/8GB RAM aarch64 linux instance for free... setting aside the weirdness (to me at least) of Oracle ARM linux that might be my next best friend while on the go.

If I get some time to try out the builds as VPN servers I'll post results.

It should work. The NSS modules for OpenVPN are not being built (and somewhere I read that the results when using them were far from stellar). The cyphers for WireGuard are probably too new for what the NSS module can do. Both WireGuard and OpenVPN should be using vanilla modules (for WireGuard) or vanilla application code (for OpenVPN).

That you can't get either to work makes me think it's not NSS related.

Is there anything interesting showing up in logs?

apccv · February 10, 2023, 8:43pm

Sorry it took so long. It's been busy days lately.

22.03.2 Imagebuilder
22.03.3 ImageBuilder

Can also post links to the SDKs and toolchains but their usefulness has always been limited to me.

For either toolchain once the GitHub repo key is added to keys and the list of repos updated, it will pick up packages from the official OpenWRT repositories but then use the build GitHub for base and kernel packages. Works well for me as I have several flavours of C2600 (as AP, as router w/WiFi and as router without WiFi). Good luck.

rickkz0r · February 11, 2023, 5:01pm

First of all thanks to all that made this possible. I've been running a 22.03 build forked from @ACwifidude 's repo for about two weeks now and it's been great, a big difference from stock OpenWRT which I've been running for the past 2 years. I can now max out my 350Mb/s uplink, with low latency on both wifi and the uplink (fq_codel) and it's been perfectly stable so far.

However, I do have a slight problem: The other day I tried to reset DSCP values from eth0 using mangle tables and I noticed the rules only work for the first odd packets, once the flow starts the original DSCP values return. I've tried a few different places (Pre/Postrouting,Forward) but I can't get it to work - at least not for traffic routed from eth0. Bridged traffic from wlan1 to eth1 is in fact being marked - which I didn't really expect.

All this made me wonder:
1. are mangle tables / is setting DSCP supposed to work with NSS?
2. could using nssifb for fq_codel cause the problem?
3. is bridged traffic not using NSS?

Thanks in advance for any help or ideas.

vochong · February 11, 2023, 7:21pm

are mangle tables / is setting DSCP supposed to work with NSS?
No
could using nssifb for fq_codel cause the problem?
No
is bridged traffic not using NSS?
No. NSS is only intended for accelerating routing/nat.

imkebe · February 12, 2023, 12:01pm

I don't know what was an issue. It started to work again... I don't know... I jus installed curl. LOL.
BTW. I'm using also a C2600. As for now no issues, no restarts. I plan to upgrade my 600/60 connection to 1GB/s in following days so I'll test WAN again.

sppmaster · February 12, 2023, 12:12pm

Thanks for the ImageBuilder. I'll try it when I have spare time.
Currently I'm on @ACwifidude latest NSS master build with only added Samba4 and banIP 0.8.0pre5.
Yesterday there was a crash. Can anyone help with the reason for it. Maybe @quarky or @Ansuel.
Here are both dmesg-ramoops files.

<1>[334235.471734] 8<--- cut here ---
<1>[334235.471780] Unable to handle kernel NULL pointer dereference at virtual address 00000004
<1>[334235.473702] pgd = 5e12f297
<1>[334235.482027] [00000004] *pgd=00000000
<0>[334235.484636] Internal error: Oops: 5 [#1] SMP ARM
<4>[334235.488454] Modules linked in: ecm nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet ath10k_pci ath10k_core ath nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_compat nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211 iptable_mangle iptable_filter ipt_REJECT ipt_ECN ip_tables cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY x_tables usblp sch_cake ppp_async nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc_ccitt compat fuse sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact qca_nss_qdisc qca_nss_pptp pptp qca_nss_pppoe
<4>[334235.489349]  pppoe pppox ppp_generic slhc ledtrig_usbport cryptodev nfsv4 nfsv3 nfs msdos ip_gre gre ifb sit ip6_tunnel tunnel6 tunnel4 ip_tunnel tun lockd sunrpc grace hfsplus hfs cdrom dns_resolver nls_utf8 nls_iso8859_15 nls_iso8859_1 nls_cp850 nls_cp437 nls_cp1250 wp512 twofish_generic twofish_common tea serpent_generic khazad cast6_generic cast5_generic cast_common camellia_generic blowfish_generic blowfish_common anubis xts crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha1_generic seqiv ecb cmac authencesn authenc uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom ohci_platform ohci_hcd phy_qcom_ipq806x_usb ahci fsl_mph_dr_of ehci_platform ehci_fsl sd_mod ahci_platform libahci_platform libahci libata scsi_mod scsi_common ehci_hcd qca_nss_drv qca_nss_gmac ramoops reed_solomon pstore gpio_button_hotplug vfat fat f2fs ext4 mbcache jbd2 exfat dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax crc32c_generic crc32_generic cbc encrypted_keys
<4>[334235.558875]  trusted tpm oid_registry asn1_encoder asn1_decoder
<4>[334235.667448] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.88 #0
<4>[334235.673437] Hardware name: Generic DT based system
<4>[334235.679424] PC is at msm_read_timer_count+0xc/0x18
<4>[334235.684112] LR is at ktime_get+0x5c/0x104
<4>[334235.688972] pc : [<c0818444>]    lr : [<c03951c4>]    psr: 60000193
<4>[334235.693142] sp : c0d01e90  ip : c0d04fc8  fp : c0d04f58
<4>[334235.699651] r10: c0d01ef0  r9 : de80200c  r8 : c0c462fc
<4>[334235.704945] r7 : 00000000  r6 : a345a940  r5 : 03fa630a  r4 : c0dc0300
<4>[334235.710243] r3 : 00000000  r2 : 1cd49000  r1 : c0d01ec0  r0 : c0d92748
<4>[334235.716580] Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
<4>[334235.723180] Control: 10c5787d  Table: 47f5006a  DAC: 00000051
<1>[334235.730468] Register r0 information: non-slab/vmalloc memory
<1>[334235.736370] Register r1 information: non-slab/vmalloc memory
<1>[334235.742187] Register r2 information: non-paged memory
<1>[334235.747914] Register r3 information: NULL pointer
<1>[334235.752948] Register r4 information: non-slab/vmalloc memory
<1>[334235.757724] Register r5 information: non-paged memory
<1>[334235.763539] Register r6 information: non-paged memory
<1>[334235.768574] Register r7 information: NULL pointer
<1>[334235.773695] Register r8 information: non-slab/vmalloc memory
<1>[334235.778469] Register r9 information: 0-page vmalloc region starting at 0xde802000 allocated at gic_of_setup+0x50/0x100
<1>[334235.784295] Register r10 information: non-slab/vmalloc memory
<1>[334235.794877] Register r11 information: non-slab/vmalloc memory
<1>[334235.800780] Register r12 information: non-slab/vmalloc memory
<0>[334235.806595] Process swapper/0 (pid: 0, stack limit = 0x07d2f525)
<0>[334235.812413] Stack: (0xc0d01e90 to 0xc0d02000)
<0>[334235.818577] 1e80:                                     c0c47000 00012ffc 01fd8c53 dd98e7a8
<0>[334235.822927] 1ea0: c1410000 000000f5 00000000 c0c462fc de80200c c0d01ef0 c0d04f58 c03a54b8
<0>[334235.831172] 1ec0: c0c462f0 c1410000 000000f5 00000000 c0c462fc c0373c38 c0d05330 c0d01f10
<0>[334235.839420] 1ee0: c0d7ea9c de802000 c0c462fc c0663bac c0307620 60000013 ffffffff c0d01f44
<0>[334235.847665] 1f00: 00000000 c0d00000 c0d04ec8 c0300b7c aa8f9230 00000000 00000001 c03147e0
<0>[334235.855912] 1f20: 00000000 c0d04f24 c0d00000 00000000 00000000 ffffe000 c0d04ec8 c0d04f58
<0>[334235.864159] 1f40: c0d04fc8 c0d01f60 c030761c c0307620 60000013 ffffffff 00000051 c0d04ec8
<0>[334235.872405] 1f60: 00000000 c03519cc 430b0000 c0c468f8 c0d01f68 ffffe000 c0d9b01c c0d9b020
<0>[334235.880651] 1f80: 00000000 000000e6 c0d9b01c c0d9b020 00000000 430b0000 c0d9b010 c0d04ec8
<0>[334235.888897] 1fa0: 00000000 c0351d50 c0d9b03c c0c0119c ffffffff ffffffff 00000000 c0c006d8
<0>[334235.897145] 1fc0: 00000000 00000000 c0c36a50 6c731f17 00000000 c0c00470 00000051 10c0387d
<0>[334235.905391] 1fe0: 0000136c 430b0000 512f04d0 10c5387d 00000000 00000000 00000000 00000000
<0>[334235.913635] [<c0818444>] (msm_read_timer_count) from [<c03951c4>] (ktime_get+0x5c/0x104)
<0>[334235.921881] [<c03951c4>] (ktime_get) from [<c03a54b8>] (tick_irq_enter+0x48/0x158)
<0>[334235.930127] [<c03a54b8>] (tick_irq_enter) from [<c0373c38>] (handle_domain_irq+0x34/0x94)
<0>[334235.937592] [<c0373c38>] (handle_domain_irq) from [<c0663bac>] (gic_handle_irq+0x8c/0xb4)
<0>[334235.945926] [<c0663bac>] (gic_handle_irq) from [<c0300b7c>] (__irq_svc+0x5c/0x78)
<0>[334235.954169] Exception stack(0xc0d01f10 to 0xc0d01f58)
<0>[334235.961720] 1f00:                                     aa8f9230 00000000 00000001 c03147e0
<0>[334235.966850] 1f20: 00000000 c0d04f24 c0d00000 00000000 00000000 ffffe000 c0d04ec8 c0d04f58
<0>[334235.975094] 1f40: c0d04fc8 c0d01f60 c030761c c0307620 60000013 ffffffff
<0>[334235.983338] [<c0300b7c>] (__irq_svc) from [<c0307620>] (arch_cpu_idle+0x44/0x50)
<0>[334235.990194] [<c0307620>] (arch_cpu_idle) from [<c03519cc>] (do_idle+0x258/0x2b8)
<0>[334235.997573] [<c03519cc>] (do_idle) from [<c0351d50>] (cpu_startup_entry+0x1c/0x20)
<0>[334236.005038] [<c0351d50>] (cpu_startup_entry) from [<c0c0119c>] (start_kernel+0x6b8/0x6c8)
<0>[334236.012511] Code: e12fff1e e30130ac e34c30dd e5933000 (e5930004) 
<4>[334236.020837] ---[ end trace 514c7311cb789d5d ]---

<1>[334235.471734] 8<--- cut here ---
<1>[334235.471780] Unable to handle kernel NULL pointer dereference at virtual address 00000004
<1>[334235.473702] pgd = 5e12f297
<1>[334235.482027] [00000004] *pgd=00000000
<0>[334235.484636] Internal error: Oops: 5 [#1] SMP ARM
<4>[334235.488454] Modules linked in: ecm nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet ath10k_pci ath10k_core ath nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_compat nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211 iptable_mangle iptable_filter ipt_REJECT ipt_ECN ip_tables cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY x_tables usblp sch_cake ppp_async nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc_ccitt compat fuse sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact qca_nss_qdisc qca_nss_pptp pptp qca_nss_pppoe
<4>[334235.489349]  pppoe pppox ppp_generic slhc ledtrig_usbport cryptodev nfsv4 nfsv3 nfs msdos ip_gre gre ifb sit ip6_tunnel tunnel6 tunnel4 ip_tunnel tun lockd sunrpc grace hfsplus hfs cdrom dns_resolver nls_utf8 nls_iso8859_15 nls_iso8859_1 nls_cp850 nls_cp437 nls_cp1250 wp512 twofish_generic twofish_common tea serpent_generic khazad cast6_generic cast5_generic cast_common camellia_generic blowfish_generic blowfish_common anubis xts crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha1_generic seqiv ecb cmac authencesn authenc uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom ohci_platform ohci_hcd phy_qcom_ipq806x_usb ahci fsl_mph_dr_of ehci_platform ehci_fsl sd_mod ahci_platform libahci_platform libahci libata scsi_mod scsi_common ehci_hcd qca_nss_drv qca_nss_gmac ramoops reed_solomon pstore gpio_button_hotplug vfat fat f2fs ext4 mbcache jbd2 exfat dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax crc32c_generic crc32_generic cbc encrypted_keys
<4>[334235.558875]  trusted tpm oid_registry asn1_encoder asn1_decoder
<4>[334235.667448] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.88 #0
<4>[334235.673437] Hardware name: Generic DT based system
<4>[334235.679424] PC is at msm_read_timer_count+0xc/0x18
<4>[334235.684112] LR is at ktime_get+0x5c/0x104
<4>[334235.688972] pc : [<c0818444>]    lr : [<c03951c4>]    psr: 60000193
<4>[334235.693142] sp : c0d01e90  ip : c0d04fc8  fp : c0d04f58
<4>[334235.699651] r10: c0d01ef0  r9 : de80200c  r8 : c0c462fc
<4>[334235.704945] r7 : 00000000  r6 : a345a940  r5 : 03fa630a  r4 : c0dc0300
<4>[334235.710243] r3 : 00000000  r2 : 1cd49000  r1 : c0d01ec0  r0 : c0d92748
<4>[334235.716580] Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
<4>[334235.723180] Control: 10c5787d  Table: 47f5006a  DAC: 00000051
<1>[334235.730468] Register r0 information: non-slab/vmalloc memory
<1>[334235.736370] Register r1 information: non-slab/vmalloc memory
<1>[334235.742187] Register r2 information: non-paged memory
<1>[334235.747914] Register r3 information: NULL pointer
<1>[334235.752948] Register r4 information: non-slab/vmalloc memory
<1>[334235.757724] Register r5 information: non-paged memory
<1>[334235.763539] Register r6 information: non-paged memory
<1>[334235.768574] Register r7 information: NULL pointer
<1>[334235.773695] Register r8 information: non-slab/vmalloc memory
<1>[334235.778469] Register r9 information: 0-page vmalloc region starting at 0xde802000 allocated at gic_of_setup+0x50/0x100
<1>[334235.784295] Register r10 information: non-slab/vmalloc memory
<1>[334235.794877] Register r11 information: non-slab/vmalloc memory
<1>[334235.800780] Register r12 information: non-slab/vmalloc memory
<0>[334235.806595] Process swapper/0 (pid: 0, stack limit = 0x07d2f525)
<0>[334235.812413] Stack: (0xc0d01e90 to 0xc0d02000)
<0>[334235.818577] 1e80:                                     c0c47000 00012ffc 01fd8c53 dd98e7a8
<0>[334235.822927] 1ea0: c1410000 000000f5 00000000 c0c462fc de80200c c0d01ef0 c0d04f58 c03a54b8
<0>[334235.831172] 1ec0: c0c462f0 c1410000 000000f5 00000000 c0c462fc c0373c38 c0d05330 c0d01f10
<0>[334235.839420] 1ee0: c0d7ea9c de802000 c0c462fc c0663bac c0307620 60000013 ffffffff c0d01f44
<0>[334235.847665] 1f00: 00000000 c0d00000 c0d04ec8 c0300b7c aa8f9230 00000000 00000001 c03147e0
<0>[334235.855912] 1f20: 00000000 c0d04f24 c0d00000 00000000 00000000 ffffe000 c0d04ec8 c0d04f58
<0>[334235.864159] 1f40: c0d04fc8 c0d01f60 c030761c c0307620 60000013 ffffffff 00000051 c0d04ec8
<0>[334235.872405] 1f60: 00000000 c03519cc 430b0000 c0c468f8 c0d01f68 ffffe000 c0d9b01c c0d9b020
<0>[334235.880651] 1f80: 00000000 000000e6 c0d9b01c c0d9b020 00000000 430b0000 c0d9b010 c0d04ec8
<0>[334235.888897] 1fa0: 00000000 c0351d50 c0d9b03c c0c0119c ffffffff ffffffff 00000000 c0c006d8
<0>[334235.897145] 1fc0: 00000000 00000000 c0c36a50 6c731f17 00000000 c0c00470 00000051 10c0387d
<0>[334235.905391] 1fe0: 0000136c 430b0000 512f04d0 10c5387d 00000000 00000000 00000000 00000000
<0>[334235.913635] [<c0818444>] (msm_read_timer_count) from [<c03951c4>] (ktime_get+0x5c/0x104)
<0>[334235.921881] [<c03951c4>] (ktime_get) from [<c03a54b8>] (tick_irq_enter+0x48/0x158)
<0>[334235.930127] [<c03a54b8>] (tick_irq_enter) from [<c0373c38>] (handle_domain_irq+0x34/0x94)
<0>[334235.937592] [<c0373c38>] (handle_domain_irq) from [<c0663bac>] (gic_handle_irq+0x8c/0xb4)
<0>[334235.945926] [<c0663bac>] (gic_handle_irq) from [<c0300b7c>] (__irq_svc+0x5c/0x78)
<0>[334235.954169] Exception stack(0xc0d01f10 to 0xc0d01f58)
<0>[334235.961720] 1f00:                                     aa8f9230 00000000 00000001 c03147e0
<0>[334235.966850] 1f20: 00000000 c0d04f24 c0d00000 00000000 00000000 ffffe000 c0d04ec8 c0d04f58
<0>[334235.975094] 1f40: c0d04fc8 c0d01f60 c030761c c0307620 60000013 ffffffff
<0>[334235.983338] [<c0300b7c>] (__irq_svc) from [<c0307620>] (arch_cpu_idle+0x44/0x50)
<0>[334235.990194] [<c0307620>] (arch_cpu_idle) from [<c03519cc>] (do_idle+0x258/0x2b8)
<0>[334235.997573] [<c03519cc>] (do_idle) from [<c0351d50>] (cpu_startup_entry+0x1c/0x20)
<0>[334236.005038] [<c0351d50>] (cpu_startup_entry) from [<c0c0119c>] (start_kernel+0x6b8/0x6c8)
<0>[334236.012511] Code: e12fff1e e30130ac e34c30dd e5933000 (e5930004) 
<4>[334236.020837] ---[ end trace 514c7311cb789d5d ]---
<0>[334236.071101] Kernel panic - not syncing: Fatal exception in interrupt
<2>[334236.071148] CPU1: stopping
<4>[334236.076526] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D           5.15.88 #0
<4>[334236.079135] Hardware name: Generic DT based system
<4>[334236.086774] [<c030e560>] (unwind_backtrace) from [<c030a17c>] (show_stack+0x14/0x20)
<4>[334236.091464] [<c030a17c>] (show_stack) from [<c0649ca0>] (dump_stack_lvl+0x40/0x4c)
<4>[334236.099451] [<c0649ca0>] (dump_stack_lvl) from [<c030cfbc>] (do_handle_IPI+0x144/0x184)
<4>[334236.106916] [<c030cfbc>] (do_handle_IPI) from [<c030d018>] (ipi_handler+0x1c/0x2c)
<4>[334236.115248] [<c030d018>] (ipi_handler) from [<c037a3dc>] (handle_percpu_devid_irq+0x84/0x16c)
<4>[334236.122630] [<c037a3dc>] (handle_percpu_devid_irq) from [<c0373c6c>] (handle_domain_irq+0x68/0x94)
<4>[334236.131309] [<c0373c6c>] (handle_domain_irq) from [<c0663bac>] (gic_handle_irq+0x8c/0xb4)
<4>[334236.140250] [<c0663bac>] (gic_handle_irq) from [<c0300b7c>] (__irq_svc+0x5c/0x78)
<4>[334236.148580] Exception stack(0xc146df58 to 0xc146dfa0)
<4>[334236.156130] df40:                                                       0dbbadf0 00000000
<4>[334236.161261] df60: 00000001 c03147e0 00000001 c0d04f24 c146c000 00000000 00000000 ffffe000
<4>[334236.169507] df80: 00000000 c0d04f58 c0dc0e80 c146dfa8 c030761c c0307620 60000013 ffffffff
<4>[334236.177751] [<c0300b7c>] (__irq_svc) from [<c0307620>] (arch_cpu_idle+0x44/0x50)
<4>[334236.185994] [<c0307620>] (arch_cpu_idle) from [<c03519cc>] (do_idle+0x258/0x2b8)
<4>[334236.193546] [<c03519cc>] (do_idle) from [<c0351d50>] (cpu_startup_entry+0x1c/0x20)
<4>[334236.201012] [<c0351d50>] (cpu_startup_entry) from [<42301570>] (0x42301570)

apccv · February 13, 2023, 2:17am

Great! I have high hopes for these builds. I don't have time to keep reflashing. They better be stable.

Being honest however, longest uptime we've experienced is about 60 days. Something or other eventually happens and we reboot routers either proactively (as in "this thing better behave perfectly tomorrow") or reactively when something goes south.

For a while we were having IPv6 issues, but that was mostly due to my incompetence and some quirks with HE's tunnel broker, as our ISP can provision insane fiber speeds but has no plan to ever offer IPv6. I've given up. Since we went back to IPv4 only everything has been much more stable.

The other common issue is if someone gets creative and starts streaming audio via RTSP or some of its flavours (think AirPlay) something gets poisoned and 2-3 days later the main router dies. This has happened consistently since we got the C2600s online, regardless of OpenWRT version or NSS. Some nasty mem leak must happen with RTSP or Bonjour or whatever. There's nothing on forums and I've never found anything online, so it is either a C2600 quirk (all our routers have shown this issue at different times as we rotate them around when flashing) or a mainline kernel problem.

Anyways... please post if anything fails as whatever bites you might come after us too and I rather know about it in advance. Cheers!

quarky · February 13, 2023, 3:13am

Looks like the issue with the L2 cache corruption due to CPU frequency scaling. Your router crashed only after 3 days of uptime. I suppose your router is using the on-demand or schedutil CPU scaler?

If the router keeps rebooting, you may want to make the router stick to one CPU frequency and see if it helps stability.

sppmaster · February 13, 2023, 6:47am

Just for my better understanding wasn't this commit by @Ansuel supposed to resolve L2 cache instability (completely or maybe just partially).
As this topic was widely discussed here on the forum. Isn't there a software tool which can be used to test for this cache issue or to test for other hardware faults? Something like MemTestx86 software.
I use the master build default ondemand optimized settings.

echo 600000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
echo 600000 > /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq
echo 25 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor

quarky · February 13, 2023, 7:02am

If I'm not wrong, the issue has been there all the while. But somehow it became more severe with the 5.4 and later kernels in OpenWRT. @Ansuel tried to improve the L2 cache clock codes, but somehow it doesn't work as intended and still cause crashes. I think that commits just turns off the L2 cache scaling module, but then (I think) the L2 cache would then run on a freq. that's configured by U-Boot and likely stability will become a lottery draw. Couple that with the changing frequency of the CPU clock mis-matching the L2 clock and data corruption likely may occur in the L2 caches, causing kernel panics.

I may be completely off-base here. Maybe @Ansuel can provide a much better explanation.

sppmaster · February 13, 2023, 10:22am

I've done this after I've installed your build 22.03.3 but still I get kernel incompatible

Otherwise looks good and NSS works for me as expected. I've tried despite the error message and the modules installed.

dr3am_r · February 13, 2023, 12:18pm

Hi all, from experience which of the two builds form @ACwifidude is more stable the Master (qsdk11) or 22.03 (qsdk10) one ? Also is there a difference in performance or they are same ? I'm currently on the older 21.02 build and looking to upgrade. Thank you in advance.

apccv · February 13, 2023, 1:56pm

I don't have a 22.03.3 online so can't check everything right away.

The https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages/Packages.sig file references the 242547a2d8baa8b1 key name so that at least is correct. I may have messed up the key file contents.

The 5.10.161-1-284a7446c259633f86cfb37f83933105 is the 22.03.3 NSS build which means that is the right one. 5.10.146-1-387fedeb9ea0ae8956009ecc00bff57b is the official OpenWRT kernel from 22.03.2. If you sysupgraded from 22.03.2 and kept settings, the distfeeds.conf file might still have references to 22.03.2.

Can you ssh to the router, do an opkg update and post the results please? That should tell us which URLs it is going out to, if signature validation is failing and for what links. The UI might give you the same details if you refresh package lists.

distfeeds should look like this (everything 22.03.3). First line needs to be commented out or just removed:

root@WRT:~# cat /etc/opkg/distfeeds.conf 
#src/gz openwrt_core https://downloads.openwrt.org/releases/22.03.3/targets/ipq806x/generic/packages
src/gz openwrt_nss https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages
src/gz openwrt_base https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/base
src/gz openwrt_luci https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/luci
src/gz openwrt_packages https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/packages
src/gz openwrt_routing https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/routing
src/gz openwrt_telephony https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/telephony

EDIT: URL for repo above was not correct. Again, apologies, as I don´t have a 22.03.3 online so I cannot test it. I think the change above should fix the issue.

Should be this for 22.03.2:
https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.2-NSS/22.03.2/bin/targets/ipq806x/generic/packages

This for 22.03.3:
https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages

The way it should behave (got a 22.03.3 running to make sure):

root@192.168.1.1's password: 


BusyBox v1.35.0 (2023-01-03 00:24:21 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 22.03.3, r20028-43d71ad93e
 -----------------------------------------------------
root@OpenWrt:~# opkg update
Downloading https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_nss
Downloading https://raw.githubusercontent.com/APCCV/OpenWRT-22.03.3-NSS/22.03.3-nss/bin/targets/ipq806x/generic/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/base/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_base
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/base/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_luci
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/luci/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_packages
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/packages/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_routing
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/routing/Packages.sig
Signature check passed.
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading https://downloads.openwrt.org/releases/22.03.3/packages/arm_cortex-a15_neon-vfpv4/telephony/Packages.sig
Signature check passed.

EDIT 4.... I better leave this alone as the more I touch it the more mistakes I make.

SiXX · February 13, 2023, 4:26pm

The MASTER is set and forget stable for me, but I’m a fairly vanilla user and don’t add in any packages except Stubby.

dr3am_r · February 14, 2023, 10:15am

Are you using the Master ath10 build or the ath11 one ?

SiXX · February 14, 2023, 2:32pm

I use the primary one he links to in the first post.

bright_plastik · February 16, 2023, 8:58pm

Hi to all!
Anyone have a second to give me a thought on this?
It might be trivial, or perhaps I just came out with an ingenious idea?

apccv · February 16, 2023, 11:48pm

Well...

It's not that there are hundreds of devices for this NSS build. There's only two dozen, at most. You either have one of the supported devices or not.

It's not that we have several flavours of the build. The main repo is ACwifidude's and that is kept current on the first two posts. When in doubt, use that. Two flavours: stable or shapshot. That's it.

And then there would be the problem of finding a spot where to host the tool you are looking at, or to try carve it a space at the OpenWRT site. While I think this builds are great, in the grand scheme of OpenWRT things, these NSS builds are just a piece of it.

In short... I for one did read your post, but would not know where to start, how to host it, or what flavours to base the selector on. I may be short sighted but i don't see the value.

You have to be relatively adventurous to try OpenWRT on your router. You have to be more than that to try a non-mainstream build on your router. Over 2 years and 3500 posts later on this thread, if you are reading this, only you know if this build is something you want to try. A friendly(er) firmware selector should not make much of a difference to it.

bright_plastik · February 17, 2023, 12:38am

Well, you're surely right, considering the big picture of OWRT.
But I think considering the limited space routers have, and the various use cases that only here in the thread we have, that custom online imagebuilder would be very practical to virtually everybody.

I mean, the firmware selector is one of the most beautiful things owrt came up with in the last decade, to allow widespread adoption of the firmware! It's a pity to fall in caveats where it is not viable anymore, like with this fantastic NSS build.

But as you already said, if you know the advantages of this build, the jump to attempt building your own image with all needed kmods and ipk is relatively small. I'm one of those simpelsimples taking advantages of a proper linux router without knowing much of linux itself, unfortunately. Every accomplishment in that sense means some weeks of steep learning curve, and I already see you struggling with pulls and commits! Immagine me, on a VM, trying to remember how to put osboxes.org in the sudoers!

apccv · February 17, 2023, 1:54am

It's all good stuff.

Now I get what you are after. That's not the firmware selector AFAIK. Closest you can get to that is using the Image Builder and I think that has a very limited appeal for these devices given how much flash they have.

If anyone needs an image, the base image is most of the time enough. For most people the router just needs to route. Anything additional that is already in their wish list (USB support, SAMBA) might already be there... so taking ACwifidude's builds is probably more than enough.

The little value I've brought to this thread is one small commit that allows 22.03.2 (&3) to build with all base modules and a repo with those frozen in time. This addresses my problem of needing to add/drop modules without having to worry about reflashing because ACwifidude's images have been rebased. Anyone wanting to experiment can then add modules as they see fit at a later date, or add stuff not in ACwifidude's build. No need really to build an image and squashfs everything unless really cramming the flash with stuff. (How much do you need there anyways? It's not that you should be using free flash space to write non-OS files like media... it's not going to last long if used like that). If using my repo, LUCI will let you install any additional modules after flash via the UI, or opkg via shell.

Some other people here have very specific builds that work exclusively for them. I would not be comfortable with any of those unless I go through the commits or changes with a fine tooth comb. No offense, but I need to know what I'm running and the customizations might do more harm than good for what I need.

So in short... if you know what you need the router to do and want NSS, posts 1 & 2 have probably all you need. If you need more than that (like NFS, or some other packet driver for like a LTE dongle) and can live with a 22.03 build instead of snapshot, I suggest my repo. Ditto if you want to play and try out different kmods on 22.03 without doing a build.

If you are after anything more sophisticated than that, an image builder won't help much. You are looking at a custom build instead. Someone here might have built what you are looking for, but those are one-offs.