Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

qosmio · November 26, 2022, 10:00pm

But somehow i'm struggling to get to 1gbps with latest firmware.
...
I keep getting stuck around 880-940

You'll be hard pressed to find anyone considering that to be "stuck" or subpar performance. Those numbers are within "spec" for sustained gigabit speeds, and depending on how you're measuring these speeds (speedtest browser vs. speedtest cli, time of day, speedtest servers), the results will vary greatly.

You also have to factor in the physical limitations of the switch. Local speeds (which depending on how your vlans are setup) bypass NSS altogether (it's not needed) and will not reach true "1Gb". Even locally, with jumbo frames enabled, the most I'm able to achieve is 990mpbs.

Sagemcom F3896LG directly connected i get flat 1000mbps.

I looked over the spec sheet, the Sagemcom has a 2.5Gb WAN port, while the R7800 WAN and LAN are both 1Gb. The 2.5Gb WAN port is likely able to cope with some of the TCP overhead, which would account for you seeing 1000mbps in your testing.

JayJax · November 27, 2022, 12:33am

Yeah might just have to invest in an x86 system as router because of next years fiber operation with an ISP that can offer 10gbps.

@ACwifidude
I come to an issue with one of your latest images:

I know for a fact i have way more space that shown here, what happen and how can i fix it?

Just to notify, i have been battling with this for quite a while.
This router has a mount point that is 4GB and people been saying this is for the default firmware.
But i refuse to believe that the default firmware and software requires 3GB.
Is there any possibly way to figure out what is within this area and if i can eliminate what ever is in it to use for my own?

SiXX · November 27, 2022, 12:44am

880-940 would be considered “normal” and within the realm of acceptable. Standard variations in routing and load would cause the minor fluctuations.

JayJax · November 27, 2022, 1:43am

Another thing i found out is that wifi is using the main cpu cores on the NBG6817 firmware.
I can only reach up to 500mbit with a cpu usage of around 41%.

qosmio · November 27, 2022, 2:10am

I can only reach up to 500mbit with a cpu usage of around 41%.

I highly advise reading over this thread, I know it’s a lot of posts, but well worth the scroll.

Currently, we’ve only been able to achieve partial NSS acceleration for wifi on kernels 5.4 and newer. Stock is able to achieve full acceleration but the trade off is being stuck with an ancient kernel (3.14) with zero community support or @quarky’s original 4.4 build which was able to achieve full NSS acceleration. However, he’s since moved onto 5.4 and the community as a whole has moved to 5.10/5.15.

I will add, 500-600mbps that’s been achieved with partial NSS acceleration is comparable to the speeds achieved on stock firmware, at the cost of CPU cycles.

QCA is also not actively developing on ipq806x/ath10k platform anymore. The attention is more on ipq807x/ath11k/ath12k, therefore doubtful we’ll see any more improvements regarding NSS acceleration. Especially since the source for the firmware is closed source (not drivers, which is open)

JayJax · November 27, 2022, 2:20am

I just checked that i'm not on the ath10k firmware, just switched to this firmware but now i have this problem:

I can't seem to figure out what is wrong.
Tried disabling them, removing them to make new ones. Nothing works

-UPDATE-
Just got worse.
Just reflashed with newest ath10k firmware without keeping my settings, now the wireless tab is gone plus all the configs.
Tried to install kmod-ath10k-ct but says: Kernel: Version incompatible.
Tried to do it with the opkg install and github link didn't work, downloaded the file and did the opkg install /tmp/filename(kmod-ath10k-ct) and still is throwing errors:

I'm lost for words, i don't even know where to start to solve this problem.

SiXX · November 27, 2022, 3:14am

This is (again) normal behavior for these routers. @qosmio provided a good suggestion to spend some time reading through this thread and familiarizing yourself with the standard operations and expectations for ipq806x NSS builds. It’s a wonderful system for many users, but perhaps your needs/wishes go beyond what it can provide.

csharper2005 · November 27, 2022, 7:04am

I noticed a lot of such messages. What does it mean?

[ 1915.242109] ipq8064-mdio 37000000.mdio eth1: nss_gmac_xmit_frames: dropping skb
[ 1915.254521] ipq8064-mdio 37000000.mdio eth1: nss_gmac_xmit_frames: dropping skb
[ 1915.254741] ipq8064-mdio 37000000.mdio eth1: nss_gmac_xmit_frames: dropping skb
[ 1915.263132] ipq8064-mdio 37000000.mdio eth1: nss_gmac_xmit_frames: dropping skb

ACwifidude · November 27, 2022, 12:56pm

Unfortunately many networking functions still are single threaded so 50% could mean one CPU is near 100%, the other is at idle. Common NAT speeds should be between 880-940 when testing against public servers. Should be upper 930’s - 940 solid if testing with iperf at home (I’ve done this to prove the issue is your ISP or somewhere else in the public sphere). Wifi speeds can be tested similarly, high 500’s (rarely low 600’s) is a typical speed for a 2x2 newer client on 80Mhz channel before you max out the CPU.

Kmods modify the kernel and create a different hash as a result. I post all my packages but unfortunately despite the original packages being the ones built for the custom kernel- frequently opkg has kernel incompatibility errors with custom builds like this one that prevent installation of individual packages. Frustrating right?

The cleanest way to have all the packages you want is to build from scratch. It seems daunting but if you have a Linux computer you can follow the instructions in post #2 above to get exactly what you want.

MrHugelberg · November 27, 2022, 8:17pm

I do have random reboots as well unfortunately. runtime is like 2-3 days at max with the build from last week, before it easily ran 7days+++

xeonpj · November 27, 2022, 8:30pm

I don't know exactly what version of acwifidude it is, kind of old, from last month.

pattagghiu · November 27, 2022, 8:50pm

honestly, also with a "recent" windows getting a WSL is trivial..

btw, just got a reboot with your 5.15 (after more or less one week), i don't know if this is useful but i got pstore dump

<1>[501559.211440] 8<--- cut here ---
<1>[501559.211476] Unable to handle kernel paging request at virtual address 2000002b
<1>[501559.213399] pgd = 6b3cbad7
<1>[501559.220683] [2000002b] *pgd=00000000
<0>[501559.223465] Internal error: Oops: 5 [#1] SMP ARM
<4>[501559.227283] Modules linked in: ksmbd ecm nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet ath10k_pci ath10k_core ath wireguard nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_compat nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211 libchacha20poly1305 iptable_mangle iptable_filter ipt_REJECT ipt_ECN ip_tables curve25519_neon cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY x_tables sch_cake ppp_async poly1305_arm nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_generic libcrc32c crc_ccitt compat chacha_neon fuse sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow
<4>[501559.228023]  cls_basic act_skbedit act_mirred act_gact qca_nss_qdisc qca_nss_crypto qca_nss_pptp pptp qca_nss_pppoe pppoe pppox ppp_generic slhc ledtrig_usbport cryptodev nfsv4 nfsv3 nfs msdos ip_gre gre ifb ip6_udp_tunnel udp_tunnel sit ip6_tunnel tunnel6 tunnel4 ip_tunnel tun lockd sunrpc grace hfsplus hfs cdrom dns_resolver nls_utf8 nls_iso8859_15 nls_iso8859_1 nls_cp850 nls_cp437 nls_cp1250 wp512 twofish_generic twofish_common tea serpent_generic khazad cast6_generic cast5_generic cast_common camellia_generic blowfish_generic blowfish_common anubis xts crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha1_generic seqiv md5 md4 ecb des_generic libdes cmac authencesn authenc uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom ohci_platform ohci_hcd phy_qcom_ipq806x_usb ahci fsl_mph_dr_of ehci_platform ehci_fsl sd_mod ahci_platform libahci_platform libahci libata scsi_mod scsi_common ehci_hcd qca_nss_drv qca_nss_gmac ramoops reed_solomon pstore
<4>[501559.297887]  gpio_button_hotplug vfat fat f2fs ext4 mbcache jbd2 exfat dm_mirror dm_region_hash dm_log dm_crypt dm_mod dax crc32c_generic crc32_generic cbc encrypted_keys trusted tpm oid_registry asn1_encoder asn1_decoder
<4>[501559.406389] CPU: 1 PID: 5913 Comm: kworker/1:3 Not tainted 5.15.77 #0
<4>[501559.426069] Hardware name: Generic DT based system
<4>[501559.432580] Workqueue: events dbs_work_handler
<4>[501559.437352] PC is at __timer_delay+0x34/0x78
<4>[501559.441868] LR is at msm_read_current_timer+0x18/0x24
<4>[501559.446382] pc : [<c0647740>]    lr : [<c0818714>]    psr: a0000093
<4>[501559.451417] sp : c139bd50  ip : 00000000  fp : c173ef80
<4>[501559.457925] r10: c17d2b18  r9 : 00000000  r8 : c139bdac
<4>[501559.463222] r7 : 20000013  r6 : dd5dd043  r5 : 00000006  r4 : c0dcf1b8
<4>[501559.468516] r3 : c0d92750  r2 : c0d9b33c  r1 : 00000000  r0 : dd5dd043
<4>[501559.474855] Flags: NzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment none
<4>[501559.481457] Control: 10c5787d  Table: 4868c06a  DAC: 00000051
<1>[501559.488742] Register r0 information: non-slab/vmalloc memory
<1>[501559.494644] Register r1 information: NULL pointer
<1>[501559.500460] Register r2 information: non-slab/vmalloc memory
<1>[501559.505148] Register r3 information: non-slab/vmalloc memory
<1>[501559.510965] Register r4 information: non-slab/vmalloc memory
<1>[501559.516692] Register r5 information: non-paged memory
<1>[501559.522422] Register r6 information: non-slab/vmalloc memory
<1>[501559.527459] Register r7 information: non-paged memory
<1>[501559.533272] Register r8 information: non-slab/vmalloc memory
<1>[501559.538309] Register r9 information: NULL pointer
<1>[501559.544121] Register r10 information: slab kmalloc-256 start c17d2b00 pointer offset 24 size 256
<1>[501559.548816] Register r11 information: slab kmalloc-128 start c173ef80 pointer offset 0 size 128
<1>[501559.557759] Register r12 information: NULL pointer
<0>[501559.566606] Process kworker/1:3 (pid: 5913, stack limit = 0xbc51e798)
<0>[501559.571209] Stack: (0xc139bd50 to 0xc139c000)
<0>[501559.577807] bd40:                                     c15a565c 00000001 00000101 c06b5e7c
<0>[501559.582241] bd60: c15a5668 c0d8094c ffffffff 00000002 c139bdac c06b7550 00000000 c0d8094c
<0>[501559.590489] bd80: ffffffff c03438c8 c17d2b00 c0d8094c c17ce900 00000002 66d16940 23c34600
<0>[501559.598737] bda0: 000927c0 c06a48a0 c158be00 c1c355c0 66d16940 23c34600 c158be00 23c34600
<0>[501559.606982] bdc0: c17ce900 00000000 c15af0c0 c06a8ec4 c17ce7e8 c15af0c0 c06b6380 c140b180
<0>[501559.615229] bde0: 66d16940 23c34600 000927c0 c06a8f08 c17ce900 00000000 23c34600 c15af0c0
<0>[501559.623475] be00: dd999010 23c34600 000927c0 c06a9134 23c34600 23c34600 00000000 ffffffff
<0>[501559.631723] be20: 23c34600 c0d82b54 dd999248 c1663ac0 23c34600 00000001 c173f100 dd999010
<0>[501559.639968] be40: 23c34600 c06a92b4 c162b200 c15b4280 00000001 c173f100 dd999010 c07dea2c
<0>[501559.648214] be60: c17ce900 c173f100 c139bec8 23c34600 00000000 c162b200 dd999010 00000006
<0>[501559.656463] be80: 23c34600 c15b4280 00000000 000927c0 00000000 c07deddc c0d90610 c03438c8
<0>[501559.664709] bea0: c162bc00 23c34600 c162bc00 c0dd1034 00000000 00000001 001a5248 c07e35f4
<0>[501559.672954] bec0: c162bc00 001a5248 000927c0 00000024 dd99c480 c162bc00 c173f300 c173f280
<0>[501559.681200] bee0: c173f300 c1669200 c173f280 dd99f305 da84fc40 c07e685c c173f338 00000000
<0>[501559.689447] bf00: c173f304 c0d908f8 00000000 00000040 dd99f305 c07e7538 c173f338 da84fc00
<0>[501559.697692] bf20: dd99c080 dd99f300 00000000 c0339fb8 c139a000 dd99c080 00000008 da84fc00
<0>[501559.705939] bf40: da84fc18 dd99c080 00000008 dd99c098 c0d03d00 dd99c240 c139a000 c033a2b8
<0>[501559.714186] bf60: c0d0c3ac c0d9badc c0fddecc c40a4e40 c4069fc0 c033a244 da84fc00 c139a000
<0>[501559.722434] bf80: c0fddecc c4069fe0 00000000 c0341f84 c40a4e40 c0341e2c 00000000 00000000
<0>[501559.730680] bfa0: 00000000 00000000 00000000 c0300130 00000000 00000000 00000000 00000000
<0>[501559.738926] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[501559.747172] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<0>[501559.755418] [<c0647740>] (__timer_delay) from [<c06b5e7c>] (krait_mux_set_parent+0xd4/0x11c)
<0>[501559.763663] [<c06b5e7c>] (krait_mux_set_parent) from [<c06b7550>] (krait_notifier_cb+0x58/0xb8)
<0>[501559.772257] [<c06b7550>] (krait_notifier_cb) from [<c03438c8>] (srcu_notifier_call_chain+0x7c/0xf4)
<0>[501559.781111] [<c03438c8>] (srcu_notifier_call_chain) from [<c06a48a0>] (__clk_notify+0x70/0x94)
<0>[501559.790225] [<c06a48a0>] (__clk_notify) from [<c06a8ec4>] (clk_change_rate+0xfc/0x2b8)
<0>[501559.798643] [<c06a8ec4>] (clk_change_rate) from [<c06a8f08>] (clk_change_rate+0x140/0x2b8)
<0>[501559.806629] [<c06a8f08>] (clk_change_rate) from [<c06a9134>] (clk_core_set_rate_nolock+0xb4/0x1f8)
<0>[501559.814964] [<c06a9134>] (clk_core_set_rate_nolock) from [<c06a92b4>] (clk_set_rate+0x3c/0x170)
<0>[501559.823990] [<c06a92b4>] (clk_set_rate) from [<c07dea2c>] (_set_opp+0x250/0x518)
<0>[501559.833018] [<c07dea2c>] (_set_opp) from [<c07deddc>] (dev_pm_opp_set_rate+0xe8/0x218)
<0>[501559.840396] [<c07deddc>] (dev_pm_opp_set_rate) from [<c07e35f4>] (__cpufreq_driver_target+0x194/0x238)
<0>[501559.848212] [<c07e35f4>] (__cpufreq_driver_target) from [<c07e685c>] (od_dbs_update+0xcc/0x1a0)
<0>[501559.857587] [<c07e685c>] (od_dbs_update) from [<c07e7538>] (dbs_work_handler+0x38/0x74)
<0>[501559.866614] [<c07e7538>] (dbs_work_handler) from [<c0339fb8>] (process_one_work+0x230/0x4bc)
<0>[501559.874687] [<c0339fb8>] (process_one_work) from [<c033a2b8>] (worker_thread+0x74/0x5d4)
<0>[501559.883106] [<c033a2b8>] (worker_thread) from [<c0341f84>] (kthread+0x158/0x174)
<0>[501559.891265] [<c0341f84>] (kthread) from [<c0300130>] (ret_from_fork+0x14/0x24)
<0>[501559.898727] Exception stack(0xc139bfb0 to 0xc139bff8)
<0>[501559.905844] bfa0:                                     00000000 00000000 00000000 00000000
<0>[501559.911061] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[501559.919307] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000
<0>[501559.927556] Code: e5933000 e12fff33 e1a06000 e5943000 (ea000000) 
<4>[501559.934405] ---[ end trace b74949a9173545ab ]---

MrHugelberg · November 27, 2022, 10:36pm

Damn, that's Rock stable.
No idea what mine would have run before.. Usually I do manually restarts every now and then, but 41d is for sure freaking amazing.

th3voic3 · November 29, 2022, 1:50pm

@ACwifidude should the dependency cycle solved? I noticed qosmio saying that it's solved and we just need to refresh the NSS feed and I saw you added that to your repository. While compiling I got an error with the NSS crypto package again though.

qosmio · November 30, 2022, 1:28am

You should use the most recent image @ACwifidude created. About 2 days ago. It includes the commit that reverted enabling Krait CC which causes this issue.

pattagghiu:

[501559.755418] [<c0647740>] (__timer_delay) from [<c06b5e7c>] (krait_mux_set_parent+0xd4/0x11c)
<0>[501559.763663] [<c06b5e7c>] (krait_mux_set_parent) from [<c06b7550>] (krait_notifier_cb+0x58/0xb8)
<0>[501559.772257] [<c06b7550>] (krait_notifier_cb) from [<c03438c8>] (srcu_notifier_call_chain+0x7c/0xf4)
<0>[501559.781111] [<c03438c8>] (srcu_notifier_call_chain) from [<c06a48a0>] (__clk_notify+0x70/0x94)
<0>[501559.790225] [<c06a48a0>] (__clk_notify) from [<c06a8ec4>] (clk_change_rate+0xfc/0x2b8)
<0>[501559.798643] [<c06a8ec4>] (clk_change_rate) from [<c06a8f08>] (clk_change_rate+0x140/0x2b8)
<0>[501559.806629] [<c06a8f08>] (clk_change_rate) from [<c06a9134>] (clk_core_set_rate_nolock+0xb4/0x1f8)
<0>[501559.814964] [<c06a9134>] (clk_core_set_rate_nolock) from [<c06a92b4>] (clk_set_rate+0x3c/0x170)
<0>[501559.823990] [<c06a92b4>] (clk_set_rate) from [<c07dea2c>] (_set_opp+0x250/0x518)
<0>[501559.833018] [<c07dea2c>] (_set_opp) from [<c07deddc>] (dev_pm_opp_set_rate+0xe8/0x218)
<0>[501559.840396] [<c07deddc>] (dev_pm_opp_set_rate) from [<c07e35f4>] (__cpufreq_driver_target+0x194/0x238)
<0>[501559.848212] [<c07e35f4>] (__cpufreq_driver_target) from [<c07e685c>] (od_dbs_update+0xcc/0x1a0)
<0>[501559.857587] [<c07e685c>] (od_dbs_update) from [<c07e7538>] (dbs_work_handler+0x38/0x74)
<0>[501559.866614] [<c07e7538>] (dbs_work_handler) from [<c0339fb8>] (process_one_work+0x230/0x4bc)
<0>[501559.874687] [<c0339fb8>] (process_one_work) from [<c033a2b8>] (worker_thread+0x74/0x5d4)
<0>[501559.883106] [<c033a2b8>] (worker_thread) from [<c0341f84>] (kthread+0x158/0x174)
<0>[501559.891265] [<c0341f84>] (kthread) from [<c0300130>] (ret_from_fork+0x14/0x24)

tapper · November 30, 2022, 9:15am

I tested out latest build on Kernel 5.15 and got reboots after about 2 mins uptime. flashed back to kernel 5.10 and all is stable again. My r7800 is running as a dumb AP and I did save configs on flashing.

csharper2005 · November 30, 2022, 3:42pm

Have you already tried to enable firewall? It helps me avoid reboots.

qosmio · November 30, 2022, 4:14pm

did it generate a ramoops?

csharper2005 · November 30, 2022, 4:20pm

My ramoops: Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500) - #3365 by csharper2005

qosmio · November 30, 2022, 4:24pm

Are you building your own image or one provided by @ACwifidude?