New build 9/9, fw bridging / offloading issues fixed. With nss.qos (eth0) even wifi gives A bufferbloat results.
iptables-mod-physdev
kmod-ipt-physdev
Thank you
Trying the 9/9 NSS build now. The firewall issue is indeed fixed for me but I'm seeing tons of repeat warning on 3 access points which I was not seeing on 9/2 build. Are you seeing similar? Last build I mostly just saw clients roaming between access points.
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2802.967238] ------------[ cut here ]------------
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2802.971896] WARNING: CPU: 0 PID: 0 at net/netfilter/nf_conntrack_rtcache.c:197 nf_rtcache_forward4+0x16c/0x1cc [nf_conntrack_rtcache]
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2802.976494] Modules linked in: ksmbd ecm ath9k ath9k_common ath9k_hw ath10k_pci ath10k_core ath qmi_wwan mac80211 iptable_nat ipt_REJECT cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_quota xt_policy xt_pkttype xt_physdev xt_owner xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_geoip xt_esp xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_addrtype xt_TCPMSS xt_REDIRECT xt_MASQUERADE xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY wireguard usbnet ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda sch_cake ppp_async nf_reject_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conntrack_netlink nf_conncount iptable_raw iptable_mangle iptable_filter ipt_ah ipt_ECN ip_tables exfat crc_ccitt compat cdc_wdm fuse act_connmark nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2802.976650] cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred qca_nss_qdisc qca_nss_pppoe pppoe pppox ppp_generic slhc ledtrig_usbport ledtrig_heartbeat xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 bonding ip6_gre ip_gre gre ifb ip6_udp_tunnel udp_tunnel sit qca_nss_drv ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel ipcomp esp4 ah4 ip6_tunnel qca_nss_gmac tunnel6 tunnel4 ip_tunnel tun xfrm_user xfrm_ipcomp af_key xfrm_algo vfat fat autofs4 nls_utf8 nls_iso8859_1 nls_cp437 rfkill sha512_generic sha1_generic seqiv md5 md4 ghash_generic gf128mul gcm echainiv ecb des_generic libdes ctr cmac ccm cbc
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.053614] authenc arc4 uas usb_storage leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom ohci_platform ohci_hcd phy_qcom_ipq806x_usb ahci fsl_mph_dr_of ehci_platform ehci_fsl sd_mod ahci_platform libahci_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug f2fs ext4 mbcache jbd2 crc32c_generic crc32_generic
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.169182] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 5.4.59 #0
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.191414] Hardware name: Generic DT based system
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.198713] [<c030fcec>] (unwind_backtrace) from [<c030bcbc>] (show_stack+0x14/0x20)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.203577] [<c030bcbc>] (show_stack) from [<c08ab580>] (dump_stack+0x90/0xa4)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.211472] [<c08ab580>] (dump_stack) from [<c0321cb4>] (__warn+0xb4/0xd0)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.218503] [<c0321cb4>] (__warn) from [<c0321d20>] (warn_slowpath_fmt+0x50/0x90)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.225365] [<c0321d20>] (warn_slowpath_fmt) from [<bf626234>] (nf_rtcache_forward4+0x16c/0x1cc [nf_conntrack_rtcache])
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.232920] [<bf626234>] (nf_rtcache_forward4 [nf_conntrack_rtcache]) from [<c07af204>] (nf_hook_slow+0x48/0xd8)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.243508] [<c07af204>] (nf_hook_slow) from [<c089ab08>] (br_nf_forward_ip+0x328/0x41c)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.253920] [<c089ab08>] (br_nf_forward_ip) from [<c07af204>] (nf_hook_slow+0x48/0xd8)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.261994] [<c07af204>] (nf_hook_slow) from [<c0882848>] (__br_forward+0xa4/0x140)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.269719] [<c0882848>] (__br_forward) from [<c088291c>] (deliver_clone+0x38/0x60)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.277273] [<c088291c>] (deliver_clone) from [<c088496c>] (br_handle_frame_finish+0x2c8/0x494)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.284912] [<c088496c>] (br_handle_frame_finish) from [<c089a37c>] (br_nf_hook_thresh+0xdc/0xe0)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.293592] [<c089a37c>] (br_nf_hook_thresh) from [<c089ad34>] (br_nf_pre_routing_finish+0x138/0x328)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.302620] [<c089ad34>] (br_nf_pre_routing_finish) from [<c089b378>] (br_nf_pre_routing+0x454/0x474)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.311823] [<c089b378>] (br_nf_pre_routing) from [<c0884c9c>] (br_handle_frame+0x164/0x3ec)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.321025] [<c0884c9c>] (br_handle_frame) from [<c07513b4>] (__netif_receive_skb_core+0x244/0xbe4)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.329531] [<c07513b4>] (__netif_receive_skb_core) from [<c0751d7c>] (__netif_receive_skb_one_core+0x28/0x58)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.338300] [<c0751d7c>] (__netif_receive_skb_one_core) from [<c0751e9c>] (netif_receive_skb_internal+0x40/0xac)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.348439] [<c0751e9c>] (netif_receive_skb_internal) from [<bf368044>] (nss_core_send_buffer+0x2314/0x245c [qca_nss_drv])
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.358834] [<bf368044>] (nss_core_send_buffer [qca_nss_drv]) from [<bf36907c>] (nss_core_handle_napi+0x194/0x238 [qca_nss_drv])
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.369622] [<bf36907c>] (nss_core_handle_napi [qca_nss_drv]) from [<c0753634>] (net_rx_action+0x118/0x374)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.381269] [<c0753634>] (net_rx_action) from [<c030227c>] (__do_softirq+0x114/0x298)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.390728] [<c030227c>] (__do_softirq) from [<c0325d7c>] (irq_exit+0xbc/0xec)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.398712] [<c0325d7c>] (irq_exit) from [<c0369870>] (__handle_domain_irq+0x6c/0xcc)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.405834] [<c0369870>] (__handle_domain_irq) from [<c05a55ac>] (gic_handle_irq+0x5c/0xb8)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.413731] [<c05a55ac>] (gic_handle_irq) from [<c0301a8c>] (__irq_svc+0x6c/0x90)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.421884] Exception stack(0xc0c01ed8 to 0xc0c01f20)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.429524] 1ec0: 00000000 0000028c
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.434577] 1ee0: 1cec6000 dda08c40 dd7d2800 00000000 dda07ff0 0000028c 0000028c 7da867c0
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.442737] 1f00: 7da79700 c0c01f80 ffed1940 c0c01f28 c06f5cc4 c06f5cc8 20000013 ffffffff
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.450886] [<c0301a8c>] (__irq_svc) from [<c06f5cc8>] (cpuidle_enter_state+0x98/0x4c8)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.459042] [<c06f5cc8>] (cpuidle_enter_state) from [<c06f6148>] (cpuidle_enter+0x3c/0x5c)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.466856] [<c06f6148>] (cpuidle_enter) from [<c034be10>] (do_idle+0x1d0/0x250)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.475188] [<c034be10>] (do_idle) from [<c034c138>] (cpu_startup_entry+0x1c/0x20)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.482740] [<c034c138>] (cpu_startup_entry) from [<c0b00e3c>] (start_kernel+0x4b4/0x4c4)
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.490120] ---[ end trace 6184e5b158842636 ]---
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.504648] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.504696] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.510866] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.518138] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.525361] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.532729] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.540005] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.547236] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.554601] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.561904] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.569183] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.576402] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.583772] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.591051] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.598361] ath10k_pci 0000:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.609356] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.612874] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.620229] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.627466] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.634839] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 1, skipped old beacon
Thu Sep 10 19:06:45 2020 kern.warn kernel: [ 2803.642097] ath10k_pci 0001:01:00.0: SWBA overrun on vdev 0, skipped old beacon
Nope, except for "ath10k nvalid peer..." but these show up on all r7800 master builds. IIRC it was addressed a few weeks ago.
Try a clean install if you haven't done so already.
[ 43.256369] nss_qdisc_init[1992]:NSS qdisc b5a3f5f6 (type 1) used along with non-nss qdiscs, or the interface is currently down
[ 43.260869] f72818fa: Found net device [eth0]
[ 43.271222] f72818fa: Net device [eth0] has NSS intf_num [1]
[ 43.275902] Nexthop successfully set for [eth0] to [nssifb]
[ 44.399556] debugfs: File 'virt_if' in directory 'stats' already present!
[ 44.399976] wlan1: Created a NSS virtual interface.
[ 44.405317] wlan1: RX-Q and tasklet initialized.
[ 50.289790] ath10k_pci 0001:01:00.0: unsupported HTC service id: 1536
[ 50.290069] ath10k_pci 0001:01:00.0: 10.4 wmi init: vdevs: 16 peers: 48 tid: 96
[ 50.295205] ath10k_pci 0001:01:00.0: msdu-desc: 2500 skid: 32
[ 50.380460] ath10k_pci 0001:01:00.0: wmi print 'P 48/48 V 16 K 144 PH 176 T 186 msdu-desc: 2500 sw-crypt: 0 ct-sta: 0'
[ 50.381255] ath10k_pci 0001:01:00.0: wmi print 'free: 132768 iram: 9624 sram: 46068'
[ 50.947333] ath10k_pci 0001:01:00.0: Firmware lacks feature flag indicating a retry limit of > 2 is OK, requested limit: 4
[ 50.948590] debugfs: File 'virt_if' in directory 'stats' already present!
[ 50.962838] wlan0: Created a NSS virtual interface.
[ 50.964129] wlan0: RX-Q and tasklet initialized.
[ 50.973090] br-lan: port 2(wlan1) entered blocking state
[ 50.973678] br-lan: port 2(wlan1) entered disabled state
[ 50.979251] device wlan1 entered promiscuous mode
[ 51.362552] ath10k_pci 0001:01:00.0: NOTE: Firmware DBGLOG output disabled in debug_mask: 0x10000000
[ 51.372468] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
[ 51.372652] br-lan: port 2(wlan1) entered blocking state
[ 51.377969] br-lan: port 2(wlan1) entered forwarding state
[ 57.206089] ath10k_pci 0000:01:00.0: unsupported HTC service id: 1536
[ 57.206403] ath10k_pci 0000:01:00.0: 10.4 wmi init: vdevs: 16 peers: 48 tid: 96
[ 57.211614] ath10k_pci 0000:01:00.0: msdu-desc: 2500 skid: 32
[ 57.293732] ath10k_pci 0000:01:00.0: wmi print 'P 48/48 V 16 K 144 PH 176 T 186 msdu-desc: 2500 sw-crypt: 0 ct-sta: 0'
[ 57.294458] ath10k_pci 0000:01:00.0: wmi print 'free: 132768 iram: 9624 sram: 46068'
[ 57.675397] ath10k_pci 0000:01:00.0: Firmware lacks feature flag indicating a retry limit of > 2 is OK, requested limit: 4
[ 57.687974] br-lan: port 3(wlan0) entered blocking state
[ 57.688007] br-lan: port 3(wlan0) entered disabled state
[ 57.692706] device wlan0 entered promiscuous mode
[ 57.721875] ath10k_pci 0001:01:00.0: Invalid peer id 0 or peer stats buffer, peer: fc8f0351 sta: 00000000
[ 58.314352] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[ 58.314666] br-lan: port 3(wlan0) entered blocking state
[ 58.319913] br-lan: port 3(wlan0) entered forwarding state
[ 105.922851] ath10k_pci 0001:01:00.0: Invalid VHT mcs 15 peer stats
[ 1745.601735] ath10k_pci 0001:01:00.0: htt tx: fixing invalid VHT TX rate code 0xff
[ 6946.013820] ath10k_pci 0001:01:00.0: wmi: fixing invalid VHT TX rate code 0xffI still have a problems with port forwarding and SqM on NSS build. Too fresh right now
SQM doesn’t directly communicate yet. You have to use a custom script. Not sure what Kong is using for a base. Here are my tweaks if that is helpful:
The 2020-09-09 trunk/ipq806x-nss build actually works with 160MHz width on channel 100! I'm seeing speeds of ~900Mbps on WiFi, finally.
The only problem is the router keeps rebooting every few minutes. Does anyone else have this issue? I've reverted back to 2020-08-18 19.07 build for the time being (which doesn't work with 160MHz even when set).
Couldn't find it at first! But yes, that fixed the issue. Will report back if it doesn't!
If the random reboots still happens, try adding this into your startup script and see if it helps:
echo 800000000 > /proc/sys/dev/nss/clock/current_freq
Edit: btw did you take note of the CPU load when you hit 900Mbps over WiFi? If you don’t mind, can you try again and let me know what’s the CPU load, especially the sirq %?
I tried the CPU frequency setting before disabking ksmbd but that didn't help. Disabling ksmbd alone worked.
Looks like the 160MHz setting doesn't work again (was it random luck before?) and so I'm capping out at ~500Mbps once again. How annoying. If it starts working again I'll let you know 
My port forwarding rules (on the NSS build) magically started working after I just restarted the firewall service after the device was fully booted. I didn't add/remove any rules either. So now I basically just have a cron job that restarts the firewall 5 minutes after each reboot and that seems to be working.
Whats your startup script ?
Under System > Scheduled Tasks I just have:
@reboot sleep 300 && /etc/init.d/firewall restart
Does not work here
I just extended the current sqm code:
Works ok for my config. Not as good as cake but my flent tests gave me:
+~50ms latency without qos (where I believe that my isp already does already do some sort of shaping)
+~6ms latency with nssfq_codel
+~3ms latency with cake
The rules are really simple right now, they should be extened to allow to prioritize certain traffic
7 Likes
So a new build is coming?
Has anyone on the NSS firmware (or other) managed to get a channel width of 160 MHz working consistently? I managed to a few weeks ago but can't reproduce it for the life of me and I'm wondering what to do without having to reset and start from scratch again.
Quick note:
in nss.qos the following line seems to be orphaned:
BURST=" burst 1Mb "
as the two nsstbl invocations both spell out "burst 1Mb" explicitly instead of just calling ${BURST}. Bit more to the point, what is the rationale for a one size fits all burst buffer? In the other scripts we try to scale such buffers by the worst-case time required to service/empty them at the configured shaper rate, as these buffers will introduce additional variable delay up to that duration. Now, I have no idea for nsstbl there might be limitations for acceptable bursts, but...
also, if we fold the module loading explicitly into nss.qos, it might be possible to just copy nss.qos and nss.qos.help into an existing sqm-scripts installation to get the goodness of an accelerated shaper.
1 Like
Yeah this was just a quick and dirty addition. I currently lack time in order to really create a few fancy rules and nssfq_codel and nsstbl works a bit different, thus can't use my good old rules that I implemented for dd-wrt qos. To anyone, feel free to play with it, as far as I could tell most people had touble to understand what's needs to be done in order to make it available to webif and run automatically.
1 Like
I don't see any real changes in master that justify to create a new build. Current build comes with nssqos and should work for most setups.