Qualcommax NSS Build

JuliusBairaktaris · 2024-04-14T18:05:36.705Z

cjom:

CONFIG_PACKAGE_kmod-tcp-bbr=y

I have added this to my build.

Will look at @jkool702's setting later this week. @glassdoor if you have time to go through the settings or find other interesting settings, ffeel free to make a pull request.

jkool702 · 2024-04-14T19:16:54.284Z

FYI - you can make bbr the default built into the kernel (rather than a kmod that sysctl automatically loads during boot) by running

cat<<'EOF' >> target/linux/qualcommax/config-6.6
CONFIG_TCP_CONG_BBR=y
# CONFIG_DEFAULT_CUBIC is not set
CONFIG_DEFAULT_BBR=y
CONFIG_DEFAULT_TCP_CONG="bbr"
EOF

Of course, replace the config-6.6 with the appropriate kernel if you arent using kernel 6.6.x

JuliusBairaktaris · 2024-04-14T19:33:45.432Z

I don't see any reason to do that. Especially since my builds are automated I want to keep maintenance low.

AgustinLorenzo · 2024-04-14T20:15:00.164Z

Hi boys,

This version include NSS offload for WiFi

I am redo the repos over the latest commit from OpenWRT.

Changelog (include upstream of OpenWRT):

Changed the package from wpad-wolfssl to wpad-openssl, with wolfssl I saw that the IPv6 connectivity was breaking over the hours, however this is not happening with openssl.
Changed the package from openvpn-mbedtls to openvpn-openssl, to match the packages to the same encryption software and avoid duplication.
Added latest @qosmio changes to nss-packages

Notes:

The build is compiled with ath11k and NSS limits of 512MB and pbuf set to auto by default.
The build is compiled with GCC 13.2.
The default TCP congestion protocol is BBR.
By default, the radio0 (IoT) is off (you can enable in LUCI) and the radio 1 (2.4 GHz) y 2 (5 GHz) is on.
The default country code for the WiFi is USA to avoid problems with 160 MHZ because it does not work if it does not have a specified country code.
I have posted on my Github all the workarounds used in this build: https://github.com/AgustinLorenzo/openwrt/issues/5 (english) and https://github.com/AgustinLorenzo/openwrt/issues/6 (spanish)
Guide to upgrade to kernel 6.6 without losing your configuration: https://github.com/AgustinLorenzo/openwrt/issues/9 (english) and https://github.com/AgustinLorenzo/openwrt/issues/8 (spanish)

Sources:

Normal build: https://github.com/AgustinLorenzo/openwrt/tree/main_nss
Mesh build: https://github.com/AgustinLorenzo/openwrt/tree/main_nss_mesh
NSS packages: https://github.com/AgustinLorenzo/nss-packages/commits/main_nss_wifi/

BUILDED (NORMAL): https://github.com/AgustinLorenzo/openwrt/releases/tag/ipq807x-nsswifi-2024-04-14-0311

BUILDED (MESH BUILD): https://github.com/AgustinLorenzo/openwrt/releases/tag/ipq807x-nsswifi-mesh-2024-04-14-1434

NOTE: Now I only compile the build for the following devices AX3600, AX6, AX9000, Dynalink DL-WRX36, Buffalo WXR-5950AX12, QNAP QHora-301W, Netgear RAX120v2, Arcadyan AW1000 and Zyxel NBG7815, if someone has used my versions with other devices (and it is supported), let me know and I will add it.

AgustinLorenzo · 2024-04-14T20:17:58.088Z

Hi @cjom

BBR is love, and if the other party also uses (increasingly used) BBR performance is improved, plus it is connections with many latency or some packet loss the download or upload speed is not seriously affected as it happens with cubic.

Regards, Agustin

qosmio · 2024-04-14T23:45:19.841Z

sppmaster:

@qosmio Do you use any other optimization settings, apart from CONFIG_TARGET_OPTIMIZATION="-O3 -pipe -mcpu=cortex-a53+crc+crypto"
As my builds are based on your repo I'd better stick to your config as close as possible and don't risk to brick my builds enabling experimental settings (with doubtful performance improvements) as this is my main router and should be available 24/7.
Had anyone actually benchmarked the performance when bellow optimization was used?
CONFIG_TARGET_OPTIMIZATION="-O3 -pipe -mcpu=cortex-a53+crc+crypto"
Is there real speed improvement?

Those are really all you need. And even still it might be marginal in gains... if at all:

My minimal settings for optimal kernel settings and nss packages are

# vim: set ft=kconfig

CONFIG_USE_MOLD=y
CONFIG_USE_LTO=y
CONFIG_USE_GC_SECTIONS=y

CONFIG_TARGET_OPTIONS=y
CONFIG_TARGET_OPTIMIZATION="-O3 -pipe -mcpu=cortex-a53+crypto+crc"

# CONFIG_SECCOMP is not set
CONFIG_PKG_RELRO_PARTIAL=y
# CONFIG_PKG_RELRO_FULL is not set

# CONFIG_KERNEL_WERROR is not set
# CONFIG_KERNEL_SECCOMP is not set
# CONFIG_KERNEL_NAMESPACES is not set
# CONFIG_KERNEL_KEYS is not set
# CONFIG_KERNEL_IPV6_SEG6_LWTUNNEL is not set
# CONFIG_KERNEL_ELF_CORE is not set
# CONFIG_KERNEL_CGROUPS is not set
CONFIG_COLLECT_KERNEL_DEBUG=y
CONFIG_KERNEL_PERF_EVENTS=y
CONFIG_KERNEL_DYNAMIC_DEBUG=y
CONFIG_KERNEL_ARM_PMU=y
CONFIG_GCC_DEFAULT_PIE=y
CONFIG_EXPERIMENTAL=y
CONFIG_DEVEL=y
CONFIG_CCACHE=y
CONFIG_BUILD_PATENTED=y

# NSS related
CONFIG_NSS_MEM_PROFILE_HIGH=y
CONFIG_ATH11K_NSS_SUPPORT=y
CONFIG_ATH11K_MEM_PROFILE_512M=y
# CONFIG_ATH11K_THERMAL is not set
# CONFIG_ATH11K_MEM_PROFILE_1G is not set
# CONFIG_ATH11K_DEBUGFS_STA is not set
# CONFIG_ATH11K_DEBUGFS_HTT_STATS is not set
CONFIG_PACKAGE_MAC80211_NSS_SUPPORT=y
CONFIG_PACKAGE_sqm-scripts-nss=y
CONFIG_PACKAGE_nss-firmware-ipq8074=y
CONFIG_PACKAGE_nss-firmware-default=y
CONFIG_PACKAGE_kmod-qca-nss-ecm=y
CONFIG_PACKAGE_kmod-qca-nss-drv=y
CONFIG_PACKAGE_kmod-qca-nss-drv-qdisc=y
CONFIG_PACKAGE_kmod-qca-nss-drv-igs=y
CONFIG_PACKAGE_kmod-qca-mcs=y
CONFIG_PACKAGE_dnsmasq_full_dnssec=y
# CONFIG_NSS_DRV_PPTP_ENABLE is not set

# CONFIG_OPENSSL_ENGINE is not set

CONFIG_ZSTD_OPTIMIZE_O3=y
CONFIG_ZLIB_OPTIMIZE_SPEED=y

I've tried to keep my NSS fork as vanilla as possible, as it makes debugging/support easier. And any config changes I've needed to:

target/linux/qualcommax/{ipq807x/config-default,config*}

I've gotten merged upstream (except for QCE as module, SKB recycling, CONFIG_HZ=1000, and the default governor as performance).

zek-cmd · 2024-04-15T02:00:47.210Z

I made my custom build from your repo (nss_mesh) to add customized packages. I also experience the error, wifi won't boot.

I reverted to your 04-13 as of now.

glassdoor · 2024-04-15T02:41:25.398Z

Will check and test.
imo.. i am all for keeping it as maintenance free and as "default" as possible. Users can tweak after install

the only tweak I run after after every firmware update (removed the settings of country and channel as that is relevant only to me)

uci set pbuf.opt.memory_profile=1gb

uci set wireless.default_radio0.dtim_period='1'
uci set wireless.default_radio0.max_inactivity='86400'
uci set wireless.default_radio0.disassoc_low_ack='0'
uci set wireless.default_radio0.wnm_sleep_mode='1'

uci set wireless.default_radio1.dtim_period='1'
uci set wireless.default_radio1.max_inactivity='86400'
uci set wireless.default_radio1.disassoc_low_ack='0'
uci set wireless.default_radio1.wnm_sleep_mode='1'

uci set wireless.default_radio2.dtim_period='1'
uci set wireless.default_radio2.max_inactivity='86400'
uci set wireless.default_radio2.disassoc_low_ack='0'
uci set wireless.default_radio2.wnm_sleep_mode='1'

I adopted qosmio's wifi settings but changed dtim to 1.. as the wifi calling on the iphones does not trigger most of the time when set to 3 or the default 2. Works fine with 1 every single time and no AP poll info spam with the above setting.

xlighting2017 · 2024-04-15T05:32:55.529Z

Hi @qosmio:
another one I'd like to confirm with you...

there is a "packet steering" in vanilla openwrt, and we have "smp_affinity" in ipq807x, will these two get along nicely, or we need to disable "packet steering"? (just like "flow offloading" and NSS-ECM)

Thanks in advance!

JuliusBairaktaris · 2024-04-15T06:34:09.423Z

Setting the congestion algorithm to brr removed all bufferbloat over wifi for me on speedtest.net and waveform. But they are using TCP, so it makes sense. My Wifi performance for WAN is now identical to my lan performance even if i sit 3 rooms away. Mind you I only have 150mbps down 70 Mbps up so results might vary for faster lines.

(tested over 160mhz channel)

qosmio · 2024-04-15T06:42:56.727Z

xlighting2017:

will these two get along nicely, or we need to disable "packet steering"? (just like "flow offloading" and NSS-ECM)

Keep that disabled. Always.

smp_affinity pinning works on tying cores to various subsystems (physical cores to wifi, nss, usb, etc). Whereas the packet steering works on tx/rx in software, which you'd end up losing the benefit of hardware offloaded via NSS.

anom3 · 2024-04-15T06:47:32.310Z

ill add that a week or so back when i was migrating my devices to the latest nss builds i flipped on packet steering and on reboot was greeted with a nearly unresponsive device.

crazy packet loss and the pings that did make it through 3000+ms.

so bad that i was unable to disable it without booting into failsafe.

glassdoor · 2024-04-15T07:32:55.598Z

with bbr and using as AP.. iperf3 test intranet and on internet is about the same (maybe marginally better) compared to cubic. 1Gbps up/down + 2.5Gbps down/1Gbps up fiber links on PBR.
sending (upload)

[SUM]   0.00-10.00  sec  1.00 GBytes   859 Mbits/sec                  sender
[SUM]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec                  receiver

receiving (download)

[SUM]   0.00-10.01  sec  1.08 GBytes   925 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec  1.06 GBytes   906 Mbits/sec                  receiver

In my situation, the limiting factor is the 1Gbps network port on the ax3600
I am really impressed with the current state(both performance and stability) of the firmware that I am seriously considering

Dynalink Dl-WRX36 (has 2.5Gbe) - good to go now and price is a steal
Linksys MX8500 (has 6GHz) - Openwrt PR imminent but 6GHz does not work. Not that cheap

quarky · 2024-04-15T08:43:22.285Z

glassdoor:

with bbr and using as AP

I may be wrong, but TCP congestion algorithm is only used when in routing mode. Using as an AP basically just bridges WiFi to LAN interfaces and packet are pushed as fast as possible via the interface queues.

glassdoor · 2024-04-15T08:57:16.056Z

ya.. thats why i don't see any significant change in performance. that being said.. it did not make it worst.
I am sure I am in the minority using it as an AP. Most will be using it as a router. So they will see benefits.

loaNga0m · 2024-04-15T09:01:32.374Z

The congestion control of the router is only used when the traffic is terminated at the router.
In other words, when the Iperf3 server / client is running directly on the router.
Or if, for example, PPPoE is used as the WAN protocol.

If traffic is simply forwarded, it is not used at all.
In addition, each device / client has its own congestion control.
Just because you set PBR on the router does not change the congestion control of your clients.

Edit: you can use iperf3 with verbose option -V
This will show you the congestion control used on both sides

JuliusBairaktaris · 2024-04-15T09:12:31.432Z

bbr also "breaks" luci (makes it unusably slow) so i reverted it in my builds

arispap2003 · 2024-04-15T09:51:34.614Z

Is there any build with the mwan3 package preinstalled using the NSS package of qosmio?
,
thanks, everyone for the time and effort

glassdoor · 2024-04-15T10:55:00.683Z

luci works normal for me. what do you mean by slow?

i just tested the latest build which reverted to cubic..
using ax3600 as iperf3 host
with bbr, receiving/dowload on wireless client

[SUM]   0.00-10.00  sec   602 MBytes   505 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec   597 MBytes   501 Mbits/sec                  receiver

with cubic, receiving/download on wireless client

[SUM]   0.00-10.01  sec   858 MBytes   719 Mbits/sec    0             sender
[SUM]   0.00-10.00  sec   852 MBytes   715 Mbits/sec                  receiver

default cubic has much higher throughput.

case in point.. which is why i almost never use the ap as iperf3 host/target

so imo it's best to leave in in the default which is cubic

cjom · 2024-04-15T13:03:08.348Z

I'm using a fork of your repository (mostly to add wireguard and adguardhome) and luci is working OK with BBR enabled.