Qualcommax NSS Build

[qosmio]

Looks I mixed up pci.c and pcic.c. The later is where it was supposed to be.

Just pushed and update.

[JuliusBairaktaris]

Revert "CONFIG_OPENSSL_WITH_DEPRECATED=n"

Doesn't build for me with the same configuration i used for Kernel 6.6.35.

[pexcn]

Hello @qosmio , My download and upload speeds are 300Mbps and 30Mbps respectively. The SQM feature seems not work, because when I downloading, the gateway delay of ping pppoe will become high. How should I debug it?

This my config:

The logging:

[sppmaster]

@qosmio
I'm testing NAPI threading branch with all the latest (amazing!) commits compiled with GCC14.
I have two questions.
First one is about

qca-nss-drv: add missing gmac patch
Add in missing gmac. Meant for IPQ806X

What is this commit (for IPQ806X) for?
How do we use it on IPQ806X?

And the second one to you and all testers.
With committing - ath11k_nss: Add mac80211 based AQL support and the older popular thread

AQL and the ath10k is *lovely*

I built openwrt head last night when the final patches for AQL and the ath10k-ct landed. Oh! what a difference in bufferbloat-related performance! Where, on this purposely terrible link (12mbits or so), delays would spike past 4 seconds, and things like the babel routing protocol fail with any background load, now it holds things steady well below 60ms with totally solid throughput. Before: [before] After: [after] I tested this on the uap mesh and mesh pro using the smallbuffers version of…

On IPQ806X I used below commands in rc.local

for ac in 0 1 2 3; do echo $ac 2000 2000 > /sys/kernel/debug/ieee80211/phy0/aql_txq_limit; done
for ac in 0 1 2 3; do echo $ac 2000 2000 > /sys/kernel/debug/ieee80211/phy1/aql_txq_limit; done

to change the AQL txq limits dynamically.

What commands do you guys use on IPQ807X?

[glassdoor]

build fails with GCC 14. @qosmio
testing NAPI NSS branch. Working well and no errors thus far.
Again.. big thank you.

[asvio]

Hi @qosmio
First of all, thank you for the work and effort you are putting in.
Today I compiled the latest changes published in the qualcommax-6.x-nss-wifi branch and I noticed a slight increase in processor usage.
It is really very little (0.3%-0.5%) but this seems to indicate that there is some problem. It seems to be seen that the increase occurs in softirq and in interruption

[qosmio]

@JuliusBairaktaris

Doesn't build for me with the same configuration i used for Kernel 6.6.35.

Race condition in your build? Things like these usually happen if you're running a parallel build, and don't explicitly run make download prior to the actual build procedure.

2024-07-03T06:45:04.6400224Z make[4]: *** [GNUmakefile:108: abort-due-to-no-makefile] Error 1
2024-07-03T06:45:04.6401836Z make[4]: Leaving directory '/home/runner/work/Qualcommax_NSS_Builder/Qualcommax_NSS_Builder/build_dir/host/libtool-2.4.7'
2024-07-03T06:45:04.6405059Z make[3]: [Makefile:67: /home/runner/work/Qualcommax_NSS_Builder/Qualcommax_NSS_Builder/build_dir/host/libtool-2.4.7/.preparedf9d99fef7f31896b861e6b6beb933763_6664517399ebbbc92a37c5bb081b5c53] Error 2 (ignored)

@pexcn

Hello @qosmio , My download and upload speeds are 300Mbps and 30Mbps respectively. The SQM feature seems not work, because when I downloading, the gateway delay of ping pppoe will become high. How should I debug it?

1.) What are your ping times to external IPs 1.1.1.1/8.8.8.8 and internal IP for wired, and wifi both before and after applying SQM?
2.) If ping is slow on wifi, please post config too.

@sppmaster

What is this commit (for IPQ806X) for?
How do we use it on IPQ806X?

I was experimenting with porting IPQ806x over to kernel 6.6 on 11.4 and this was one of the patches that wasn't brought over during the branch switch. Just wanted to have a "universal" base for IPQ8XXX. I have not been successful yet in fully porting it over. Still A LOT needs to be done. Especially due to swconfig dependency.

@sppmaster

On IPQ806X I used below commands in rc.local

for ac in 0 1 2 3; do echo $ac 2000 2000 > /sys/kernel/debug/ieee80211/phy0/aql_txq_limit; done
for ac in 0 1 2 3; do echo $ac 2000 2000 > /sys/kernel/debug/ieee80211/phy1/aql_txq_limit; done

to change the AQL txq limits dynamically.

What commands do you guys use on IPQ807X?

Same commands will work for IPQ807x since debugfs entries in /sys/kernel/debug/ieee80211/ are managed by mac80211 and not tied to a specific driver.

@glassdoor

build fails with GCC 14. @qosmio

@glassdoor

testing NAPI NSS branch. Working well and no errors thus far.

So it fails but works? Or did you have to use GCC 13?

@asvio

I noticed a slight increase in processor usage.
It is really very little (0.3%-0.5%) but this seems to indicate that there is some problem. It seems to be seen that the increase occurs in softirq and in interruption

Yes, this is primarly due to:

date	commit	desc
2024-07-02	07cee3d	qca-nss-drv: Rework smp_affinity logic

Previously I had all IRQs balanced semi-evenly across all cores. However, my approach was not entirely correct. IRQs like nss_empty_buf_sos,nss_empty_buf_queue, and nss_queue0 (which there are 2 sets of, one for each NSS), need to be kept on the same CPU core. While only the remaining nss_queue1,nss_queue2, and nss_queue3 needed to be pinned to a specific CPU core.

With a combination of removing CONFIG_HZ=1000, resetting back to CONFIG_HZ_100, and the smp_affinity changes stability in threaded NAPI mode for qca-nss-drv, ath10k and ath11k improved a lot, with a slight bump in CPU usage.

[JuliusBairaktaris]

Hey thanks for the quick reply. I have the same issue as @glassdoor though that with GCC 13 it builds just fine but with GCC 14 it doesn't. I have taken care of the race condition and will try to build with gcc 14 now

Enable GCC 14

[sppmaster]

@qosmio

Same commands will work for IPQ807x since debugfs entries in /sys/kernel/debug/ieee80211/ are managed by mac80211 and not tied to a specific driver.

Thanks,
By default I have this currently

root@QNAP:~# cat /sys/kernel/debug/ieee80211/phy0/netdev:phy0-ap0/stations/*/aql
Q depth: VO: 0 us VI: 0 us BE: 0 us BK: 0 us
Q limit[low/high]: VO: 5000/12000 VI: 5000/12000 BE: 5000/12000 BK: 5000/12000
Q depth: VO: 0 us VI: 0 us BE: 0 us BK: 0 us
Q limit[low/high]: VO: 5000/12000 VI: 5000/12000 BE: 5000/12000 BK: 5000/12000
Q depth: VO: 0 us VI: 0 us BE: 0 us BK: 0 us
Q limit[low/high]: VO: 5000/12000 VI: 5000/12000 BE: 5000/12000 BK: 5000/12000
Q depth: VO: 0 us VI: 0 us BE: 0 us BK: 0 us
Q limit[low/high]: VO: 5000/12000 VI: 5000/12000 BE: 5000/12000 BK: 5000/12000

I'm testing NAPI threading branch with NSS Autoscale ON and 512MB Memory profile.
Currently after 12 hours runtime not a single error in logs.
Anyone having this pstore message
kern.err kernel: [17.854682] pstore: zlib_inflate() failed, ret = -3!

[eww]

@JuliusBairaktaris @qosmio
GCC 14 on oracle oci, built without problem
NSS-12.5-K6.x-NAPI branch
Linux version 6.6.36 (ubuntu@arm) (aarch64-openwrt-linux-musl-gcc (OpenWrt GCC 14.1.0 r26823-e38bbbed7b)

[littlebeatle-git]

Hi, as Netgear SXR80 + SXS80 is now supported in Openwrt Snapshot. Can you please include them in your build cycle. Im very interested to test NSS with a 5 node batman mesh environment as main seems to be still unstable..
Thx in advance!!

[qosmio]

@littlebeatle-git

Im very interested to test NSS with a 5 node batman mesh

batman mesh is not compatible with NSS wifi offloading. You need to use:
1.) Native native mesh feature of ath11k (802.11s)
2.) qca-nss-wifi-meshmgr with NSS FW 11.4 (not 12.1)
3.) mesh11sd to manage settings that can't be done via ubus alone.

I believe @AgustinLorenzo produces both 12.1 and 11.4 from my NSS repo, not sure if that's still the case.

[pexcn]

1.) What are your ping times to external IPs 1.1.1.1/8.8.8.8 and internal IP for wired, and wifi both before and after applying SQM?
2.) If ping is slow on wifi, please post config too.

Thank you for your reply.

Whether SQM is enabled or disabled, when I am downloading and pinging the external IP at the same time, the delay will become higher.

But I think the wifi configs has nothing to do with it, because I disabled wifi.

This is a screenshot of disabled SQM:

This is a screenshot of enabled SQM:

There seems to be traffic coming into the ifb@wan interface, but the SQM doesn't seem to be working.

[xlighting2017]

it seems, your interface is actually "pppoe-wan" , while in SQM config, the interface is "wan"

[JuliusBairaktaris]

Still getting an error with gcc 14: https://github.com/JuliusBairaktaris/Qualcommax_NSS_Builder/actions/runs/9783723609/job/27012975856

[pexcn]

@xlighting2017

it seems, your interface is actually "pppoe-wan" , while in SQM config, the interface is "wan"

No, the pppoe-wan interface is generated after pppoe dial-up, and its actual interface is wan. And I tried setting it to pppoe-wan, then it will report an error.

[michaelchen644]

Show the log file in /var/run/sqm. And nssqos only works on physical interfaces (such as wan, lan1, eth0). Your config is right.

[pexcn]

I posted the log here, It looks like there are no errors from the logs?

root@OpenWrt:/var/run/sqm# cat wan.start-sqm.log
start-sqm: Log for interface wan: Thu Jul  4 14:31:35 CST 2024

Thu Jul  4 14:31:35 CST 2024: Starting.
Starting SQM script: nss-zk.qos on wan, in: 285000 Kbps, out: 28500 Kbps
fn_exists: function candidate name: sqm_start
fn_exists: TYPE_OUTPUT: sqm_start is a function
fn_exists: return value: 0
Using script specific sqm_start function overriding the generic sqm_start_default.
sqm_start: Starting sqm_start from nss.qos
check_addr: IP on interface wan: 218.20.55.55
sqm_prepare_script: loading required kernel module: qca_nss_qdisc
sqm_prepare_script: loading required kernel module: act_nssmirred
fn_exists: function candidate name: egress
fn_exists: TYPE_OUTPUT: egress is a function
fn_exists: return value: 0
IFACE: wan MTU: 1500
calc_limit: Queue limit should be 234 packets for speed 28500, MTU 1518 and maximum delay 100 ms
add_nsstbl wan: interface:wan speed:28500 burst:1000 mtu:1518 overhead:18
get_burst: 1: 1518, 2: 28500, 3: 1000
get_burst (by duration): BURST [Byte]: 3562, BANDWIDTH [Kbps]: 28500, DURATION [us]: 1000
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan root handle 1: nsstbl rate 28500kbit burst 3036 mtu 1518 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan root handle 1: nsstbl rate 28500kbit burst 3036 mtu 1518 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 1: handle 10: nssprio bands 3 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 1: handle 10: nssprio bands 3 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 10:1 handle 100: nsstbl rate 2850kbit burst 1518 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 10:1 handle 100: nsstbl rate 2850kbit burst 1518 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 100: handle 1000: nsspfifo limit 100 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 100: handle 1000: nsspfifo limit 100 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 10:2 handle 200: nsstbl rate 11400kbit burst 1518 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 10:2 handle 200: nsstbl rate 11400kbit burst 1518 accel_mode 0
get_ECN: ECN CURECN: ecn IECN: ECN EECN: ECN
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 200: handle 2000: nssred limit 100k min 30k max 80k avpkt 1k burst 55 ecn probability 0.50 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 200: handle 2000: nssred limit 100k min 30k max 80k avpkt 1k burst 55 ecn probability 0.50 accel_mode 0
add_nssfq_codel wan: interface:wan speed:28500 target:5ms limit:234 mtu:1518 overhead:18 opts:
add_nssfq_codel wan: No interval specified via advanced option string, going with 100ms
add_nssfq_codel wan: No quantum specified via advanced option string, setting default to 1518 based on an MTU of 1518
Interface speed is less than 100 Mb/s, lowering Codel quantum to 304
get_limit: 234 CURLIMIT: 234
cur_target: 5ms cur_bandwidth: 28500
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan parent 10:3 handle 300: nssfq_codel limit 234 target 5ms flows 1024 quantum 304 interval 100ms set_default accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan parent 10:3 handle 300: nssfq_codel limit 234 target 5ms flows 1024 quantum 304 interval 100ms set_default accel_mode 0
sqm_start: egress shaping activated
fn_exists: function candidate name: ingress
fn_exists: TYPE_OUTPUT: ingress is a function
fn_exists: return value: 0
IFACE: wan MTU: 1500
calc_limit: Queue limit should be 2346 packets for speed 285000, MTU 1518 and maximum delay 100 ms
cmd_wrapper: COMMAND: /sbin/ip link add name ifb@wan type ifb
cmd_wrapper: ip: SUCCESS: /sbin/ip link add name ifb@wan type ifb
cmd_wrapper: COMMAND: /sbin/ip link set dev ifb@wan up
cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev ifb@wan up
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev wan handle ffff: ingress
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev wan handle ffff: ingress
cmd_wrapper: COMMAND: /sbin/tc filter add dev wan parent ffff: protocol all u32 match u32 0 0 action nssmirred redirect dev ifb@wan fromdev wan
cmd_wrapper: tc: SUCCESS: /sbin/tc filter add dev wan parent ffff: protocol all u32 match u32 0 0 action nssmirred redirect dev ifb@wan fromdev wan
add_nsstbl ifb@wan: interface:ifb@wan speed:285000 burst:1000 mtu:1518 overhead:18
get_burst: 1: 1518, 2: 285000, 3: 1000
get_burst (by duration): BURST [Byte]: 35625, BANDWIDTH [Kbps]: 285000, DURATION [us]: 1000
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan root handle 1: nsstbl rate 285000kbit burst 34914 mtu 1518 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan root handle 1: nsstbl rate 285000kbit burst 34914 mtu 1518 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 1: handle 10: nssprio bands 3 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 1: handle 10: nssprio bands 3 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 10:1 handle 100: nsstbl rate 28500kbit burst 3491 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 10:1 handle 100: nsstbl rate 28500kbit burst 3491 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 100: handle 1000: nsspfifo limit 100 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 100: handle 1000: nsspfifo limit 100 accel_mode 0
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 10:2 handle 200: nsstbl rate 114000kbit burst 10474 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 10:2 handle 200: nsstbl rate 114000kbit burst 10474 accel_mode 0
get_ECN: ECN CURECN: ecn IECN: ECN EECN: ECN
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 200: handle 2000: nssred limit 100k min 30k max 80k avpkt 1k burst 55 ecn probability 0.50 accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 200: handle 2000: nssred limit 100k min 30k max 80k avpkt 1k burst 55 ecn probability 0.50 accel_mode 0
add_nssfq_codel ifb@wan: interface:ifb@wan speed:285000 target:5ms limit:2346 mtu:1518 overhead:18 opts:
add_nssfq_codel ifb@wan: No interval specified via advanced option string, going with 100ms
add_nssfq_codel ifb@wan: No quantum specified via advanced option string, setting default to 1518 based on an MTU of 1518
get_limit: 2346 CURLIMIT: 2346
cur_target: 5ms cur_bandwidth: 285000
cmd_wrapper: COMMAND: /sbin/tc qdisc add dev ifb@wan parent 10:3 handle 300: nssfq_codel limit 2346 target 5ms flows 1024 quantum 1518 interval 100ms set_default accel_mode 0
cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb@wan parent 10:3 handle 300: nssfq_codel limit 2346 target 5ms flows 1024 quantum 1518 interval 100ms set_default accel_mode 0
sqm_start: ingress shaping activated
nss-zk.qos was started on wan successfully
root@OpenWrt:/var/run/sqm#

[qosmio]

What's the output of:

nss_stats pppoe
nss_stats pptp

and

ecm_dump.sh

[pexcn]

OK, I dump it, please read those output:

The ecm_dump log pushed here: https://github.com/pexcn/share/blob/49a5145/temp/ecm_dump.log