Ipq806x NSS build (Netgear R7800 / TP-Link C2600 / Linksys EA8500)

@ACwifidude, @quarky ... and anyone else interested in kernel patches to enhance TCP throughput - I read (most of) Aggregating Without Bloating: Hard Times for TCP on Wi-Fi -- and for all I know, some of you may have helped author it (!) BUT it has some recent concrete changes to linux tcp which may invite a look here.

Tested are linux ATH9, ATH10 drivers with linux ipv4 patched to enhance TSQ tunability, and report in their Conclusions section:

We developed and tested a
patch that enables to tune the default TSQ size to address the
issue, which is now included in the Linux kernel mainline.

Given the recent dating (Oct 5, 2022) This looks like some new work in buffer bloat and changing TSQ for increased throughput ('goodput'?) as well as effects on the set of linux traffic control algorithms available.

If nothing else, it looks like an excellent overview of the issues, mechanisms to deal with them as well as concrete research to back everything up. The middle is pretty thick and sciencey. But there are graphs. in color.

But the patch is relatively short and may well warrant inclusion in @ACwifidude's tree as it provides a couple of knobs to turn to get some real improvements.

The whole package (1+Gb) is linked in reference 42 - the patches and pdf are available Doc and TSQ_patch. BBRp_patch.

I should note that 1 change in TSQ.patch is to set Italy as the country for regdb purposes. starts at line 36. Going to skip that one.

I have looked at an openwrt master kernel for ramips_mt7621 but they aren't there yet; the authors applied them against kernel 5.4 for their testing.

2 Likes

As an update, I'm trying to integrate the patch into my (@ACwifidude's) 22.03 kernel 5.10 tree but it's not a clean matchup -- describing the new data to existing TCP has changed.

I've emailed one of the article's authors as to where in the mainline kernel this lives.

I obviously haven't dealt with mainline kernels at the patch level so if anyone has suggestions for git trees to examine I'd appreciate them.

M.

  Hi people just a heads up about the reboots. OpenWrt master is making my r7800 reboot with out of the NSS patches. I was just using stock master with Kernel 5.15 and addblock luci plus SQM.
1 Like

same, i got the same packages installed too, that’s all

1 Like

@tohojo is listed as one of the author, so I believe this should make it's way into OpenWrt sooner or later.

At first glance, this is way over my head tbh :stuck_out_tongue: I'm still trying to figure out the ath10k flow ... haha.

PSA: @ACwifidude Not sure if you're aware that the QCA repository in the Code Aurora Git repository will no longer be accessible from 1 Apr 2023. They have moved to Code Linaro Git repository. New clones of the NSS branch that uses the Code Aurora Git repo will not be able to get the source codes from 1 Apr 2023.

The URLs in the Makefiles will have to point to the Code Linaro Git repo.

3 Likes

@ACwifidude These should already be updated in my nss repo.

4 Likes

Took a stab at modifying the patch for 5.15. I was able to build and boot with it. Haven't done any extensive test yet. Not sure why they hardcoded Italy in regd.c vs. setting it using regulatory.db

master branch does not include this patch.
@Ansuel
Is it necessary for stability?

1 Like

Lots of interesting stuff lately.
@tapper and @Pmtroutok do you use ondemand scheduler. Have you tried performance governor just to check if there are any reboots.

it got backported to 5.15 and present upstream

2 Likes

I mistake. I had overlooked it being included in the master branch.
I think there might be something in clk-krait.c, so
invesitigated related patches.

I've done the same - but changing the 2 key tcp_limit params in /proc hasn't changed thruput one bit.

I'm going to drill down into what's in the way, but that will take some time.

I'm also thinking about moving this work over to a dual-core MT7921 router with pathologicaly poor 5ghz performance (netgear AC2400, R6700v2) - there's less going on for that target and the poor performance has been explained away as a weak cpu.

It's also a spare router i can setup as a test platform ...

ADDING an afterthought - I'm wondering if calls to this area of tcp have been patched around by openwrt in favor of other pacing/aggregation algorithms.

results this un-flaky and regular have to be coded!

1 Like

hello what is the latest status for ZyXEL NBG6817 ? Can I use it ?

Hi, errors on qca-nss-drv compiling:

make[3]: Entering directory '/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.15.76'
  CC [M]  /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_cmn.o
  CC [M]  /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.o
  CC [M]  /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_coredump.o
  CC [M]  /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_drv_stats.o
In file included from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_coredump.c:24:
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h: In function 'nss_core_dma_cache_maint':
In file included from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.c:22:
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h: In function 'nss_core_dma_cache_maint':
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:116:17: error: implicit declaration of function 'dmac_inv_range'; did you mean 'outer_inv_range'? [-Werror=implicit-function-declaration]
  116 |                 dmac_inv_range(start, start + size);
      |                 ^~~~~~~~~~~~~~
      |                 outer_inv_range
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:116:17: error: implicit declaration of function 'dmac_inv_range'; did you mean 'outer_inv_range'? [-Werror=implicit-function-declaration]
  116 |                 dmac_inv_range(start, start + size);
      |                 ^~~~~~~~~~~~~~
      |                 outer_inv_range
In file included from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_hal/include/nss_hal.h:26,
                 from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_tx_rx_common.h:25,
                 from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_cmn.c:26:
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h: In function 'nss_core_dma_cache_maint':
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:116:17: error: implicit declaration of function 'dmac_inv_range'; did you mean 'outer_inv_range'? [-Werror=implicit-function-declaration]
  116 |                 dmac_inv_range(start, start + size);
      |                 ^~~~~~~~~~~~~~
      |                 outer_inv_range
In file included from /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_drv_stats.c:17:
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h: In function 'nss_core_dma_cache_maint':
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:119:17: error: implicit declaration of function 'dmac_clean_range'; did you mean 'dmac_flush_range'? [-Werror=implicit-function-declaration]
  119 |                 dmac_clean_range(start, start + size);
      |                 ^~~~~~~~~~~~~~~~
      |                 dmac_flush_range
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:116:17: error: implicit declaration of function 'dmac_inv_range'; did you mean 'outer_inv_range'? [-Werror=implicit-function-declaration]
  116 |                 dmac_inv_range(start, start + size);
      |                 ^~~~~~~~~~~~~~
      |                 outer_inv_range
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:119:17: error: implicit declaration of function 'dmac_clean_range'; did you mean 'dmac_flush_range'? [-Werror=implicit-function-declaration]
  119 |                 dmac_clean_range(start, start + size);
      |                 ^~~~~~~~~~~~~~~~
      |                 dmac_flush_range
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:119:17: error: implicit declaration of function 'dmac_clean_range'; did you mean 'dmac_flush_range'? [-Werror=implicit-function-declaration]
  119 |                 dmac_clean_range(start, start + size);
      |                 ^~~~~~~~~~~~~~~~
      |                 dmac_flush_range
/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.h:119:17: error: implicit declaration of function 'dmac_clean_range'; did you mean 'dmac_flush_range'? [-Werror=implicit-function-declaration]
  119 |                 dmac_clean_range(start, start + size);
      |                 ^~~~~~~~~~~~~~~~
      |                 dmac_flush_range
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:289: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_coredump.o] Error 1
make[4]: *** Waiting for unfinished jobs....
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:289: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_cmn.o] Error 1
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:289: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_drv_stats.o] Error 1
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:289: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/nss_core.o] Error 1
make[3]: *** [Makefile:1901: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43] Error 2
make[3]: Leaving directory '/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.15.76'
make[2]: *** [Makefile:288: /home/ubuntu/Desktop/ipq806x-kernel515-ac2203/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/qca-nss-drv-2020-03-20-3cfb9f43/.built] Error 2
make[2]: Leaving directory '/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/feeds/nss/qca-nss-drv'
time: package/feeds/nss/qca-nss-drv/compile#5.15#0.55#2.72
    ERROR: package/feeds/nss/qca-nss-drv failed to build.
make[1]: *** [package/Makefile:116: package/feeds/nss/qca-nss-drv/compile] Error 1
make[1]: Leaving directory '/home/ubuntu/Desktop/ipq806x-kernel515-ac2203'
make: *** [/home/ubuntu/Desktop/ipq806x-kernel515-ac2203/include/toplevel.mk:231: package/feeds/nss/qca-nss-drv/compile] Error 2
ubuntu@ubuntu:~/Desktop/ipq806x-kernel515-ac2203$ 

kernel 5.15 branch, default configs. How to solve this? Thanks.

1 Like

full branch 5.15 and run:

./scripts/feeds update -a && ./scripts/feeds install -a && cp diffconfig .config && make defconfig && ./scripts/getver.sh

make -j5
1 Like

Added the missing patch on my repo.

Built an image for just the r7800 with the 5.15 kernel and qsdk11 if anyone wants to try it. I’m testing it and found my main router (wifi turned off) seems to work just fine. My two dumb AP r7800s did not like it and would reboot after just a couple minutes. Haven’t trouble shot it yet.

organized all the commits on top of master for simplicity (master from about 4 days ago):

R7800 test images here. Seem to still have a bug (see above) but they post and work well on one of my devices:

4 Likes

sorry, I mean the full 5.15 branch from AC repo. What diff is your repo? Thanks.

I am currently on the 5.15-qsdk11 branch of qosmio with no changes. I've no repo.
The account i made on github was to test things but it is useless at present. You should not take it into account if you are referring to that repo.