IPQ806x NSS Drivers

I was also playing a bit with affinity settings:

echo 2 > /proc/irq/33/smp_affinity
echo 1 > /proc/irq/34/smp_affinity
echo 1 > /proc/irq/35/smp_affinity
echo 2 > /proc/irq/36/smp_affinity
echo 2 > /proc/irq/37/smp_affinity
echo 1 > /proc/irq/38/smp_affinity
echo 2 > /proc/irq/47/smp_affinity

Going to check if commenting those lines is going to improve stability. For the LEDs please remember there was a change in the naming recently.

Yeah I know, but the router was not accessible so it was something else.

I had one AP that I had made multiple affinity settings changes and nearly every tweak I could find - it didn’t like the initial flash to NSS several months ago (wifi was messed up, etc)

I found that resetting the config to the defaults and setting the CPU to performance fixed all my troubles. I then put in more reasonable settings and it has been working well ever since.

Is there anything strange in this build that could affect the time based firewall rules? I have a 'mac' based time rule for my kids and I cannot seem to get it to work. I have posted into the help forum but am wondering if it is a problem here.

My iptables problem appears to be a lingering bug in fw3...


Discussed here: time based discussion
Any possibility @ACwifidude could implement this in the next build?

Hope that bug fix makes it to master. I’m keeping my build simple with just the NSS changes. Feel free to clone off my repo and build with that patch included. :sunglasses:

@ACwifidude Cloning your src repro and building it causes a non-bootable image for me.
Not sure why, I included everything that's needed but maybe it's some other conflict with some package I have included, I have wireguard etc. Tried both sysupgrade and factory images.

@ACwifidude Just looked at trying to build this. Seems a bit daunting to me. I cannot figure out how/where I might add this patch doe to me being a noob... I run Arch Linux and have a distccd set up here for building RPi images for armv6/7/8, but this feels a lot different.

Is there any way we can help this fix make it into the master? One reason I went to OpenWrt from ddwrt was b/c this didn't work in ddwrt and it seems it doesn't work here, either. Is the link on ozlabs the 'right way' to get a patch into master? If there is a 'better way'? I can champion it, but would like to test it first.

Here are the steps that worked for me, I used the steps @ACwifidude posted earlier (2-3 weeks back)
I put them into a small script

Before you run the script make sure to update the variables at the beginning of the script
Additionally, make sure to create the file that contains your initial config aka diff-config.
Keep in mind that every option you specify with "=y" will be included into the img file
Options with "=m" will be compiled as package, but will not be part of the image.
Be careful not to fill up the OS space with extra packages, you might brick the router

I have used these steps to build an image with base FW and Lucy
(For my use case I add ipsec, VTI and FRR for BGP routing on a top of this)


#/bin/bash

echo
echo "Defining variables:"

export GH=<My_compile_location_path_here>
export CFG_FILE=<My_default_config_file_name_here>
echo

echo "Doing cleanup"
cd $GH
rm -rf ./openwrt
echo

echo "Setting up the repo for NSS"
echo

git clone -b kernel5.4-nss-qsdk10.0 https://github.com/Ansuel/openwrt.git
cd $GH/openwrt

echo
echo "Copying " $CFG_FILE
cp $GH/$CFG_FILE $GH/openwrt/.config
echo

echo "Doing housekeeping"
echo

make dirclean

echo 
echo "Adding upstream:"
echo

git remote add upstream https://git.openwrt.org/openwrt/openwrt.git
git fetch upstream && git rebase upstream/master

echo
echo " Updating feeds info"
echo
./scripts/feeds update -a

echo
echo " Installing source for generic package feed"
./scripts/feeds install -p packages -a

echo
echo "Pulling feed for luci"
echo

./scripts/feeds install -p luci -a

echo 
echo "Making final config"

cp $GH/$CFG_FILE $GH/openwrt/.config
echo

make defconfig

If you see a menu popping up, just hit save and exit

Once this completed run

cd openwrt
(if you are not already there)

make -j

1 Like

@shelterx - any logs or anything point towards a culprit for the non bootable image?

No, it doesn't respond and is stuck with a blinking white led. I don't have serial access so I don't know... :frowning:

I'm assuming you know how to build OpenWRT, if you want to include the patch you can cherry-pick this commit: https://github.com/facboy/openwrt/commit/643977d31e6d315f8516cb4b7b0028869f7e3f13

This will be my first time. @ACwifidude My first question is what .config do I use. Can I use yours? I do not want to add anything, just fix fw3.
The menuconfig is pretty confusing to me as I know very little and the oldconfig asks a ton of questions that I do not know the answer to.
@Tyco89 I cloned @ACwifidude git and am trying to just build what he has built (baby steps). DO I need to do all that as well?

@keithspg, I think you would need those steps to get an usable FW image with the web interface for management. When I built my initial images, it was lacking without these options.
As for the .config, in this thread @ACwifidude has shared a working dif config. Find it and copy it as your initial .config.

If menuconfig is confusing for you, after you created your initial .config, just run "make defconfig"
This will start with the .config you created previously and add the missing stuff assuming the default values

@Tyco89 I reviewed the post by @ACwifidude and the defconfig in the git repo I cloned from him was the same as what was in the post. I did the 'make defconfig' and got this:

WARNING: Makefile 'package/utils/busybox/Makefile' has a dependency on 'libpam', which does not exist
WARNING: Makefile 'package/utils/busybox/Makefile' has a dependency on 'libpam', which does not exist
WARNING: Makefile 'package/utils/busybox/Makefile' has a build dependency on 'libpam', which does not exist
WARNING: Makefile 'package/boot/kexec-tools/Makefile' has a dependency on 'liblzma', which does not exist
WARNING: Makefile 'package/network/services/lldpd/Makefile' has a dependency on 'libnetsnmp', which does not exist
WARNING: Makefile 'package/utils/policycoreutils/Makefile' has a dependency on 'libpam', which does not exist
WARNING: Makefile 'package/utils/policycoreutils/Makefile' has a dependency on 'libpam', which does not exist
WARNING: Makefile 'package/utils/policycoreutils/Makefile' has a build dependency on 'libpam', which does not exist
WARNING: Makefile 'package/qca/qca-nss-drv/Makefile' has a dependency on 'kmod-qca-nss-dp', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-hyfi-bridge', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-mcs', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-hyfi-bridge', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-mcs', which does not exist
time: target/linux/prereq#0.64#0.01#0.66

Then tried to build it and it gets to here and quits:

 make[3] -C package/network/utils/iw compile
 make[3] -C package/kernel/mac80211 compile
make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make: *** [/home/kgrider/Downloads/openwrt/include/toplevel.mk:236: world] Error 1

Running with make -i1 V=s right now.

I get similar warnings when I run defconfig. Don’t worry too much about those dependencies.

Build process-
I change directory to my build directory and usually clean my directory (old habit)

make dirclean

I have a linux home theater PC that I build with. I rebase with master / update feeds / install feeds / copy my diffconfig from my backup folder like this (change the folder to your “backup” location):

git fetch upstream && git rebase upstream/master && ./scripts/feeds update -a && ./scripts/feeds install -a && cp /home/HTPC/OpenWRT/backup/diffconfig diffconfig && cp diffconfig .config && make defconfig && ./scripts/getver.sh

I have the version number and I’m ready to make after that.

If I get compile errors I run this and let this thread know which patch is the culprit:


make V=s 2>&1 | tee build.log | grep -i -E "^make.*(error|[12345]...Entering dir)"

Let us know if you are missing a file or you spot a bad patch. I’ve been busy this week so I’ll be rebasing this weekend too.

echo 2 > /proc/irq/33/smp_affinity
echo 1 > /proc/irq/34/smp_affinity
echo 1 > /proc/irq/35/smp_affinity
echo 2 > /proc/irq/36/smp_affinity

The NSS cores (33,35) and NSS queues (34,36) should ideally be on separate CPUs.

echo 2 > /proc/irq/37/smp_affinity
echo 1 > /proc/irq/38/smp_affinity

ipq806x based routers use adm_bma, 37 and 38 is for bam_dma which is meant for ipq40xx routers. Switching the affinity won't help.

echo 2 > /proc/irq/47/smp_affinity

If you move 47 (USB 2.0), I'd suggest also moving 48 (USB 3.0) to CPU1

?

fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Tried that string of commands and it failed... I apologize for being such a noob

Easy fix:


git remote add upstream https://git.openwrt.org/openwrt/openwrt.git

This shows the repositories you have added:


git remote -v

Let us know if you have any other issues.

Still unable to build. I wiped everything clean and started again and got a compile error. I ran it again and got this:

#
# configuration written to .config
#
Building backport-include/backport/autoconf.h ... done.
  CC [M]  /home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.o
/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.c: In function 'netif_rx_nss':
/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.c:2561:25: error: 'struct ieee80211_sub_if_data' has no member named 'rx_queue'; did you mean 'skb_queue'?
  skb_queue_tail(&sdata->rx_queue, skb);
                         ^~~~~~~~
                         skb_queue
/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.c:2563:24: error: 'struct ieee80211_sub_if_data' has no member named 'rx_work'; did you mean 'work'?
  schedule_work(&sdata->rx_work);
                        ^~~~~~~
                        work
/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.c: In function 'ieee80211_invoke_fast_rx':
/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.c:4300:32: warning: unused variable 'sdata' [-Wunused-variable]
  struct ieee80211_sub_if_data *sdata = rx->sdata;
                                ^~~~~
make[9]: *** [scripts/Makefile.build:266: /home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211/rx.o] Error 1
make[8]: *** [scripts/Makefile.build:500: /home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/net/mac80211] Error 2
make[7]: *** [Makefile:1729: /home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1] Error 2
make[6]: *** [Makefile.build:13: modules] Error 2
make[5]: *** [Makefile.real:93: modules] Error 2
make[4]: *** [Makefile:121: modules] Error 2
make[4]: Leaving directory '/home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1'
make[3]: *** [Makefile:571: /home/kgrider/Downloads/openwrt/build_dir/target-mips_24kc_musl/linux-ath79_generic/backports-5.8-1/.built] Error 2
make[3]: Leaving directory '/home/kgrider/Downloads/openwrt/package/kernel/mac80211'
time: package/kernel/mac80211/compile#2.12#0.34#2.32
make[2]: *** [package/Makefile:114: package/kernel/mac80211/compile] Error 2
make[2]: Leaving directory '/home/kgrider/Downloads/openwrt'
make[1]: *** [package/Makefile:108: /home/kgrider/Downloads/openwrt/staging_dir/target-mips_24kc_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/kgrider/Downloads/openwrt'
make: *** [/home/kgrider/Downloads/openwrt/include/toplevel.mk:236: world] Error 2
kgrider@grider-desktop:~/Downloads/openwrt$

The commands I ran to get here were:

git clone https://github.com/ACwifidude/openwrt.git
cd openwrt
git remote add upstream https://git.openwrt.org/openwrt/openwrt.git 
git fetch upstream && git rebase upstream/master && ./scripts/feeds update -a && ./scripts/feeds install -a && cp /home/HTPC/OpenWRT/backup/diffconfig diffconfig && cp diffconfig .config && make defconfig && ./scripts/getver.sh

when I get here, I do get this:

...
Installing package 'luci-theme-freifunk-generic' from freifunk
cp: cannot stat '/home/HTPC/OpenWRT/backup/diffconfig': No such file or directory

then I continue.

make dirclean
make menuconfig (to check things and add wireguard)
make -j1 V=s

I am making a noob error for sure.