Qualcommax NSS Build

McGiverGim · February 17, 2023, 3:16pm

I'm trying to build myself an image, adding my own packages. But Is too overhelming for a novice...
I can see the menuconfig, but I'm totally unsure about what selecting and what not. Too much options and I don't want to break anything.

Maybe someone can make a guide, step by step, about what to select to create an image with the basic (luci and NSS)? I suppose that after that, adding more packages will be easy. Or if someone can simply paste this default .config will be great.

And a question, after booting into the router, we need to add some extra configuration to "start" the nss driver, or it will be on by default? I've seen some strange commands in the thread that I don't understand.

Thanks!

juliank · February 18, 2023, 3:08am

Bitthief posted his .config file in this thread that you can use when compiling:

rmandrad · February 18, 2023, 8:46am

has anyone noticed that the nss core1 doesn't get any ipv4 traffic?

also only the core1 nss_queue0 interrupt has any counters over 0

admitedly it could be that I am running nss on the qnap ? with a nss dts designed for the ax3600 ... or it could be by design that the core 1 is doing some kind of buffering mgmt

do you see any of this behaviour on the ax3600?

mattimat · February 18, 2023, 8:48am

I'm getting errors in builds even when using .config file from bitthief. I managed to built couple of images about a week ago, after that problems. I have started from scratch in new directory.

ERROR: modpost: "nss_c2c_rx_stats_register_notifier" [/home/matti/nss/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/netlink/qca-nss-netlink.ko] undefined!

Oops, just a couple of posts before mine there is a similar error with solution. Sorry

mattimat · February 18, 2023, 9:23am

                        NSS STATS
Stats for core 0
Node         RX Pkts       TX Pkts       Drops     Exceptions
----         -------       -------       -----     ----------
eth_rx       6,675,593     6,675,593     0         14,098
ipv4         6,675,288     0             0         6,675,288
ipv6         305           0             0         305
lso_rx       652,067       2,965,127     0
n2h          8,243,699     3,926,618     0

Stats for core 1
Node         RX Pkts       TX Pkts       Drops     Exceptions
----         -------       -------       -----     ----------
n2h          1,034,079     58            0
 buf_cmd     0             140
 buf_emty    3,895,879     79,843
 buf_pkt     6,689,691     3,926,536
 status_sync 0

rmandrad · February 18, 2023, 1:54pm

thank you

it seems that the core1 nss dts definition is missing most of the "enabled" parameters of core0 like "qcom,ipv4-enabled" .... interesting ...

i copied all of the core0 enabled value to core1 ... will report back with findings later

rmandrad · February 18, 2023, 1:57pm

have you pulled the latest from bitthief's repo ? i remember this was an issue sometime ago and it is now fixed ... it compiles/links fine for me

  LD [M]  /homebitthief-ipq807x-qca/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/netlink/qca-nss-netlink.ko
rstrip.sh: /homebitthief-ipq807x-qca/build_dir/target-aarch64_cortex-a53_musl/linux-ipq807x_generic/qca-nss-clients-2022-03-18-a93764b6/ipkg-aarch64_cortex-a53/kmod-qca-nss-drv-netlink/lib/modules/5.15.93/qca-nss-netlink.ko: relocatable

mattimat · February 18, 2023, 2:16pm

I just started from the scratch by cloning the repo. Now I can see the netlink in menuconfig as an selectable option.

rmandrad · February 18, 2023, 2:53pm

nope ... it doesn't make any change .. core 0 seems to process traffic core 1 buffering

kirdes · February 18, 2023, 7:18pm

core 1 is crypto only.
core 0 is the npu core

McGiverGim · February 19, 2023, 10:09am

@bitthief can you rebase your branch against openwrt master? I'm having some strange errors while building your branch that I don't have with master.
There is a new kernel version too.
Thanks!

EDIT: I've just seen that @bitthief just rebased it yesterday. I will test the new code. Thanks!

mattimat · February 19, 2023, 12:23pm

Same situation, learning by bricking (almost). I managed to create almost functioning build for AX9000 with bitthiefs .config. Today I try to refine the build by adding the QCN6024/9024/9074 radio and stripping not needed packages and applications. The basic nss drivers start automaticallly imo

McGiverGim · February 19, 2023, 12:36pm

I'm trying to generate a simply GitHub action, similar you the one that robimarko used on his builds. In this way is easier to customize it adding some packages and github will simply build it. And I don't need to mess with a big menu full of options. I will report if I get to something.

michaelchen644 · February 19, 2023, 2:04pm

I saw a new commit in bitthief's repositoriy today, and now I have completed the compilation, but a warning about the SHA512 Bug appeared before the compilation ended, which should be fine. I can't wait to try it!
The new commit adds the compile option of kmod-qca-nss-drv, if there are errors about kmod-qca-nss-drv and qca-nss-client when compiling, please consider this. Here is my choice, for your reference.

McGiverGim · February 19, 2023, 3:10pm

I think I'm far from there. I'm trying first to build a version without any of the NSS modules. The default packages adding only luci and full wpad. With master OpenWrt this compiles. With bitthief branch it fails.

I will keep investigating.

doveki · February 19, 2023, 9:17pm

Can you add nss main packages to ipq807x/Makefile?
I don't know the meaning of the most nss packages.
thank you!

adamhnat · February 19, 2023, 10:35pm

I was able to successfully compile using yesterday's sources, and using this config: https://pastebin.com/ZEBjmnam
During tests I've found out (all default settings) :

extra network is propagated (don't know where it came from) 169.254.0.0/16 trough all interfaces
also temporary IP is not released from wan: (fibre to ethernet converter):

ip address show wan

#ip a s wan
3: wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 85.ZZ.XX.AA/24 brd 85.ZZ.XX.255 scope global wan
       valid_lft forever preferred_lft forever
    inet 85.ZZ.XX.BB/24 brd 85.ZZ.XX.255 scope global secondary dynamic noprefixroute wan
       valid_lft 1271sec preferred_lft 1046sec
    inet6 fe80::e762:aba5:8147:4cb6/64 scope link
       valid_lft forever preferred_lft forever

during that -> no internet access at all until lifetime won't expire

when connect to router via wireless 5GHz and 2,4GHz on intel 8265, I've noticed that ssh connection is frequently dropped same as WiFi
latency (ICMP) increased for almost 2ms on WiFi and 1ms on wired.
when bring up wireguard interface -> all connection from lan to internet gone.
When distance from router is increasing count of dropped/wrong packet is increased - behind 2 walls - speed is going very low (without nss it goes from 300Mbit to 180Mbit, with NSS to 2-3Mbit)

I'm pretty sure that I must do something wrong.. as above doesn't exist on clean build without NSS.

McGiverGim · February 20, 2023, 7:10am

Maybe I need some extra command to build from bitthief repo? This is what I'm using:

./scripts/feeds update -a
./scripts/feeds install -a

# -->Generate the .config here. First vanilla with luci and full wpad only. No NSS included.
echo "CONFIG_TARGET_ipq807x=y" > .config
echo "CONFIG_TARGET_ipq807x_generic=y\n" >> .config
echo "CONFIG_TARGET_MULTI_PROFILE=y\n" >> .config
echo "CONFIG_TARGET_ALL_PROFILES=y\n" >> .config
echo "CONFIG_TARGET_PER_DEVICE_ROOTFS=y\n" >> .config
echo "CONFIG_TARGET_ROOTFS_INITRAMFS=y\n" >> .config
echo "CONFIG_ALL_NONSHARED=y\n" >> .config
echo "CONFIG_PACKAGE_luci=y\n" >> .config
echo "CONFIG_PACKAGE_wpad-basic-mbedtls=m\n" >> .config
echo "CONFIG_PACKAGE_wpad-mbedtls=y\n" >> .config

make defconfig
make download V=s

make tools/install -j$(nproc) V=s || \
make tools/install V=s

make toolchain/install -j$(nproc) V=s || \
make toolchain/install V=s

make -j$(nproc) V=s || \
make V=s

As I say, are the same commands were used by robi to build his images, and with master OpenWrt this works. The error is here if someone can look at it to see what is wrong: https://github.com/McGiverGim/openwrt/actions/runs/4216106371/jobs/7317704588

mattimat · February 20, 2023, 8:37am

When i build my nss I use the -f parameter with feeds. It reports to update some nss feeds. Just a wild goose hunt.

McGiverGim · February 20, 2023, 9:15am

I didn't know about the -f parameter. I think this can be useful on local build, but I'm building on a Github Action, with a new Virtual Machine each time, so I don't think this can't make a difference.

I think I've found the reason why I can't build and others have no problem. I was building ALL the IPQ807x targets, and others only build one target, for example the AX3600. The error is being produced compiling some bluetooth drivers, I think remember that some IPQ807x target has added bluetooth support recently. So I modified:

echo "CONFIG_TARGET_ipq807x=y" > .config
echo "CONFIG_TARGET_ipq807x_generic=y\n" >> .config

by:

echo "CONFIG_TARGET_ipq807x_generic=y\n" > .config
echo "CONFIG_TARGET_ipq807x_generic_DEVICE_xiaomi_ax3600=y\n" >> .config

and seems that now is building ok.

@bitthief are you aware of this? do you build for all targets? OpenWrt master seems to build for all of them, but your branch fails if I try to build for all. Building for AX3600 for example seems ok.

When I get to something, I can push a PR to @bitthief repo with the Github Action that builds for all and in this way you will have a way to check it at least, if interested.

EDIT: the build has failed again, a lot later than before, but has failed.