IPQ806x NSS Drivers

If you are running the NSS builds, packet steering doesn’t really help as the Krait CPUs are not used in routing, apart from the initial connection setup.

1 Like

Thanks quarky. I remember reading some of your old posts about packet steering would not be applicable for NSS, and that's why I tried to disable it and the disabling cleared the errors.

Many thanks for starting this awesome NSS initiative for OpenWrt in the first place.

Best regards,
Eric

2 Likes

I have an Asus BRT-AC828 router and I'm currently working my way through this thread as I believe it should be applicable to my router as well.
But before I do read it all, can someone confirm that it indeed should be relevant for my router too?

Regardless of the answer, thanks for initiatives like these :slight_smile:

I would suggest before you attempt NSS acceleration, it would be better to get the current master branch of OpenWrt running on it, as this device is not listed as supported by OpenWrt at the moment.

This looks very similar to the R7800, with the exception of the switch (there are 2 of them) and maybe the flash?

If you can get OpenWrt running correctly, it should be trivial (I hope) to get NSS working for it.

Edit: Hopefully the QCA8033 chip is similar enough to the QCA8327 chip for the qca-nss-gmac driver to recognise it and take over. I think that is the only hurdle to get NSS acceleration running. Probably QCA's source repository will have the proper drivers.

1 Like

I'm quite a bit removed from even trying OpenWrt on it as I have no prior experience with OpenWrt and I only have 1 router, so I'm dependent on it's proper working

I'm currently (just) gathering info on my device (and documenting it), figuring out how to attach an UART cable (as I'm pretty sure it has an UART), ways to (potentially) unbrick it if needed and putting Asus' code in git gradually (so no huge 'initial commit' in 1 dump) and with some (preliminary) analyses.

Reading through this topic helps me get a better understanding of things and hopefully will allow me to better document it and have a better/clear picture of the 'before' (OpenWrt) situation.

I saw that chip mentioned before but couldn't find it in my (current) documentation, but then I saw Atheros AR8033-AL1A, so that must be it :slight_smile:

Which repo to look at is a big mystery to me as there are so many. I'm now ~ post 500 and I did see some references to NSS.AK... things and that strings could help with that.
When I found a folder named qca-nss-fw I figured I was at the right place:

strings retail_router0.bin
Version: NSS.AK.1.0.c10-00024-R-1.59374.1.69702.1

What to make of this? The other version strings seem to have NSS.AK.[F|G|H|etc].* where F was QSDK 8.x (IIUC), but mine didn't have a letter, but a number ...

This thread will likely confuse you if your objective is to get OpenWrt running for your Asus router. I would suggest to go thru the Netgear R7800 exploration (IPQ8065, QCA9984) thread. It will be more relevant as your Asus is very similar to the R7800.

1 Like

You mean as in this goes too deep as a starting point?

I know I have a LOT to learn and I'm fine with that; I see this as a long term project. I may not understand everything fully what I've read so far in this thread (now), but I've already picked up several useful bits. Usually when I've learned more, pieces that I didn't (fully) understand before, will fall into place later.

I have several objectives. One is getting a more secure router by amongst other things run a (more) recent and still supported kernel (instead of 3.4.103). Another is showing the power of Open Source and OpenWrt seems like the best project to do that with (when it comes to routers). My assumption is that I'd also need IPQ806x NSS drivers to get good performance, which I of course want (if possible).
If I can help get my router into a supported state with OpenWrt, that would be really cool :slight_smile:

1 Like

After you get your device working with OpenWRT feel free to evaluate one of my repos (or clone them) if you want to try NSS for your device.

1 Like

Not really. It doesn’t contain information on how to get an ipq806x router running OpenWrt. It assumes OpenWrt is already running properly before attempting NSS. So applying the knowledge in this thread on a router without OpenWrt running will likely be confusing.

1 Like

Hi guys,

I've been following this even though I'm from a different party called IPQ807x, and I have a doubt and maybe you guys can enlighten me.
Do all these offloading packages have support for nftables?
Since openwrt is migrating away from iptables for good, is there a way to make this work with nftables? I've seen most of the dependencies have to do with conntrack which is the same for iptables and nftables, but it also has some dependencies on iptables-mox-extra, that should be replaceable with nftables right?

Also, for the TC offloading, could there be a way in the future of adding cake offloading? Is this TC offloading currently transparent? Or is there a need to use a special nss tool?

Thanks!

@ACwifidude
got some update or what is the best / most stable release for now?

thanks!

I updated the 21.02, 22.03, and master builds on April 15th. If you are looking for a recent stable build I’d load the 22.03 build. :sunglasses:

1 Like

thanks but is there anything specific behind the fact that still version 21x and 22x are kept?

I'm still on 21.02.1 with my R7800 as I don't want to run into this 5GHz wifi stalling issue that was likely introduced in 21.02.2.

I don't get the sense it was solved in 21.02.3 either. Or am I wrong and has it been fixed now? Should I wait for 22.03?

I realize it's quite an old post, but I didn't see someone reporting who was running OEM firmware ... and I am. It's an Asus BRT-AC828 (unsupported by OpenWrt (currently)) running asuswrt (although that isn't relevant afaict).

Firmware version (from IPQ806x NSS Drivers - #2603 by diederik):

strings retail_router0.bin
Version: NSS.AK.1.0.c10-00024-R-1.59374.1.69702.1
root@BRT-AC828:/tmp/home/root# cat /sys/kernel/debug/qca-nss-drv/stats/wifi
wifi stats start:

WIFI ID: 0
RX_PACKETS = 0
RX_DROPPED = 0
TX_PACKETS = 0
TX_DROPPED = 0
TX_TRANSMIT_COMPLETED = 0
TX_MGMT_RECEIVED = 0
TX_MGMT_TRANSMITTED = 0
TX_MGMT_DROPPED = 0
TX_MGMT_COMPLETED = 0
TX_INV_PEER_ENQ_CNT = 0
RX_INV_PEER_RCV_CNT = 0
RX_PN_CHECK_FAILED = 0
RX_PKTS_DELIVERD = 0
RX_BYTES_DELIVERED = 0
TX_BYTES_COMPLETED = 0
RX_DELIVER_UNALIGNED_DROP_CNT = 0
TIDQ_ENQUEUE_CNT_0 = 0
TIDQ_ENQUEUE_CNT_1 = 0
TIDQ_ENQUEUE_CNT_2 = 0
TIDQ_ENQUEUE_CNT_3 = 0
TIDQ_ENQUEUE_CNT_4 = 0
TIDQ_ENQUEUE_CNT_5 = 0
TIDQ_ENQUEUE_CNT_6 = 0
TIDQ_ENQUEUE_CNT_7 = 0
TIDQ_DEQUEUE_CNT_0 = 0
TIDQ_DEQUEUE_CNT_1 = 0
TIDQ_DEQUEUE_CNT_2 = 0
TIDQ_DEQUEUE_CNT_3 = 0
TIDQ_DEQUEUE_CNT_4 = 0
TIDQ_DEQUEUE_CNT_5 = 0
TIDQ_DEQUEUE_CNT_6 = 0
TIDQ_DEQUEUE_CNT_7 = 0
TIDQ_ENQUEUE_FAIL_CNT_0 = 0
TIDQ_ENQUEUE_FAIL_CNT_1 = 0
TIDQ_ENQUEUE_FAIL_CNT_2 = 0
TIDQ_ENQUEUE_FAIL_CNT_3 = 0
TIDQ_ENQUEUE_FAIL_CNT_4 = 0
TIDQ_ENQUEUE_FAIL_CNT_5 = 0
TIDQ_ENQUEUE_FAIL_CNT_6 = 0
TIDQ_ENQUEUE_FAIL_CNT_7 = 0
TIDQ_TTL_EXPIRE_CNT_0 = 0
TIDQ_TTL_EXPIRE_CNT_1 = 0
TIDQ_TTL_EXPIRE_CNT_2 = 0
TIDQ_TTL_EXPIRE_CNT_3 = 0
TIDQ_TTL_EXPIRE_CNT_4 = 0
TIDQ_TTL_EXPIRE_CNT_5 = 0
TIDQ_TTL_EXPIRE_CNT_6 = 0
TIDQ_TTL_EXPIRE_CNT_7 = 0
TIDQ_DEQUEUE_REQ_CNT_0 = 0
TIDQ_DEQUEUE_REQ_CNT_1 = 0
TIDQ_DEQUEUE_REQ_CNT_2 = 0
TIDQ_DEQUEUE_REQ_CNT_3 = 0
TIDQ_DEQUEUE_REQ_CNT_4 = 0
TIDQ_DEQUEUE_REQ_CNT_5 = 0
TIDQ_DEQUEUE_REQ_CNT_6 = 0
TIDQ_DEQUEUE_REQ_CNT_7 = 0
TOTAL_TIDQ_DEPTH = 0
RX_HTT_FETCH_CNT = 0
TOTAL_TIDQ_BYPASS_CNT = 0
GLOBAL_Q_FULL_CNT = 0
TIDQ_FULL_CNT = 0

WIFI ID: 1
RX_PACKETS = 0
RX_DROPPED = 0
TX_PACKETS = 0
TX_DROPPED = 0
TX_TRANSMIT_COMPLETED = 0
TX_MGMT_RECEIVED = 0
TX_MGMT_TRANSMITTED = 0
TX_MGMT_DROPPED = 0
TX_MGMT_COMPLETED = 0
TX_INV_PEER_ENQ_CNT = 0
RX_INV_PEER_RCV_CNT = 0
RX_PN_CHECK_FAILED = 0
RX_PKTS_DELIVERD = 0
RX_BYTES_DELIVERED = 0
TX_BYTES_COMPLETED = 0
RX_DELIVER_UNALIGNED_DROP_CNT = 0
TIDQ_ENQUEUE_CNT_0 = 0
TIDQ_ENQUEUE_CNT_1 = 0
TIDQ_ENQUEUE_CNT_2 = 0
TIDQ_ENQUEUE_CNT_3 = 0
TIDQ_ENQUEUE_CNT_4 = 0
TIDQ_ENQUEUE_CNT_5 = 0
TIDQ_ENQUEUE_CNT_6 = 0
TIDQ_ENQUEUE_CNT_7 = 0
TIDQ_DEQUEUE_CNT_0 = 0
TIDQ_DEQUEUE_CNT_1 = 0
TIDQ_DEQUEUE_CNT_2 = 0
TIDQ_DEQUEUE_CNT_3 = 0
TIDQ_DEQUEUE_CNT_4 = 0
TIDQ_DEQUEUE_CNT_5 = 0
TIDQ_DEQUEUE_CNT_6 = 0
TIDQ_DEQUEUE_CNT_7 = 0
TIDQ_ENQUEUE_FAIL_CNT_0 = 0
TIDQ_ENQUEUE_FAIL_CNT_1 = 0
TIDQ_ENQUEUE_FAIL_CNT_2 = 0
TIDQ_ENQUEUE_FAIL_CNT_3 = 0
TIDQ_ENQUEUE_FAIL_CNT_4 = 0
TIDQ_ENQUEUE_FAIL_CNT_5 = 0
TIDQ_ENQUEUE_FAIL_CNT_6 = 0
TIDQ_ENQUEUE_FAIL_CNT_7 = 0
TIDQ_TTL_EXPIRE_CNT_0 = 0
TIDQ_TTL_EXPIRE_CNT_1 = 0
TIDQ_TTL_EXPIRE_CNT_2 = 0
TIDQ_TTL_EXPIRE_CNT_3 = 0
TIDQ_TTL_EXPIRE_CNT_4 = 0
TIDQ_TTL_EXPIRE_CNT_5 = 0
TIDQ_TTL_EXPIRE_CNT_6 = 0
TIDQ_TTL_EXPIRE_CNT_7 = 0
TIDQ_DEQUEUE_REQ_CNT_0 = 0
TIDQ_DEQUEUE_REQ_CNT_1 = 0
TIDQ_DEQUEUE_REQ_CNT_2 = 0
TIDQ_DEQUEUE_REQ_CNT_3 = 0
TIDQ_DEQUEUE_REQ_CNT_4 = 0
TIDQ_DEQUEUE_REQ_CNT_5 = 0
TIDQ_DEQUEUE_REQ_CNT_6 = 0
TIDQ_DEQUEUE_REQ_CNT_7 = 0
TOTAL_TIDQ_DEPTH = 0
RX_HTT_FETCH_CNT = 0
TOTAL_TIDQ_BYPASS_CNT = 0
GLOBAL_Q_FULL_CNT = 0
TIDQ_FULL_CNT = 0

WIFI ID: 2
RX_PACKETS = 0
RX_DROPPED = 0
TX_PACKETS = 0
TX_DROPPED = 0
TX_TRANSMIT_COMPLETED = 0
TX_MGMT_RECEIVED = 0
TX_MGMT_TRANSMITTED = 0
TX_MGMT_DROPPED = 0
TX_MGMT_COMPLETED = 0
TX_INV_PEER_ENQ_CNT = 0
RX_INV_PEER_RCV_CNT = 0
RX_PN_CHECK_FAILED = 0
RX_PKTS_DELIVERD = 0
RX_BYTES_DELIVERED = 0
TX_BYTES_COMPLETED = 0
RX_DELIVER_UNALIGNED_DROP_CNT = 0
TIDQ_ENQUEUE_CNT_0 = 0
TIDQ_ENQUEUE_CNT_1 = 0
TIDQ_ENQUEUE_CNT_2 = 0
TIDQ_ENQUEUE_CNT_3 = 0
TIDQ_ENQUEUE_CNT_4 = 0
TIDQ_ENQUEUE_CNT_5 = 0
TIDQ_ENQUEUE_CNT_6 = 0
TIDQ_ENQUEUE_CNT_7 = 0
TIDQ_DEQUEUE_CNT_0 = 0
TIDQ_DEQUEUE_CNT_1 = 0
TIDQ_DEQUEUE_CNT_2 = 0
TIDQ_DEQUEUE_CNT_3 = 0
TIDQ_DEQUEUE_CNT_4 = 0
TIDQ_DEQUEUE_CNT_5 = 0
TIDQ_DEQUEUE_CNT_6 = 0
TIDQ_DEQUEUE_CNT_7 = 0
TIDQ_ENQUEUE_FAIL_CNT_0 = 0
TIDQ_ENQUEUE_FAIL_CNT_1 = 0
TIDQ_ENQUEUE_FAIL_CNT_2 = 0
TIDQ_ENQUEUE_FAIL_CNT_3 = 0
TIDQ_ENQUEUE_FAIL_CNT_4 = 0
TIDQ_ENQUEUE_FAIL_CNT_5 = 0
TIDQ_ENQUEUE_FAIL_CNT_6 = 0
TIDQ_ENQUEUE_FAIL_CNT_7 = 0
TIDQ_TTL_EXPIRE_CNT_0 = 0
TIDQ_TTL_EXPIRE_CNT_1 = 0
TIDQ_TTL_EXPIRE_CNT_2 = 0
TIDQ_TTL_EXPIRE_CNT_3 = 0
TIDQ_TTL_EXPIRE_CNT_4 = 0
TIDQ_TTL_EXPIRE_CNT_5 = 0
TIDQ_TTL_EXPIRE_CNT_6 = 0
TIDQ_TTL_EXPIRE_CNT_7 = 0
TIDQ_DEQUEUE_REQ_CNT_0 = 0
TIDQ_DEQUEUE_REQ_CNT_1 = 0
TIDQ_DEQUEUE_REQ_CNT_2 = 0
TIDQ_DEQUEUE_REQ_CNT_3 = 0
TIDQ_DEQUEUE_REQ_CNT_4 = 0
TIDQ_DEQUEUE_REQ_CNT_5 = 0
TIDQ_DEQUEUE_REQ_CNT_6 = 0
TIDQ_DEQUEUE_REQ_CNT_7 = 0
TOTAL_TIDQ_DEPTH = 0
RX_HTT_FETCH_CNT = 0
TOTAL_TIDQ_BYPASS_CNT = 0
GLOBAL_Q_FULL_CNT = 0
TIDQ_FULL_CNT = 0


wifi stats end

So that's the same as @ACwifidude's output.
Some other data:

root@BRT-AC828:/sys/kernel/debug# uptime
 16:38:33 up 60 days,  2:19,  load average: 3.16, 3.08, 3.06
root@BRT-AC828:/sys/kernel/debug# uname -a
Linux BRT-AC828 3.4.103 #1 SMP PREEMPT Wed Apr 8 11:19:22 CST 2020 armv7l ASUSWRT

HTH

1 Like

Regarding the spinlock error when running br-lan in promisc mode.
I checked the boot log from the OEM firmware, it seems like the bridge (actually all interfaces) is set to promisc mode. So it's a default setting in OEM FW. But for some reason it doesn't work in the OpenWrt NSS implementation/port.

Well, the NSS ECM driver used in the stock firmware is very old, I think QSDK 6.x. Also, IIRC, the Linux kernel is v3.x in the stock firmware.

The drivers we're using for 21.02/master is at least QSDK 10.x on Linux v.5.4.x. I wouldn't be surprised if there're bugs introduced since then.

1 Like

I have try your latest 22.03 build but encounter random disconnect from router. Is it nss driver currently unstable on kernel 5.10?

ath10 devices are working out a bug with wifi. Hopefully will have a fix here shortly. Thankfully it is not NSS related.

hi, @ACwifidude
is there currently stable / wifi bug free image for r7800? if so which image for ath10k or ath10k-ct?

thanks