IPQ806x NSS Drivers

These are DTS configuration to allow the qca-nss-gmac driver to probe and attached itself to drive the GMAC ports. The standard OpenWRT repo comes with the stmmac driver, which had to be disable. The NSS core drivers depends on the qca-nss-gmac driver so the stmmac driver have to be prevented from successfully probing and take over the GMAC port.

My repo tries to enable support for both drivers with the DTS configuration, relying manually disabling the stmmac driver via kernel build configuration. Actually this is not necessary. Changing the DTS files to force the use of the qca-nss-gmac driver is correct.

But having the NSS drivers to work now which is good. But does flow offloading and QOS work with the NSS cores?

From what I've read so far, yes, the NSS cores works with qdisc of the Linux kernel.

Does your patch include 18.06 flow offloading and hw offloading?

-- EDIT --
Patiently waiting for slh to push his message :stuck_out_tongue: i think he has the same router as i have with a lot more knowledge :stuck_out_tongue:

You don't quite understand how the NSS cores work, with qca-nss-gmac being at work, all the traffic (at least the parts understood by the NSS firmware) is offloaded to these two 800 MHz little-endian ubicom32 derived cores, which are running a proprietary 'firmware' (similar to how a FPGA is used) with support for many low- and highlevel protocols (from ppp to IPsec and QoS) - the linux kernel never gets to see these packets. Assuming qca-nss-gmac would work correctly, enabling the linux kernel's own flow-offloading or SQM would signal to qca-nss-gmac "hey, the linux kernel wants to deal with these packets directly, stop whatever you're doing and let it pass through the normal netfilter code and the two 1.7 GHz ARMv7 cores".

Yes, qca-nss-gmac could be rewritten into a hardware flow-offloading module for the linux kernel's flow-offloading framework in a remotely similar way to what's already working on mt7621, but that is a very non-trivial amount of work. Looking at this thread I don't anyone has started to look into this endeavour.

1 Like

I would think so too. The amount of code or hacks required to enable the NSS cores is already too much for my little brain to handle. At the moment, I've only targeting to get the NSS cores working correctly and make it ready for 'production' use. After that, maybe think of how it can be merged back into the mainstream.

Hi folks,

Managed to get newer NSS drivers working now. Created a new branch:

This one is 'cleaner' and the drivers are all from the same set targeting the Linux 4.4.60 kernel. My previous test branch is a mis-mash of drivers and kernel patches of different versions.

I'm planning to use this for actual 'production' use .... soon ... hopefully.

Still with ECM enabled only though.

Will tackle qca-nss-crypto next.

2 Likes

Hi folks,

Happy to report that the NSS core crypto function has been successfully enabled for OpenWRT/LEDE. I only performed some simple test using the built-in crypto-bench tools. Looks like everything is working without any issues.

I'll upload the firmware images to my Box account soon for anyone interested to have a go at it. Otherwise you can build a working image using the repo link I've posted earlier.

Have fun.

5 Likes

you mean qca nss drv? (from what branch did you make it working?)

now the problem is find more guy to port this to newer kernel...
my work has stopped due to an exam... think will restart in the christmas time...

@Ansuel I used the NHSS.QSDK.6.1.r1 branch from CodeAurora. Crypto is working now.

I wasted a lot of time testing against the R7800 NSS firmware, which was meant for older drivers. Then I thought using the firmware for IPQ807x could work and so extracted from Netgear RAX120 — Nighthawk AX12. That didn't work as well.

Luckily the Synology folks used kernel 4.4 for their RT2600ac. I extracted the NSS firmware from the Synology firmware and everything started working perfectly!

From what I see, it should be trivial to port the NHSS.QSDK.6.1.r1 branch drivers to 4.14. This branch is built against 4.4.60. My repo is running 4.4.165. Looks OK so far.

Right now, I'm compiling more of the drivers, like qca-ssdk, etc. Once tested OK, I'll check in to my repo.

anyway just to make things more clear....

ssdk inizialize everthing then nss-drv load the firmware blob and nss-eth inizialize the gmac port...

right?

qca-ssdk is not needed to start network packet acceleration. It seems SSDK is needed mainly for bridging acceleration (i.e. L2 stuffs.) Right now, my R7800 does not have the qca-ssdk installed but routing is accelerated and the 4 crypto engines works.

Can you test if only with eth drv and nss drv ethernet traffic works?
Also can you check if WiFi goes to the nss core?

I've tested that before. Without qca-nss-ecm, nothing goes to the NSS cores. All network packets gets routed to the Linux kernel for processing. To get routing acceleration, these will be needed:

qca-nss-gmac
qca-nss-drv
qca-nss-ecm

The drivers below depends on the one above.

Then Ethernet doesn't work or goes through CPU?
As I have GMAC and drv working and want to understand if it works

qca-nss-gmac basically handles the GMAC device. It doesn't accelerate anything. It only provide hooks to the upper layer drivers to capture in-coming to send out-going network packets.

qca-nss-drv is the interface to the NSS cores to ties the GMAC to the NSS cores.

qca-nss-ecm is the one that orchestrates all the acceleration.

This link will help you understand:

Btw, WiFi doesn't seem accelerated. When I tested using WiFi, the router Krait CPU sirq shot up.

From DTS it looks like it should be also accelerated
Think you should check that

DTS configurations for the GMAC interfaces basically configured the GMAC parameters. I don't see anything specifically that accelerate packet routing.

Besides, the GMAC is a physical layer, at L1/L2. Routing is at layer 3 and above. There's no way a GMAC driver can help accelerate routing. It can at most do GRO/LRO. I also don't see any code in the GMAC drivers that accelerate routing performance.

All routing acceleration are done by the NSS cores, not GMAC, and it's coordinated by the qca-nss-drv driver as the interface to the NSS firmware. The NSS firmware and qca-nss-emc driver basically does what the shortcut-fe and its connections manager does (i.e. fast-classifier or shortcut-fe-cm). It's benefit is that it does it all within it's own CPU with very minimal OS overhead, so it does it really really fast. And at 800MHz with two cores, it can do a lot.

Someone should check if WiFi trigger some CPU load on stock firmware.