Short version: the NSS core on the IPQ5018 accelerates routed NAT and both Wi-Fi radios on kernel 6.18 with the upstream stmmac driver - no qca-nss-dp, no qca-ssdk - and the CPU stays above 90 % idle while it does. It has been running here since late August on a GL.iNet GL-B3000.
This replaces the original opening post. Its Wi-Fi section said the offload could not work and blamed the firmware; that was wrong, and the reason is in post #166. Everything else stands.
Both branches are public:
- https://github.com/kuncy7/openwrt-nss-edma/tree/ipq50xx-nss
- https://github.com/kuncy7/nss-packages/tree/ipq50xx-nss (the feed - you need both)
Documentation: https://github.com/kuncy7/openwrt-nss-edma/blob/ipq50xx-nss/README.ipq50xx.md
- build steps, the
.config, how the plane comes up, the porting checklist for other IPQ5018 boards.
They are layered on @JuliusBairaktaris's edma-nss tree, which is where the NSS-on-mainline work actually lives; my part is the ipq50xx half.
This is a second data point rather than a first. @ADCDS got NSS running on an IPQ5018 (Xiaomi AX3000T) back in July and wrote it up in Xiaomi AX3000T (RD03v2, IPQ5018): working OpenWrt, installs to NAND. That port uses qca-nss-dp and an Airoha switch on 6.12; this one goes the other way - upstream stmmac on 6.18, QCA8337 - so between the two, most IPQ5018 boards are now covered by some known-working combination.
@georgem83 asked in that thread whether NSS would work with the new ethernet stack. It does, and this is what it takes.
The glue
The NSS firmware drives the Synopsys GMAC DMA itself, with its own descriptor rings. Rather than replace the MAC driver, a small module (qca-dwmac-nss) claims the data path through an API added to stmmac: TX queues drained, NAPI off, DMA stopped, ndo_start_xmit redirected, and only then the controller handed over. phylink, MDIO and the netdev stay with the host, so the PHY and the link keep being managed by the kernel exactly as before. Releasing the claim deliberately leaves TX disabled - after the firmware has owned the controller the host ring state is stale, and the only correct way back is a full reinit, which the owner triggers by bouncing the interface.
None of it happens at probe. Which GMACs to take over is a runtime decision through a debugfs mask, and if the takeover fails the port stays on the host data path with working networking. That property is worth more than it sounds: every failed experiment during this work ended with a reachable router.
Three orderings that fail silently
All measured, each cost me days:
- arming on a down interface - the firmware starts, TX works, and not one ingress frame is ever delivered;
- arming while netifd is still applying config - netifd bounces the interface mid-takeover, and the promisc flag the bridge sets afterwards never reaches the hardware;
- loading
qca-nss-vlanbefore the arm - the VLAN manager resolves the trunk's NSS interface number atNETDEV_REGISTERtime; before the arm that number does not exist, registration fails silently (the driver's debug macros are compiled out at the default log level), and every tagged frame disappears.
Working order: wait until netifd is done, arm, wait for the takeover, then load the VLAN manager - its notifier replays NETDEV_REGISTER for existing netdevs, so ordering against netifd stops mattering. That is what the nss-tools-dwmac service in the branch does.
One prerequisite that is easy to miss: the CMN PLL clock fix. The driver lets pm_clk gate the CMN block's bus clocks a few milliseconds after probe, which races with early boot on IPQ5018; a build that loses the race dies right after the CMN PLL probe - watchdog reset, nothing on the console. Whether a given board and build lose it varies (@georgem83's IPQ5018 boards boot without it; my B3000 boot-loops 8/8, and on the AX6000 the stock snapshot boots while an NSS build does not), so treat it as required for this branch rather than universally. Merged into openwrt main as 86b584bd0994; kernel v4 still on the clk list: https://lore.kernel.org/linux-clk/20260813093351.178419-1-kuncy7@gmail.com/
The switch, and why one CPU port is enough
The NSS firmware parses 802.1Q natively but cannot parse the two-byte Atheros header that DSA's tag_qca puts where the ethertype should be. So on this SoC DSA user ports and ECM acceleration are mutually exclusive. The port model is also reversed compared to ipq807x: the NSS phys_if is the GMAC, and the netdev is the switch trunk.
If you are seeing eth_rx_unknown_l3_protocol counting most of your frames, with iface_count=0 and accelerated_count=0 on a QCA8337 board
- @LS3434 posted exactly that in the Qualcommax NSS Build thread - this is why, and no ECM patch will fix it.
What works instead: let qca8k do the hard bring-up (SerDes, clocks, uniphy), then unbind it and drive the fabric directly. qca8337-nss wakes the front PHYs, re-enables the MACs, turns the Atheros header off and programs the VTU from a module parameter. The CPU port then carries a plain 802.1Q trunk - eth0.1 for LAN, eth0.2 for WAN - and those two become NSS dynamic interfaces of type 17, which the firmware understands.
The rule then carries two distinct NSS interface numbers over a single physical port, which is why this works on a board that has only one CPU port. That is the practical difference from the AX3000T approach, which needs LAN and WAN on different conduits. The cost is that the LAN ports share one VLAN, so per-port separation inside the LAN is gone; WAN/LAN isolation, VLAN-aware bridging on top and the clean fallback are intact.
Second board: Archer AX55
The reason this is written as a platform port rather than a device port: the same arrangement should carry to the TP-Link Archer AX55 v1, also IPQ5018 + QCN6122, whose OpenWrt support I maintain separately. What differs is the switch - RTL8367S instead of QCA8337 - so it needs an rtl8367-nss counterpart doing the same job: unbind rtl8365mb, program the VLAN table and PVIDs over MDIO. The register knowledge is already in the kernel driver, so that is transcription rather than reverse engineering.
Two things make it interesting beyond "one more board". Its CPU port runs at 2.5 Gbit/s, which removes the ceiling described below entirely. And it is the only board here with a foreign tagger, so getting it to work would show the approach is not shaped around one switch.
It is not done yet. I am saying what it needs, not claiming it runs.
Warm reboot, and a note for @ADCDS
Your docs/no-uart-reflash.md describes a state where, after a chain of warm reboots, the switch and the NSS core do not come up cleanly and only a cold start recovers. I had exactly that, and it has a one-line cause.
A soft reboot does not reset the NSS block, so the previous kernel's firmware is still executing out of the load region while nss_hal_firmware_load() memcpy's the new image over it. The new firmware then comes up dead or mute; cold boots are fine, which is what makes it look intermittent. TFTP-booted test images hide it completely, because u-boot's ethernet init crashes the old firmware first. The fix is to assert the core-local reset clamp before the copy - the same thing nss_hal_remove() already does on rmmod. Four out of four warm reboots clean afterwards, and the full matrix (cold, warm, warm-after-cold, sysupgrade) passes. It is patch 0136 in the feed branch.
One warning attached: do not pulse the GCC block resets on IPQ5018 from the driver. I tried; asserting them kills ethernet until a power cycle, because - as your own patch 0029 explains - asserting the GCC reset clears the CSM registers including the clamp you just set, so the core runs from an unprogrammed boot address.
In the other direction: your patch 0031 (meminfo block table ioremap_wc) applies to the 12.2 line as well. The vendor feed's patch 0004 removed the cache clean there too, so the same latent hazard is in every tree built on that feed, mine included. It is carried as 0137 with your name on it. Thanks for measuring it - 33 cold boots is not a number anyone enjoys collecting.
Wi-Fi on the NSS path
Both radios - the internal 2.4 GHz IPQ5018 and the 5 GHz QCN6122 - run the firmware's Wi-Fi data path (wifili), with ath11k on the host doing management only.
What used to kill it was a single clock branch, gcc_ubi0_core_clk. The bootloader leaves it running; Linux has no consumer for it until qca-nss-drv probes, some 25 s into boot, so clk_disable_unused() gates it at ~2 s and the driver switches it back on at probe - after __nss_hal_core_reset() has already released the UBI32 core from reset. The core comes out of reset unclocked, gets its clock about a millisecond later, boots, answers, and then the first WPA2 client takes the offload down. CLK_IGNORE_UNUSED on that one branch is the whole fix: on cold boots, with a client as the test, 11 lives / 0 deaths with the branch kept and 0 / 3 with it gated.
Same family as the CMN PLL fix above - the kernel gating a clock it does not know is used by an agent outside Linux - which is what makes it easy to miss twice.
What I wrote here before, about the core trapping inside the blob on the first RX exception, was wrong, and so were the host-side hypotheses built on top of it. The firmware was never at fault. Two side notes for anyone who followed that hunt: results from warm reboots are worthless here (one arm lived 5/0 on reboot and died on its first cold boot), and the empty WIFILI section in nss_stats was a bug in my own N2H bounds check, which dropped every SOC statistics message because those are larger than the data frame size the host advertises.
The NSS core also runs at 1 GHz now instead of 850 MHz, and the rcg didn't update its configuration warnings are gone: the driver was setting the rate before enabling the clock, and configuring the AXI buses before the core.
Numbers
Test setup: a gigabit host on a LAN port, an upstream router on WAN, NAT and routing on the B3000, iperf3 between the two.
- Sustained routed flow, 28 s steady: ~66k packets/s through the firmware, CPU 93-96 % idle across both cores, zero ECM rule churn during the transfer.
- The opposite direction reaches ~100k packets/s at the same CPU cost.
- The lowest idle figure seen under any load was 68 %. The offload is never the bottleneck.
Two honest caveats, because the headline number would mislead without them.
My test host tops out at 569 Mbit/s transmitting - measured against the upstream router directly, with the B3000 out of the path (569 up / 925 down, zero retransmits both ways) - so I never loaded the box to its limit in that direction.
And this board has a single 1 G CPU port, so a routed flow crosses it twice and data shares a wire direction with the opposing ACK stream. That caps routed TCP at
2 x 1448 B payload / (2 x 1542 + 94) B on the wire = 91.1 % -> ~911 Mbit/s
while the upstream router can send 925. Push a line-rate stream through it and TCP collapses into an 8-seconds-on / 8-seconds-off sawtooth with ~9200 retransmits, averaging 724 Mbit/s - CPU still 95 % idle, zero drops on any counter I can read, because the loss happens inside the switch. UDP one-way at 900 Mbit/s gets through with 0.45 % loss.
So ~910 Mbit/s is a ceiling of this board's design, not of the offload, and it is exactly why the AX3000T port moves WAN to a second CPU port. I could not measure the real ceiling cleanly with the hardware on hand, and would rather say that than quote a nicer number.
With Wi-Fi on the NSS path - board as a router, one laptop on 5 GHz, iperf3 to a server on the WAN side: 734/447 Mbit/s single stream, 791/534 with three streams, CPU ~90 % idle. Single runs, so read them as magnitudes rather than a benchmark.
Disclosure
This was done with heavy use of Claude Code - the bring-up debugging, the blob disassembly, the patches and this post. The measurements are real and were taken on hardware; the reasoning about what they mean had a machine in the loop, and I would rather say so than have someone work it out from the writing style.
Credits
- @JuliusBairaktaris - the edma-nss tree everything here is layered on, and the glue-over-stmmac idea.
- qosmio - the NSS packages and firmware repository this whole SoC family still stands on.
- @ADCDS - patch 0029 (core-boot reset ordering) and 0031 (meminfo cacheability); the second is a real bug in every tree using that feed.
- @georgem83 - the qualcommax/ipq50xx target itself.
Questions welcome, patches more so - particularly from anyone with another IPQ5018 board who wants to try the switch half.
eth_rx_unknown_l3_protocol doesn't rise anymore !!!