OpenWrt 21.02.0 third release candidate

[eginnc]

That may help explain why I get close to or full line rate between devices connected through my ER-X, and only see slow speeds when the ER-X is functioning as the server in iperf3 tests? The other area I've noticed slower performance than 19.07 is SQM on the wan (~140 vs ~180 Mbps) - all of which goes through the CPU. My ISP is 230 Mbps, so SQM is CPU limited either way, just more so with DSA I guess.

[ByteEnable]

I was just googling for some performance numbers on DSA but haven't found any yet. However, I did run across a DSA paper which mentions openwrt.

Distributed Switch Architecture , A . K . A . DSA

• A. Lunn, Vivien Didelot, Florian Fainelli
• Published 2017
• Computer Science

The Distributed Switch Architecture was first introduced to Linux nearly 10 years ago. After being mostly quiet for 6 years, it recently became actively worked on again by a group of tenacious contributors. In this paper, we will cover its design goals and paradigms and why they make it a good fit for supporting small home/office routers and switches. We will also cover the work that was done over the past 4 years, the relationship with switchdev and the networking stack, and finally give a heads-up on the upcoming developments to be expected.

Paper is here: https://legacy.netdevconf.info/2.1/papers/distributed-switch-architecture.pdf

Kernel doc: https://www.kernel.org/doc/html/latest/networking/dsa/dsa.html

[vevere]

@slh

That is incorrect.

what is incorrect ? and why please. I'm always happy (really) to know about my errors.
My last post was not about technical problems, but more about how he replied to @madires in a "RC" topic

@madires

Edit: DSA doesn't have any impact on ethernet traffic handled inside the switch chipset. Maybe a slightly higher delay.

So I had misunderstood.
What I understood (regarding lan to lan traffic) in the case where we have neither vlan nor sqm:

• case 1, swconfig: all the traffic remains in the switch, the CPU is not affected, we reach the maximum speed of the switch (say 1Gbps)
• case 2, VSA: each packet must now go through the CPU to have a switch tag added. The throughput is therefore dependent on the CPU power (and therefore less CPU power useful for other tasks)

[anomeome]

The only shortcoming that I am currently aware of regarding the switch to DSA is that there is only one CPU port in play (I think this is true on all supported platforms). If the switch device can be utilised, it still does the job as it did under the previous swconfig driver. This issue should only manifest if there is more than 1 CPU port, and you have a symmetric connection approaching the CPU link speed; i.e. 1Gb / 1Gb.

[slh]

@vevere

@slh

That is incorrect.

what is incorrect ? and why please.

DSA does (transparently) offload to the switch fabric, inter-LAN traffic is accordingly handled within the switch fabric, in hardware - at line speed, and never seen by the SOC's CPU.

Edit: Just to take an example, the realtek target with its single-core 500 MHz RTL83xx mips4k SOC has a rather underpowered CPU (it's fast enough to run luci smoothly, but you don't want it to do routing at any meaningful WAN speed), but it does support devices with 8 to 52 1000 MBit/s ethernet ports (including some 10 GBit/s ports for RTL93xx), it does use a DSA switch driver and does deliver full speed and features like link aggregation and similar at line-speed of the switch fabric (so e.g. 48 GBit/s for the gs1900-48).

[ByteEnable]

I am seeing in dmesg that mv88e6xxx_probe is being called twice during boot per the following printk on a WRT3200ACM. Shouldn't the probe only be called once?

mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1

[Sat Jul  3 21:28:33 2021] libphy: Fixed MDIO Bus: probed
[Sat Jul  3 21:28:33 2021] libphy: orion_mdio_bus: probed
[Sat Jul  3 21:28:33 2021] mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1
[Sat Jul  3 21:28:33 2021] libphy: mv88e6xxx SMI: probed
[Sat Jul  3 21:28:33 2021] mvneta_bm f10c8000.bm: Buffer Manager for network controller enabled
[Sat Jul  3 21:28:33 2021] mvneta f1070000.ethernet eth0: Using hardware mac address 30:23:03:df:49:28
[Sat Jul  3 21:28:33 2021] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_
netfilter if you need this.
[Sat Jul  3 21:28:33 2021] mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1
[Sat Jul  3 21:28:33 2021] libphy: mv88e6xxx SMI: probed
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan4 (uninitialized): PHY [mv88e6xxx-1:00] driver [Marvell 88E1540] (irq=6
4)
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan3 (uninitialized): PHY [mv88e6xxx-1:01] driver [Marvell 88E1540] (irq=65)
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan2 (uninitialized): PHY [mv88e6xxx-1:02] driver [Marvell 88E1540] (irq=66)
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan1 (uninitialized): PHY [mv88e6xxx-1:03] driver [Marvell 88E1540] (irq=67)
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 wan (uninitialized): PHY [mv88e6xxx-1:04] driver [Marvell 88E1540] (irq=68)
[Sat Jul  3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00: configuring for fixed/ link mode
[Sat Jul  3 21:28:34 2021] DSA: tree 0 setup

[madires]

@slh

DSA does (transparently) offload to the switch fabric, inter-LAN traffic is accordingly handled within the switch fabric, in hardware - at line speed, and never seen by the SOC's CPU.

Yep, a DSA capable switch chipset still works as a plain standard ethernet switch. And switch tags are pushed or popped in hardware by the switch. But it can bite you at L2 in case you have set up a bridge between some ethernet ports of the switch and the WiFI interfaces. Traffic between ethernet ports and WiFi interfaces has to pass the CPU, i.e. the CPU needs to push or pop the switch tags. I hope we'll see more linux drivers for hardware offloading (routing, NAT, PPPoE and so on). Unfortunately some vendors of switch chipsets declared their offloading APIs a state secret.

[vevere]

Oh, thanks, that is very interesting.

That's probably one reason why wifi download speed is limited on this MT7621 AC1300 2x2 device:

Sorry for the screenshot in French : Mo/s == MB/s

Even if that doesn't explain why the upload speed is so low in my case:

[slh]

@madires

But it can bite you at L2 in case you have set up a bridge between some ethernet ports of the switch and the WiFI interfaces. Traffic between ethernet ports and WiFi interfaces has to pass the CPU

There is no material difference between swconfig or dsa in this regard, hardware offloading of wireless interrupts and processing is an orthogonal question (e.g. NSS).

[anon80727944]

Found a number of bugs in rc3 for link aggregation LuCI interface.

Also looking for some guidance what additional manual tweaks are needed to make it work.

OpenWrt 21.02.0-rc3 Link Aggregation

BACKGROUND I recently upgraded my unmanaged switch to a Linksys LGS326 managed switch with support for LACP link aggregation. I was able to connect my three TrueNAS CORE NAS systems through the switch using 2 port LCAP and was able to confirm simultaneous bandwidth from multiple clients using iperf3 [91c3b96d4ce7f07be9eea0851cb4b0d30e00440e] OPENWRT LACP I wanted to test my Linksys WRT1900AC in static or dynamic LAG mode and test for any performance improvement before I spend any coin on a…

[Pico]

is that MT76 benchmark with hardware offloading enabled (in Luci firewall section)?

[Barney]

@vevere

and now with DSA causing every packet to go to the CPU and consume a lot more CPU than before, you will have a big loss of speed from LAN to LAN.

If this is really true, then I would install a dumb/unmanaged switch for all LAN devices and connect this switch to the router. As a consequence only traffic, which needs routing, goes to the router.

[vevere]

@Pico

is that MT76 benchmark with hardware offloading enabled (in Luci firewall section)?

yes, SW+HW offloading on ... but it appears that HW offloading is broken in actual builds => OpenWrt 21.02.0 third release candidate - #126 by danghuy1994

Edit: thanks for asking, so I made an upload test without HW offloading, and it is better:

No difference in download speed, 67MB/s
These tests were made with an Intel AX200 card on my laptop and a Cudy wr1300 AP (so: 866.7 Mbit/s, 80 MHz, VHT-MCS 9, VHT-NSS 2, Short GI)

Edit 2:

Edit 3:
I made more testing, and I'm so sorry, in fact it seems to have no difference:

• with or without sw offloading
• with or without hw offloading

I have now 40 MBps instead of 31 simply because I moved my laptop a few cm
oops

[vevere]

@slh

Edit: Just to take an example, the realtek target with its single-core 500 MHz RTL83xx mips4k SOC has a rather underpowered CPU (it's fast enough to run luci smoothly, but you don't want it to do routing at any meaningful WAN speed), but it does support devices with 8 to 52 1000 MBit/s ethernet ports (including some 10 GBit/s ports for RTL93xx), it does use a DSA switch driver and does deliver full speed and features like link aggregation and similar at line-speed of the switch fabric (so e.g. 48 GBit/s for the gs1900-48).

Under OpenWRT 21.02-rc3 ?

[slh]

Yes (well, up to 28 ports in the form of the DGS-1210-28 rev F1, gs1900-48 support is pending).

[flygarn12]

https://git.openwrt.org/?p=openwrt/openwrt.git;a=shortlog;h=refs/heads/openwrt-21.02

It starts to be days now between the updates…

Is there any clue for the future if we are expecting a RC4 or a formal release soon?

[phinn]

Right but take a look at LuCI commits, there have been a ton of fixes since rc3. If there was a fix you needed you can always install a 21.02-snapshot.

openwrt/luci

[flygarn12]

The Luci commits you refers to is also two days old now, this was in the beginning only hours…
The master branch seems to be where the works are happening now.

But I thought more like this forum tread is 160 posts now and we actually now discuss more how a network router cpu and a switch works than actual faults. We should probably need a big DSA tread where everyone can discuss how DSA works because it seems to be a hot topic.

So from my viewpoint it feels like 21.02 need to try its wings or it will probably never really fly because it will never be fault free, in the best of worlds that version will be named 21.02.7 or maybe 8.
Alternatively what exactly are we waiting for now?

[vevere]

I would be very happy to see the bug regarding DHCPv6 fixed soon (no leases and no IPv6-PD if leasetime <12h)

[madires]

@slh

There is no material difference between swconfig or dsa in this regard, hardware offloading of wireless interrupts and processing is an orthogonal question (e.g. NSS).

It appears to me that the processing overhead for DSA's switch tags is a little bit larger than for VLAN tags. If I understand DSA correctly the VLAN tags are replaced with the DSA tags, at least for the top VLAN tag. Do you happen to know how VLAN tagged ports forwarding several VLANs are handled by DSA? Does that also work with just a switch tag and the switch chipset inserts the corresponding VLAN tag for egress frames? Or does the CPU need to push the VLAN tag and the switch tag?