The size of the switch tag differs based on the vendor and tag type:
- Broadcom 4 bytes
- Marvell DSA 4 bytes
- Marvell EDSA 8 bytes
- Qualcomm 2 bytes
2 Likes
Yes, I had this issue, to the extent that I abandoned wifi on my C7 v2 and went to a commercial AP. As at RC2 I tried it again and have been using for a month without any issues except the channel survey tool causes the 5G radio to crash and recover. I also had the 5G radio decide to go into client isolation mode by itself with nothing showing in the logs, that seems to be a one-off though. The 2.4G has been solid. I am a bit nervous about going to RC3 but I do like an adventure 
The world has simply moved on. 100Mbit internet feels like it starts to be low speed standard in the industrial world as of today.
My feeling on my Linksys WRT3200ACM with DSA is that the data flows a lot faster with 21.02 then with 19.07 or older. My EdgeRouter 4 didn’t have any 19.07 experience but it runs very fast on 21.02 with DSA.
If your hardware cant handle this additional bytes that DSA has with the internet speed the world demands then you are on the very edge of internet connection collapse and really should be thinking about a upgrade of hardware because the world isn’t stopping on this internet thing, that I can promise you.
1 Like
Instead of the discussion being based on individual feelings I suggest measured numbers for latency, maximum capacity for throughput and packets per second. This is comparable.
3 Likes
It is 98Mbit/800Mbit on both. It isn’t the speed itself that it smoother or faster. It is the lagging that is less noticeable.
So now with the facts do you think this will stop DSA in the world?
This whole forum is only based on individual feelings like “I successfully installed it”! So what?
After installing 21.02-RC3, I found that my browsing experience was miserable, on both wireless and wired connections. There were large delays in displaying content and many sites completely failed. After some investigations, I found that the issue appeared to be with DNS and in particular with DNS over IPv6. DNS over IPv4 is fine.
As a workaround, I disabled the WAN6 interface, and performance was great again. I'm still using IPv6 internally, but not externally.
Are there any commands that I can execute to provide further information? I am comfortable in the CLI but not expert.
What I see from ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 60:e3:27:c8:4a:11 brd ff:ff:ff:ff:ff:ff
inet xxx.xxx.120.28/18 brd xxx.xxx.127.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 xxxx:xxxx:c002::1:b923/128 scope global dynamic noprefixroute
valid_lft 3586sec preferred_lft 3586sec
inet6 fe80::62e3:27ff:fec8:4a11/64 scope link
valid_lft forever preferred_lft forever
from Luci
Protocol: DHCPv6 client
Uptime: 0h 0m 18s
MAC: 60:E3:27:C8:4A:11
RX: 326.29 MB (44339348 Pkts.)
TX: 2.76 GB (20814158 Pkts.)
IPv6: xxxx:xxxx:c002::1:b923/128
IPv6-PD: xxxx:xxxx:c802:9c2a::/64
What I suspect is the issue: my ISP is providing short Valid and Preferred IPv6 lifetimes. The Valid lifetime is the same as Preferred, when Valid should be twice the Preferred lifetime.
This interacts with changes to odhcpd for 21.02-RC3 regarding the lifetime of IPv6 addresses and leases. All IPv6 addresses, both externally routed and internal fda6:xxx addresses on the internal network are given mostly 30 mins leases, occasional 60 mins leases, and occasionally no IPv6 leases is allocated at all.
After disabling the WAN6 interface, all devices are given 14d leases on the fda6:xxxx addresses.
The ISP (in Singapore) provides no means to report this issue - surprise, surprise.
We won’t find consensus if we only discuss individual feelings.
I could understand „the lagging is less noticeable“ if I would see measured numbers for the perceived lag. For example: measured latency for specific router tasks with one release compared to measured latency for the same task in the same configuration on another release.
This would be a fact based discussion.
Exactly. I tested my WRT32X a bit in with the RC builds (21.02-snapshot has had a ton of bug fixes last few weeks, mostly with LuCI, so it'll only improve).
Going from 19.07.7 and 21.02-snapshot my performance is roughly the same. I do use irqbalance now, mainly because it puts WiFi from CPU0 over to CPU1. SQM Cake on 500Mbit down/ 35Mbit up cable modem: maxes out the connection set to 95% throughput on up/dl. My ping spread is 1-5ms range under max load, A+ bufferbloat A+ quality on dslreports speedtest. Haven't gone any further, but it's certainly working well, no stability problems over the last couple weeks. It's good news because I'd rather use upstream kernel code whenever possible, DSA included.
I have a WRT3200 which is the same hardware as WRT32X. I also have a similar cable modem ratio. Mine is 400/20. I have not experienced any issues other than with my Android phone and wifi.
I think the wifi issue may be related to the max power of 23db. If I am in the house everything is good but once I get outside and walk 5 or so yards from the house the android drops. On the 19.x series and Android 10 I was able to sit in my car parked on the street and get good wifi. I'm running Android 11 and a few weeks old 21 snapshot that I built now. Maybe the distance issue is related to both Android 11 and 21 snapshot marvell updates (kernel 5.10.47).
If I use a wifi analzyer on the phone my db is approximately -70 sitting outback maybe 30 feet away from the router. Most of my neighbors are running 2.4ghz. I am running 5ghz with VHT80. All those 2.4ghz are -50 to -70db on the analzyer. I am the only one on my 5ghz channel which is 120.
Other than the distance issue with Android I have not experienced any problems.
Hi,
Internet connection speed is one thing
Wifi connection speed is another thing
Lan to Lan connection speed is yet another thing
But, well I think, a lot of us use OpenWRT powered devices not only for Wifi or as an internet router, but also as an internal network switch.
If I just take my use case:
- Any of my routers can handle the 300 / 300Mbps of my internet line (X86_64, IQP4x, MT7621 etc ...) no problem for that (I'm not talking about SQM)
- I have a very powerful wired router only as the main router
- My main AP is running OpenWRT
- I use Wifi a lot from my laptop to move (large) files to my LAN file server
- I use my main (wired) computer a lot to copy large files to the same file server even more often ... unfortunately my AP also acts as a standard switch between my wired computer and the file server (not the case today, but that was a few days ago)
if before DSA (so with swconfig) you could have a speed of 1 Gbps from LAN to LAN (because the "thing" inside the router with 4 RJ45 ports on the outside is only ... . a switch!) and now with DSA causing every packet to go to the CPU and consume a lot more CPU than before, you will have a big loss of speed from LAN to LAN.
So it's understandable that they can't be happy with DSA, a lot of them don't use vlan so the penalty is immediate for them.
Therefore:
=> if DSA adds a speed penalty, but it's not really a problem if I'm only talking about internet browsing. So if I judge DSA purely for the speed of internet usage, that will never be a problem until I have a faster ISP ... because any of my access points can do over 300 Mbps, even acting like a dumb switch.
BUT
=> how should I judge DSA if it gives a big penalty to my LAN speed ???
=> And what about the power consumption (and therefore the internal temperature of the router) if the CPU is more used?
Some guys are reporting here that DSA is adding speed impact, I think it's not really smart to say 'change your old hardware' to them.
these are just questions, not a claim (in fact I haven't taken the test yet).
And it will be even worse if the router that one of these guys uses is also used for other CPU related things (VPN endpoint, SQM, Adblock of whatever you can imagine) because the CPU power is already being used for DSA
I'm very happy to learn that your Ubiquiti Edgerouter 4 seems faster now but you perhaps have to understant that:
- it does not have any internal siwtch (4 independant Gb Nics)
- it perahps has a fast CPU (don't know nothing about the 4 cores Cavium CN7130)
=> so it is perhaps expected for YOUR router to have a better speed now with DSA
'legacy' routers have a very very different hardware, using DSA with a standard switch should perhaps be a big problem isn't it ?
We are here to talk about a Release Candidate before it will become the stable branch ... it is a GREAT thing to have many guys to test many things.
It's good to know that is it working fine on your ER4
it is ALSO good to know that we WILL have a speed penalty for 95% of the routers using OpenWRT
it is not good (at all) to tell him to replace its hardware
my 2 cents
3 Likes
That is incorrect.
I'd like to add some clarification to the DSA discussion. DSA doesn't imply that the internet access or NAT throughput is lower in any case. It's only lower if the ethernet frames pass the CPU (no hardware offloading) and the CPU isn't fast enough to process network traffic at line speed. But the additional processing overhead for the DSA switch tags will always increase the frame/packet delay by a tiny amount which isn't critical for SOHO networks.
Edit: DSA doesn't have any impact on ethernet traffic handled inside the switch chipset. Maybe a slightly higher delay.
1 Like
That may help explain why I get close to or full line rate between devices connected through my ER-X, and only see slow speeds when the ER-X is functioning as the server in iperf3 tests? The other area I've noticed slower performance than 19.07 is SQM on the wan (~140 vs ~180 Mbps) - all of which goes through the CPU. My ISP is 230 Mbps, so SQM is CPU limited either way, just more so with DSA I guess.
I was just googling for some performance numbers on DSA but haven't found any yet. However, I did run across a DSA paper which mentions openwrt.
Distributed Switch Architecture , A . K . A . DSA
- A. Lunn, Vivien Didelot, Florian Fainelli
- Published 2017
- Computer Science
The Distributed Switch Architecture was first introduced to Linux nearly 10 years ago. After being mostly quiet for 6 years, it recently became actively worked on again by a group of tenacious contributors. In this paper, we will cover its design goals and paradigms and why they make it a good fit for supporting small home/office routers and switches. We will also cover the work that was done over the past 4 years, the relationship with switchdev and the networking stack, and finally give a heads-up on the upcoming developments to be expected.
Paper is here: https://legacy.netdevconf.info/2.1/papers/distributed-switch-architecture.pdf
Kernel doc: https://www.kernel.org/doc/html/latest/networking/dsa/dsa.html
3 Likes
what is incorrect ? and why please. I'm always happy (really) to know about my errors.
My last post was not about technical problems, but more about how he replied to @madires in a "RC" topic
So I had misunderstood.
What I understood (regarding lan to lan traffic) in the case where we have neither vlan nor sqm:
- case 1, swconfig: all the traffic remains in the switch, the CPU is not affected, we reach the maximum speed of the switch (say 1Gbps)
- case 2, VSA: each packet must now go through the CPU to have a switch tag added. The throughput is therefore dependent on the CPU power (and therefore less CPU power useful for other tasks)
1 Like
The only shortcoming that I am currently aware of regarding the switch to DSA is that there is only one CPU port in play (I think this is true on all supported platforms). If the switch device can be utilised, it still does the job as it did under the previous swconfig driver. This issue should only manifest if there is more than 1 CPU port, and you have a symmetric connection approaching the CPU link speed; i.e. 1Gb / 1Gb.
1 Like
DSA does (transparently) offload to the switch fabric, inter-LAN traffic is accordingly handled within the switch fabric, in hardware - at line speed, and never seen by the SOC's CPU.
Edit: Just to take an example, the realtek target with its single-core 500 MHz RTL83xx mips4k SOC has a rather underpowered CPU (it's fast enough to run luci smoothly, but you don't want it to do routing at any meaningful WAN speed), but it does support devices with 8 to 52 1000 MBit/s ethernet ports (including some 10 GBit/s ports for RTL93xx), it does use a DSA switch driver and does deliver full speed and features like link aggregation and similar at line-speed of the switch fabric (so e.g. 48 GBit/s for the gs1900-48).
8 Likes
I am seeing in dmesg that mv88e6xxx_probe is being called twice during boot per the following printk on a WRT3200ACM. Shouldn't the probe only be called once?
mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1
[Sat Jul 3 21:28:33 2021] libphy: Fixed MDIO Bus: probed
[Sat Jul 3 21:28:33 2021] libphy: orion_mdio_bus: probed
[Sat Jul 3 21:28:33 2021] mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1
[Sat Jul 3 21:28:33 2021] libphy: mv88e6xxx SMI: probed
[Sat Jul 3 21:28:33 2021] mvneta_bm f10c8000.bm: Buffer Manager for network controller enabled
[Sat Jul 3 21:28:33 2021] mvneta f1070000.ethernet eth0: Using hardware mac address 30:23:03:df:49:28
[Sat Jul 3 21:28:33 2021] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_
netfilter if you need this.
[Sat Jul 3 21:28:33 2021] mv88e6085 f1072004.mdio-mii:00: switch 0x3520 detected: Marvell 88E6352, revision 1
[Sat Jul 3 21:28:33 2021] libphy: mv88e6xxx SMI: probed
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan4 (uninitialized): PHY [mv88e6xxx-1:00] driver [Marvell 88E1540] (irq=6
4)
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan3 (uninitialized): PHY [mv88e6xxx-1:01] driver [Marvell 88E1540] (irq=65)
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan2 (uninitialized): PHY [mv88e6xxx-1:02] driver [Marvell 88E1540] (irq=66)
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 lan1 (uninitialized): PHY [mv88e6xxx-1:03] driver [Marvell 88E1540] (irq=67)
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00 wan (uninitialized): PHY [mv88e6xxx-1:04] driver [Marvell 88E1540] (irq=68)
[Sat Jul 3 21:28:34 2021] mv88e6085 f1072004.mdio-mii:00: configuring for fixed/ link mode
[Sat Jul 3 21:28:34 2021] DSA: tree 0 setup
Yep, a DSA capable switch chipset still works as a plain standard ethernet switch. And switch tags are pushed or popped in hardware by the switch. But it can bite you at L2 in case you have set up a bridge between some ethernet ports of the switch and the WiFI interfaces. Traffic between ethernet ports and WiFi interfaces has to pass the CPU, i.e. the CPU needs to push or pop the switch tags. I hope we'll see more linux drivers for hardware offloading (routing, NAT, PPPoE and so on). Unfortunately some vendors of switch chipsets declared their offloading APIs a state secret.
3 Likes
