Edgerouter ER-X: can eth0 be put back on the hardware switch?

In the default DSA configuration, eth0 is the WAN port and my understanding is that it is configured to connect directly to the CPU for more bandwidth.

But if I want to use the ER-X (ramips/mt7621) as a 5-port managed switch, what can I do to put eth0 back on the hardware switch? I know I can remove the wan and wan6 interfaces and put eth0 in the br-lan config, but I think that will make a software bridge so all traffic via eth0 will go through the CPU rather than being switched in the fabric.

Would I need to reconfigure the device tree/DSA and compile a custom image or is there something that can be done on an official 24.10 image?

The easy workaround for now is to put a device on eth0 that generates only minimal traffic, but it would be nice to have 5 identical ports if possible.

Have you tested it what happens?

Even if I add eth0 to br-lan, ip link still shows eth0 standalone, while eth1-4 show up as eth1@dsa, eth2@dsa, etc. I’m not sure how to tell what really happens to the packets whether they go through the CPU.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: dsa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc mq state UP qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
3: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-lan state DOWN qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
4: eth1@dsa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
5: eth2@dsa: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
6: eth3@dsa: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
7: eth4@dsa: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff
9: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether b4:fb:e4:... brd ff:ff:ff:ff:ff:ff

That is limitation of dsa, you can change dts and rebuild for different/no switch partition or ease current port mapping on CPU using bridger package.

2 Likes

Is this what removes eth0 from the switch?

There is other place where a port is connected to secondary gmac or serdes or something, that you need to disable.

You can quickly determine whether packets are traveling through the switch or the CPU.

Simply run htop in a terminal, connect to eth0 and, for example, eth2, and run a speed test. You'll see if the CPU is participating (and it is) in forwarding packets.

The next step is to test between eth1 and ethx (x = 2-3), skipping eth0 – then you're using the switch. The CPU should be idle 99% of the time.

You can see that via ip link - dsa ports will be in form lan5@eth1

2 Likes

I'm still learning :wink: Thx @brada4.

Looks like it’s in the same file:
https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/dts/mt7621_ubnt_edgerouter-x.dts#L8-L15

The SFP version of the same router doesn’t do any of this, I assume because the wiring is different?
https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/dts/mt7621_ubnt_edgerouter-x-sfp.dts

It is here

It looks complicated to modify DTS when you have 0 experience with it :cry: Maybe a kind soul with DSA/DTS experience could modify the files correctly and then I can figure out how to build the image myself.

They were modified this way to double throughput in the default case.
Do you get normal speed using bridger package?

I have not tried bridger Is there a guide on proper bridger configuration? Do you have to delete the default br-lan and replace it with a bridger?

Just install the package. You can exclude interfaces in config file, but that is normally not needed. No config change needed, just magic happens with couple of tc-bpf programs.

1 Like

So just install bridger, delete wan/wan6 interfaces, add eth0 to br-lan?

do not change anything. measure speed/cpu usage, should be 2-3x down from before.

1 Like

Not sure I understand. Do I not need to put eth0 on br-lan to make it just like the other 4 ports?

That should have been in the bridge to start with?

By default eth0 is not in br-lan, only eth1-4 are. eth0 is standalone on the wan/wan6 interfaces.

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1'
        list ports 'eth2'
        list ports 'eth3'
        list ports 'eth4'

config interface 'wan'
        option device 'eth0'
        option proto 'dhcp'

config interface 'wan6'
        option device 'eth0'
        option proto 'dhcpv6'