Difference between DSA and driver-level VLANs in the context of RPi4 (no switch)?

What is the difference (if any) between using Bridge VLAN filter/DSA to setup VLAN tagging and creating a driver-level VLAN in the context of a device without a switch like a RPi4?

My setup is using the RPi4 as the router/firewall with VLAN tagging with a separate dumb AP. Goal is isolation between the VLANs.

For more context, here is the network config on the RPi4 router/firewall:

/etc/config/network
# vim:set ts=2 sw=2 et:
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd1a:184b:b879::/48'
	option packet_steering '1'

config bridge-vlan
	option device 'br-main'
	option vlan '1'
	list ports 'eth0:t*'

config bridge-vlan
	option device 'br-main'
	option vlan '3'
	list ports 'eth0:t'

config bridge-vlan
	option device 'br-main'
	option vlan '4'
	list ports 'eth0:t'

config bridge-vlan
	option device 'br-main'
	option vlan '5'
	list ports 'eth0:t'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'br-main'
	option type 'bridge'
	list ports 'eth0'
	option ipv6 '0'

config device
	option name 'br-wan'
	option type 'bridge'
	list ports 'eth1'
	option ipv6 '0'

config device
	option name 'br-lxc'
	option type 'bridge'
	list ports 'br-main.4'
	option ipv6 '0'

config device
	option name 'br-main.1'
	option type '8021q'
	option ifname 'br-main'
	option vid '1'
	option ipv6 '0'

config device
	option name 'br-main.3'
	option type '8021q'
	option ifname 'br-main'
	option vid '3'
	option ipv6 '0'

config device
	option name 'br-main.4'
	option type '8021q'
	option ifname 'br-main'
	option vid '4'
	option ipv6 '0'

config device
	option name 'br-main.5'
	option type '8021q'
	option ifname 'br-main'
	option vid '5'
	option ipv6 '0'

config device
	option name 'wg0'
	option ipv6 '0'

config interface 'wan'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'
	option device 'br-wan'

config interface 'lan'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'
	option device 'br-main.1'

config interface 'guest'
	option proto 'static'
	option ipaddr '10.9.7.1'
	option netmask '255.255.255.0'
	option device 'br-main.3'

config interface 'lxc'
	option proto 'static'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'
	option device 'br-lxc'

config interface 'iot'
	option proto 'static'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'
	option device 'br-main.5'

config interface 'wg0'
	option proto 'wireguard'
...
# omitted wireguard section

I suspect there is little difference from a usability perspective. However, the config for a driver VLAN is substantially simpler.

How would I modify the network config to switch this over to driver VLANs?

The following should do it (although I would check the IP addresses and VLAN notations are correct). The loopback, global, and wireguard sections would remain the same.

config interface 'lan'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'
	option device 'eth0.1'

config interface 'wan'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'
	option device 'eth1'

config interface 'guest'
	option proto 'static'
	option device 'eth0.3'
	option netmask '255.255.255.0'
	option ipaddr '10.9.7.1'

config interface 'LXC'
	option proto 'static'
	option device 'eth0.4'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'

config interface 'iot'
	option proto 'static'
	option device 'eth0.5'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'

Thank you. Is it a literal drop in/replace my existing interfaces? All other section remain as-is?

All of:

gets replaced. So you just keep the loopback interface section, globals section, and wireguard section(s).

That's most of the config... So the tagging is inferred in the 0.x stuff? I am not understanding it.

As I said, it's substantially simpler. As you've guessed, the VLAN tagging is provided by number used in the option device 'eth0.x' declaration within each interface. You don't need all the additional config bridge-vlan or config device sections.

Edit: The situation would likely be different if a hardware switch were involved and in that case it may well be that DSA provides a simpler/easier to understand config.

OK. Here is the new file based on your suggestion. Seem good?

network.new

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd1a:184b:b879::/48'
	option packet_steering '1'

config device
	option name 'wg0'
	option ipv6 '0'

config interface 'wan'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'
	option device 'eth1'

config interface 'lan'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'
	option device 'eth0.1'

config interface 'guest'
	option proto 'static'
	option ipaddr '10.9.7.1'
	option netmask '255.255.255.0'
	option device 'eth0.3'

config interface 'lxc'
	option proto 'static'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'
	option device 'eth0.4'

config interface 'iot'
	option proto 'static'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'
	option device 'eth0.5'

config interface 'wg0'
	option proto 'wireguard'
...

I will give it try later this evening. Thanks for the help with it.

And visually, here is a vimdiff (original on right):

That was a fail :smiley: Nothing worked upon rebooting both devices... I simply replaced /etc/config/network with the contents above.

@richb-hanover-priv - I noticed that you authored the DSA mini tutorial on the wiki. Can you take a stab at my question:

Hi! I know less than you do... (I only tweaked up those pages from earlier drafts that were confusing to me.) I am cc'ing @arinc9 to see if he might be able to help.

1 Like

@richb-hanover-priv Thanks Rich.

@darksky Let's start by explaining a bit of how things work.

DSA only sets up the switch ports. E.g. interfaces called wan, lan1, lan2, etc.

To configure VLANs on a switch using DSA framework, you'd have to put all of the interfaces (wan, lan1, lan2, etc.) into one bridge (e.g. br0). Then, create a subinterface of the bridge interface (e.g. br0.5 which is based off of br0 and set to use VLAN ID 5) to send & receive frames with the specified VLAN. Check the Converting to DSA page I wrote for more details.

These interfaces (br0.5 in this example) are also called subinterfaces. This just means that they're based off of another interface. E.g. if the br0.5 is based off of br0, it's a subinterface of br0.

The name of your interface actually doesn't matter. You could name your interface whatever you want and still make it based off of, for example, br0 and set it to use VLAN 5.
Here's a screenshot where I call my interface name "whatever", it's still based off of br0 and set to use VLAN 5:


To do the same thing on the command line is this command: ip link add link br0 name whatever type vlan id 5

You can see VLAN and other information of the interfaces on the command line with this command: ip -d link (you need ip-full package on OpenWrt to use the "-d" option). E.g. the output of the "whatever" interface I created:

6: whatever@br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether a0:b0:c0:d0:e0:f0 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 0 maxmtu 65535
    vlan protocol 802.1Q id 5 <REORDER_HDR> addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 62780 gso_max_segs 65535

It's more convenient to name your subinterface e.g. br0.5 so you can easily figure out which interface it's based off of and what VLAN ID it uses by just looking at the name of the subinterface.

OpenWrt makes things more easier. If you just add an interface (it's incorrectly called "device" on LuCI) called br0.5 on one of the networks on the "Interfaces" page, OpenWrt will automatically create a subinterface of br0 and set the interface to use VLAN 5.

When you have a subinterface called br0.5 based off of br0 with VLAN 5, this is how frames are forwarded:
Any untagged frames going to br0.5 are forwarded out of br0 as tagged frames with VLAN 5.
Any tagged frames with VLAN 5 going to br0 will be forwarded to br0.5 as untagged frames.

VLAN filtering needs to be enabled on the bridge. The Linux kernel will then forward frames per the VLAN configuration.


Back to your case

On devices without a switch, the ethernet port(s) - mostly singular - is directly connected to the CPU. Therefore the interface of the ethernet port is usually called something like eth0, eth1, etc. You don't use DSA to initialise these ports in this case.

You only have one port so you don't need to put it under a bridge (if you don't want to include wireless interfaces on the VLANs. If you intend to, the wiki page I linked above should keep you covered).

Let's say you have 3 networks. On "Interfaces" page, for the first network, specify the interface as eth0.10.
For the second, specify eth0.20. For the third eth0.30.

And then you can set up the router to route between these networks.

I hope this was inclusive enough. Let me know if you have any questions.

2 Likes

Thank you very much for the detailed reply. So DSA for my is not needed, although it is working. I will try reconfiguring as you said as I have time and update the thread. Thank you.

I think I got it working. The difficult part was figuring out the LXC I'm running.

/etc/config/network
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd1a:184b:b879::/48'
	option packet_steering '1'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'eth0.1'
	option type '8021q'
	option ifname 'eth0'
	option vid '1'
	option ipv6 '0'

config device
	option name 'eth0.3'
	option type '8021q'
	option ifname 'eth0'
	option vid '3'
	option ipv6 '0'

config device
	option name 'eth0.4'
	option type '8021q'
	option ifname 'eth0'
	option vid '4'
	option ipv6 '0'

config device
	option name 'eth0.5'
	option type '8021q'
	option ifname 'eth0'
	option vid '5'
	option ipv6 '0'

config device
	option name 'br-lxc.4'
	option type 'bridge'
	option ipv6 '0'
	list ports 'eth0.4'

config device
	option name 'wg0'
	option ipv6 '0'

config interface 'wan'
	option device 'eth1'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'

config interface 'lan'
	option device 'eth0.1'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'

config interface 'guest'
	option device 'eth0.3'
	option proto 'static'
	option ipaddr '10.9.7.1'
	option netmask '255.255.255.0'

config interface 'lxc'
	option device 'br-lxc.4'
	option proto 'static'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'

config interface 'iot'
	option device 'eth0.5'
	option proto 'static'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'

config interface 'wg0'
	option proto 'wireguard'
...
lxc network section config
lxc.net.0.type = veth
lxc.net.0.link = br-lxc.4
lxc.net.0.flags = up
lxc.net.0.ipv4.address = 10.0.4.250/24
lxc.net.0.ipv4.gateway = 10.0.4.1

Note that if I didn't create br-lxc.4 and if I tried using a macvlan device, when I reboot my dumb AP, the contain's interface would mysteriously but consistently go down. I would then need to restart the lxc which is not optimal. So that bridge was my solution. Bug maybe. The bridge is more robust.

Thanks again!

1 Like

I did a slight edit to correct misinformation about how VLAN filtering works in Linux and how DSA comes into play, please read it. Also, because of your situation with LXC, you should put the eth0 into bridge (e.g. br0) and configure VLAN filtering there. Here is an example according to your set up:


And then use br0.1, br0.3, br0.4 & br0.5 on the networks.

You can simply attach any wireless interfaces to any of the networks this way. For example, if you attach one to the network using br0.5, it will be in the bridge and in the specified VLAN.

That is similar to my original setup. What is wrong with my current one? Why pull Bridge VLAN filtering into the mix again?

I think your current setup will work, I just don't find it as a proper solution. Either do VLAN filtering for eth0 on the bridge interface or do it as subinterfaces of eth0.

I made the changes to /etc/config/network to use VLAN filtering on eth0/bridging setup. But I am unable to start the container.

When I try:

# lxc-start -n pihole -F
lxc-start: pihole: network.c: lxc_ovs_attach_bridge: 2781 Failed to attach "vethUaru4x" to openvswitch bridge "br0.4": lxc-start: pihole:
lxc-start: pihole: network.c: netdev_configure_server_veth: 715 Operation not permitted - Failed to attach "vethUaru4x" to bridge "br0.4"
lxc-start: pihole: network.c: lxc_create_network_priv: 3419 Operation not permitted - Failed to create network device
lxc-start: pihole: start.c: lxc_spawn: 1826 Failed to create the network
lxc-start: pihole: start.c: __lxc_start: 2053 Failed to spawn container "pihole"
lxc-start: pihole: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: pihole: tools/lxc_start.c: main: 313 Additional information can be obtained by setting the --logfile and --logpriority options
/etc/config/network (using br0+VLAN filtering)
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd1a:184b:b879::/48'
	option packet_steering '1'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'wg0'
	option ipv6 '0'

config device
	option name 'br0.1'
	option type '8021q'
	option ifname 'br0'
	option vid '1'
	option ipv6 '0'

config device
	option name 'br0.3'
	option type '8021q'
	option ifname 'br0'
	option vid '3'
	option ipv6 '0'

config device
	option name 'br0.4'
	option type '8021q'
	option ifname 'br0'
	option vid '4'
	option ipv6 '0'

config device
	option name 'br0.5'
	option type '8021q'
	option ifname 'br0'
	option vid '5'
	option ipv6 '0'

config device
	option type 'bridge'
	option name 'br0'
	list ports 'eth0'
	option ipv6 '0'

config bridge-vlan
	option device 'br0'
	option vlan '1'
	list ports 'eth0:t'
	option ipv6 '0'

config bridge-vlan
	option device 'br0'
	option vlan '3'
	list ports 'eth0:t'
	option ipv6 '0'

config bridge-vlan
	option device 'br0'
	option vlan '4'
	list ports 'eth0:t'
	option ipv6 '0'

config bridge-vlan
	option device 'br0'
	option vlan '5'
	list ports 'eth0:t'
	option ipv6 '0'

config interface 'wan'
	option device 'eth1'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'

config interface 'lan'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'
	option device 'br0.1'

config interface 'guest'
	option proto 'static'
	option ipaddr '10.9.7.1'
	option netmask '255.255.255.0'
	option device 'br0.3'

config interface 'lxc'
	option proto 'static'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'
	option device 'br0.4'

config interface 'iot'
	option proto 'static'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'
	option device 'br0.5'

config interface 'wg0'
	option proto 'wireguard'
...
LXC config
# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = aarch64

# Container specific configuration
lxc.rootfs.path = dir:/srv/lxc/pihole/rootfs
lxc.uts.name = pihole

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br0.4
lxc.net.0.flags = up
lxc.net.0.ipv4.address = 10.0.4.250/24
lxc.net.0.ipv4.gateway = 10.0.4.1

Either do VLAN filtering for eth0 on the bridge interface or do it as subinterfaces of eth0.

The other option you mentioned might be appealing, but what configuration in LXC would you recommend to do so?

Ok, now I see what you're trying to do. Change interface name of lxc to anything but br0 & eth0 (to avoid confusion), for example: lxc.net.0.link = lxc

Add lxc interface to the bridge and set up VLAN 4 tagged for this interface. Use br0.4 on the network configuration at the Interfaces page.

Edit: Actually, use lxc interface on the network at the Interfaces page.
Set up both eth0 & lxc interface VLAN 4 tagged.

This way, anything at VLAN 4 connecting to your Pi (over eth0) and your router itself can reach lxc.

Can you tell what do you use lxc for?
Do you need it to be in the same VLAN as something connecting to your Pi? Or do you just need to route to lxc?