LXC container networking with DSA VLANs

I have configure two VLANs on my router, here is the relevant configuration:

config device
        option name 'br0'
        option type 'bridge'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br0'
        option vlan '1'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4:t'

config bridge-vlan
        option device 'br0'
        option vlan '2'
        list ports 'lan4:t'

config interface 'lan'
        option device 'br0.1'
        option proto 'static'
        option ipaddr '192.168.255.1'
        option netmask '255.255.255.0'
        option ip6hint 'ffff'
        option ip6ifaceid '::1'
        option ip6assign '64'

config interface 'dmz'
        option device 'br0.2'
        option proto 'static'
        option ipaddr '192.168.250.1'
        option netmask '255.255.255.0'
        option ip6hint 'fffa'
        option ip6ifaceid '::1'
        option ip6assign '64'

Now I want create two LXC containers, one connected to lan (VLAN 1 br0.1) and other connected to dmz (VLAN 2 br0.2).

I have started trying to configure the LXC container connected to lan (VLAN 1 br0.1):

lxc.net.0.type = veth
lxc.net.0.link = br0.1
lxc.net.0.veth.pair = veth1

But I get these errors:

# lxc-start -n server -F
lxc-start: server: network.c: lxc_ovs_attach_bridge: 2377 Failed to attach "veth1" to openvswitch bridge "br0.1":
lxc-start: server: utils.c: run_command_internal: 1638 Failed to exec command
lxc-start: server: network.c: instantiate_veth: 330 Operation not permitted - Failed to attach "veth1" to bridge "br0.1"
lxc-start: server: network.c: lxc_create_network_priv: 3068 Operation not permitted - Failed to create network device
lxc-start: server: start.c: lxc_spawn: 1786 Failed to create the network
lxc-start: server: start.c: __lxc_start: 1999 Failed to spawn container "server"
lxc-start: server: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: server: tools/lxc_start.c: main: 314 Additional information can be obtained by setting the --logfile and --logpriority options

If I configure the LXC container to use br0 without specify VLAN, all work well but then seems connected to both VLANs (and I do not want that):

lxc.net.0.type = veth
lxc.net.0.link = br0  
lxc.net.0.veth.pair = veth1

I also tried with:

lxc.net.0.type = vlan
lxc.net.0.link = br0.1

And the LXC container starts fine but has no connectivity (Inside the container):

# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr D8:58:D7:00:60:C0  
          inet addr:192.168.255.51  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::da58:d7ff:fe00:60c0/64 Scope:Link
          inet6 addr: fd7a:6428:ac97:ffff::51/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:1284 (1.2 KiB)

# ping -c 3 192.168.255.1
PING 192.168.255.1 (192.168.255.1): 56 data bytes

--- 192.168.255.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

So, I don't know how to configure the LXC container network. Any idea?

1 Like

guest veth link to lxc only works on bridgeed host.

May be this could help :
https://wiki.debian.org/LXC/VlanNetworking

Yeah, I had already seen that link, but it looks the same as what I was testing in my previous post and it didn't work.

I will need to try more, but this seems to work:

/etc/config/network

config device
        option name 'br0'
        option type 'bridge'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br0'
        option vlan '1'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4:t'
        list ports 'veth1'

config bridge-vlan
        option device 'br0'
        option vlan '2'
        list ports 'lan4:t'
        list ports 'veth2'

/srv/lxc/containet1/config

lxc.net.0.type = veth
lxc.net.0.veth.pair = veth1
lxc.net.0.link = br0

/srv/lxc/containet2/config

lxc.net.0.type = veth
lxc.net.0.veth.pair = veth2
lxc.net.0.link = br0

Hi,

You can do it in a different way too which may be more future proof, in terms if you want to run more or dynamically create/start/stop/delete containers .

As said above you need a bridge to add veth device. So if you create two new bridges from the devices br0.1 and br0.2 respectively, assign your lan, dmz interface to them, then simply you can specify the two new bridges in the lxc.config.

config device
        option name 'br0'
        option type 'bridge'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'

config bridge-vlan
        option device 'br0'
        option vlan '1'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4:t'

config bridge-vlan
        option device 'br0'
        option vlan '2'
        list ports 'lan4:t'

### new bridge for lan
config device 
        option name 'br-lan'
        option type 'bridge'
        list ports 'br0.1'

### new bridge for dmz
config device 
        option name 'br-dmz'
        option type 'bridge'
        list ports 'br0.2'

config interface 'lan'
        option device 'br-lan' # <--- link to the new bridge
         option proto 'static'

config interface 'dmz'
        option device 'br-dmz' # <--- link to the new bridge
        option proto 'static'

then in your respective lxc.config:

lxc.net.0.type = veth
lxc.net.0.link = # br-lan or br-dmz
# and you dont even need to specify veth device name here
# lxc will create a new veth device, name dynamically
# and automatically add to the respective bridge
# lxc.net.0.veth.pair = veth1
3 Likes

Good idea, I'll try it.

the trick is to add the veth interface as a port to the device switch in

luci/admin/network/network

and then activate the vlan on the veth device

like:

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'
        list ports 'vethvm1'

and

config bridge-vlan
        option device 'br-lan'
        option vlan '10'
        list ports 'lan2:u*'
        list ports 'lan3:u*'
        list ports 'lan4:u*'
        list ports 'vethvm1:u*'

Do you have openvswitch? Do you have it properly configured? (I didn't see it in your config...)

Edit: Oh damn. I did not see that this thread was necro bumped.... 2 years old :confused:

bit off topic but i've been using systemd containers (systemd-nspawn/machinectl) without issue for a while now, even passing in wireless phy works

I took an old amlogic s905 tv box, plugged in a comfast mt7921au wireless adapter into one of the usb2 ports, downloaded ophub armbian, gutted it out to be running Arch Linux with rc-local service instead and then compiled the latest kernel. Then I compiled Openwrt armsr target, unpakced the rootfs to /var/lib/machines directory, and it worked. I had to be careful with some things like making sure openssl had no cryptodev support.

gotta do this one time

echo "console::askfirst:/usr/libexec/login.sh"  >> /var/lib/machines/owrt/etc/inittab

start commands

systemd-nspawn --personality=arm64 --private-users=no --network-interface ibr0 --network-interface lan1 --network-interface veth1 --console=passive  -bD /var/lib/machines/owrt/ &
sleep 3
owrtpid=$(machinectl show owrt | awk -F= 'FNR == 7 {print $2}')
ophydev=$(iw list | awk 'FNR == 1 {print $2}')
iw phy $ophydev set netns $owrtpid

Two vlan network interfaces, one wan, the other lan, then a veth interface so the Arch Linux host is accessible on the lan as well