Difference between DSA and driver-level VLANs in the context of RPi4 (no switch)?

I see now that you edited your reply. I created lxcbr0 in luci and I bridged it to eth0 like this:

config device
  option type 'bridge'
  option name 'lxcbr0'
  list ports 'eth0'
  option ipv6 '0'

Is that correct?

My lxc config is now the following but it does not start:

lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.ipv4.address = 10.0.4.250/24
lxc.net.0.ipv4.gateway = 10.0.4.1

Yet:

# lxc-start -n pihole -F
lxc-start: pihole: network.c: netdev_configure_server_veth: 708 No such file or directory - Failed to attach "veth0ZzBNB" to bridge "lxcbr0", bridge interface doesn't exist
lxc-start: pihole: network.c: lxc_create_network_priv: 3419 No such file or directory - Failed to create network device
lxc-start: pihole: start.c: lxc_spawn: 1826 Failed to create the network
lxc-start: pihole: start.c: __lxc_start: 2053 Failed to spawn container "pihole"
lxc-start: pihole: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: pihole: tools/lxc_start.c: main: 313 Additional information can be obtained by setting the --logfile and --logpriority options

I must also need to create an interface in LuCI for the bridge to use?

Don't add eth0. Just an empty bridge.

This looks good.
Did you do this step below?

That's a network, not an interface. I suppose you mean what I just quoted above, so do that.

I removed the list ports so it now reads:

config device
  option type 'bridge'
  option name 'lxcbr0'
  option ipv6 '0'
  option bridge_empty '1'

I'm not sure what you mean when you say "create networking using lxcbr0.."

In LuCi, Network>Interfaces has a button "Add new interface..."
In LuCi, Network>Devices has a button "Add device configuration..."

EDIT: Ah! I am calling that an interface due to the naming in the button!

I know that LuCI represents confusing networking terms. I'm actually in the middle of writing a mail to correct them. For now, think of "Interfaces" as "Networks", and "Devices" as "Interfaces".

So,

That's "network", not "networking".

1 Like

That is confusing. I think I have it setup as you recommend. The container runs, I can ssh into it, but DNS resolution is not working.

/etc/config/network
config interface 'loopback'
  option device 'lo'
  option proto 'static'
  option ipaddr '127.0.0.1'
  option netmask '255.0.0.0'

config globals 'globals'
  option ula_prefix 'fd1a:184b:b879::/48'
  option packet_steering '1'

config device
  option name 'eth0'
  option ipv6 '0'

config device
  option name 'eth1'
  option ipv6 '0'

config device
  option type 'bridge'
  option name 'lxcbr0'
  option ipv6 '0'
  option bridge_empty '1'

config device
  option name 'wg0'
  option ipv6 '0'

config device
  option name 'eth0.1'
  option type '8021q'
  option ifname 'eth0'
  option vid '1'
  option ipv6 '0'

config device
  option name 'eth0.3'
  option type '8021q'
  option ifname 'eth0'
  option vid '3'
  option ipv6 '0'

config device
  option name 'eth0.5'
  option type '8021q'
  option ifname 'eth0'
  option vid '5'
  option ipv6 '0'

config interface 'wan'
  option device 'eth1'
  option proto 'dhcp'
  option peerdns '0'
  option delegate '0'
  list dns '1.1.1.1'
  list dns '1.0.0.1'

config interface 'lxc'
  option proto 'static'
  option device 'lxcbr0'
  option ipaddr '10.0.4.1'
  option netmask '255.255.255.0'

config interface 'lan'
  option proto 'static'
  option ipaddr '10.9.8.1'
  option netmask '255.255.255.0'
  option device 'eth0.1'

config interface 'guest'
  option proto 'static'
  option ipaddr '10.9.7.1'
  option netmask '255.255.255.0'
  option device 'eth0.3'

config interface 'iot'
  option proto 'static'
  option ipaddr '10.9.5.1'
  option netmask '255.255.255.0'
  option device 'eth0.5'

config interface 'wg0'
  option proto 'wireguard'
...
/etc/config/firewall

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule 'wg'
	option name 'Allow-WireGuard'
	option proto 'udp'
	option target 'ACCEPT'
	option src 'wan'
	option dest_port '4500'

config include
	option path '/etc/firewall.user'

config zone 'lan'
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'lan'

config zone
	option name 'guest'
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'REJECT'
	list network 'guest'
	list network 'wg0'

config zone
	option name 'lxc'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'lxc'

config zone 'wan'
	option name 'wan'
	option output 'ACCEPT'
	option masq '1'
	option mtu_fix '1'
	option input 'DROP'
	option forward 'DROP'
	list network 'wan'

config rule
	option src 'guest'
	option target 'ACCEPT'
	option name 'guest dhcp and dns'
	list proto 'tcp'
	list proto 'udp'
	option dest_port '53 67 68'

config forwarding 'lan_wan'
	option src 'lan'
	option dest 'wan'

config forwarding
	option src 'guest'
	option dest 'wan'

config rule
	option src 'iot'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	option dest_port '53 67 68'
	option name 'iot dhcp and dns'

config rule
	list proto 'udp'
	option src 'lxc'
	option dest_port '53'
	option target 'ACCEPT'
	option name 'pi-hole-dns lxc to input'

config rule
	list proto 'udp'
	option src 'guest'
	option dest 'lxc'
	option dest_port '53'
	option target 'ACCEPT'
	option name 'pi-hole-dns guest to lxc'

config zone
	option name 'iot'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'REJECT'
	list network 'iot'

config forwarding
	option src 'lan'
	option dest 'guest'

config forwarding
	option src 'lan'
	option dest 'iot'

config forwarding
	option src 'lxc'
	option dest 'guest'

config forwarding
	option src 'guest'
	option dest 'lxc'

config forwarding
	option src 'lan'
	option dest 'lxc'

I have to be missing something simple.

The network configuration looks good to me. It's kind of hard to read the firewall configuration though. That's about all I can do. And make sure /etc/resolv.conf inside the container has a correct nameserver defined.

Within the container /etc/resolv.conf is good:

nameserver 1.1.1.1
nameserver 1.0.0.1

Here is a screenshot of the firewall forwarding page. Seems right to me.

No idea why DNS resolution is failing inside the container.

If you can ping 1.1.1.1 from inside the container, there must be something wrong with the container itself or firewall rules allowing UDP/TCP port 53 for the lxc zone.

I cannot ping external addresses from within the container, just the router and other devices behind it. Very confusing.

From container:

% ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
From 10.0.4.1 icmp_seq=1 Destination Port Unreachable

% ping 10.9.8.1
PING 10.9.8.1 (10.9.8.1) 56(84) bytes of data.
64 bytes from 10.9.8.1: icmp_seq=1 ttl=64 time=0.121 ms

AH! You're also supposed to allow forwarding to wan zone (or wireguard if you use it for internet access) for lxc zone. Or did you only allow port 53 to be forwarded?

1 Like

I think I've been focused on this for too long... losing my common sense and objectivity. You are of course 100% correct.

I have a few rules to allow the lxc to get to the router and to the guest zone and I think it's function:

Deep and sincere THANK YOU for all the time you spent with me on this. Once my head clears, I think I might write-up this for others.

1 Like

You're very welcome. It was entertaining, to say the least. Good luck with your future endeavors!

1 Like

Acknowledgement to @arinc9. For others wanting this setup, see below.

  • RPi4 primary router/firewall using VLANs (my setup has WiFi provided by a dumb AP and uses these VLANs to maintain isolation between three networks but that is outside the scope of this) via subinterfaces
  • LXC running pi-hole in my case but can be anything obviously (great post on LXC setup)
  • Dumb AP (VLAN-aware) connected for WiFi (outside of scope of this but mentioned for clarity
/etc/config/network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd1a:184b:b879::/48'
	option packet_steering '1'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'wg0'
	option ipv6 '0'

config device
	option type 'bridge'
	option name 'lxcbr0'
	option ipv6 '0'
	option bridge_empty '1'

config device
	option name 'eth0.1'
	option type '8021q'
	option ifname 'eth0'
	option vid '1'
	option ipv6 '0'

config device
	option name 'eth0.3'
	option type '8021q'
	option ifname 'eth0'
	option vid '3'
	option ipv6 '0'

config device
	option name 'eth0.5'
	option type '8021q'
	option ifname 'eth0'
	option vid '5'
	option ipv6 '0'

config interface 'wan'
	option device 'eth1'
	option proto 'dhcp'
	option peerdns '0'
	option delegate '0'
	list dns '1.1.1.1'
	list dns '1.0.0.1'

config interface 'lxc'
	option device 'lxcbr0'
	option proto 'static'
	option ipaddr '10.0.4.1'
	option netmask '255.255.255.0'

config interface 'lan'
	option device 'eth0.1'
	option proto 'static'
	option ipaddr '10.9.8.1'
	option netmask '255.255.255.0'

config interface 'guest'
	option device 'eth0.3'
	option proto 'static'
	option ipaddr '10.9.7.1'
	option netmask '255.255.255.0'

config interface 'iot'
	option device 'eth0.5'
	option proto 'static'
	option ipaddr '10.9.5.1'
	option netmask '255.255.255.0'

config interface 'wg0'
	option proto 'wireguard'
...
# wireguard specifics omitted for privacy
/etc/config/firewall
# standard firewall rules omitted
config rule 'wg'
	option name 'Allow-WireGuard'
	option proto 'udp'
	option target 'ACCEPT'
	option src 'wan'
	option dest_port '4500'

config include
	option path '/etc/firewall.user'

config zone 'lan'
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'lan'

config zone
	option name 'guest'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'guest'
	list network 'wg0'
	option input 'REJECT'

config zone
	option name 'lxc'
	option output 'ACCEPT'
	list network 'lxc'
	option forward 'ACCEPT'
	option input 'REJECT'

config zone
	option name 'iot'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'REJECT'
	list network 'iot'

config zone 'wan'
	option name 'wan'
	option output 'ACCEPT'
	option masq '1'
	option mtu_fix '1'
	option input 'DROP'
	option forward 'DROP'
	list network 'wan'

config rule
	option src 'guest'
	option target 'ACCEPT'
	option name 'guest dhcp and dns'
	list proto 'tcp'
	list proto 'udp'
	option dest_port '53 67 68'

config forwarding 'lan_wan'
	option src 'lan'
	option dest 'wan'

config forwarding
	option src 'guest'
	option dest 'wan'

config rule
	option src 'iot'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	option dest_port '53 67 68'
	option name 'iot dhcp and dns'

config rule
	list proto 'udp'
	option src 'lxc'
	option dest_port '53'
	option target 'ACCEPT'
	option name 'pi-hole-dns lxc to input'

config rule
	list proto 'udp'
	option src 'guest'
	option dest 'lxc'
	option dest_port '53'
	option target 'ACCEPT'
	option name 'pi-hole-dns guest to lxc'

config forwarding
	option src 'lan'
	option dest 'guest'

config forwarding
	option src 'lan'
	option dest 'iot'

config forwarding
	option src 'lan'
	option dest 'lxc'

config forwarding
	option src 'lxc'
	option dest 'wan'

/srv/lxc/pihole/config
# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: --dist archlinux --release current
# Template script checksum (SHA-1): 8dff53d9a72ba3c071585c5762e2d14c57943cfa
# For additional config options, please look at lxc.container.conf(5)

# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = aarch64

# Container specific configuration
lxc.rootfs.path = dir:/srv/lxc/pihole/rootfs
lxc.uts.name = pihole

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.ipv4.address = 10.0.4.250/24
lxc.net.0.ipv4.gateway = 10.0.4.1

Networks a.k.a. Interfaces and Interfaces a.k.a. Devices:

Firewall zones and manually added traffic rules

3 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.