Problem running lxc in OpenWrt host

Hello, I need some help with setting up a lxc container in openwrt running on a vm:
Actually, everything works fine, I used this perfect manual: https://openwrt.org/lxc_openwrt_host

I just took an existing lxc-container instead of setting up a new one. I can start this container and using it, all network configurations are working fine for about few (30-50) seconds.
I created a new bridge (br-lxc) and used the same lxc settings (like in the manual).

So now my lxc gets the correct ip address (10.0.4.15) and I can see a new dhcp device in my lxc-interface. When I try to reach (ping) the wan interface -> it works sometimes for about 30-50 seconds. When I restart the interface or change the br-lxc it works again for a short time.
Of course I checked my routing on my lxc.

If I set a tcpdump on the gateway I can even NOT see that the lxc is trying to send a icmp!
Only for the gateway itself, it works always

> default dev eth0 scope link>
> default via 10.0.4.1 dev eth0 proto dhcp src 10.0.4.15 metric 100
> 10.0.4.0/24 dev eth0 proto kernel scope link src 10.0.4.15 metric 100
> 10.0.4.1 dev eth0 proto dhcp scope link src 10.0.4.15 metric 100 
> 169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.157.227
>
> 
> config device
>         option name 'br-lan'
>         option type 'bridge'
>         list ports 'eth0'
> 
> config interface 'wan'
>         option device 'br-lan'
>         option proto 'dhcp'
> 
> config device
>         option type 'bridge'
>         option name 'br-lxc'
>         option bridge_empty '1'
>         option macaddr '00:16:bb:bb:dd:d1'
>         list ports 'vethX5JBWQ'
> 
> config device
>         option name 'lxc'
>         option macaddr '00:16:bb:bb:dd:d0'
> 
> config interface 'lxc1'
>         option proto 'static'
>         option device 'br-lxc'
>         option ipaddr '10.0.4.1'
>         option netmask '255.255.255.0'

So this is coming frequently in my syslog:

Tue Mar 29 10:05:20 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:05:20 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15 00:16:bb:bb:cc:c1
Tue Mar 29 10:05:25 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:05:25 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:05:30 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:05:30 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:05:56 2022 daemon.info dnsmasq-dhcp[14521]: DHCPREQUEST(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:05:56 2022 daemon.info dnsmasq-dhcp[14521]: DHCPACK(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1 test1
Tue Mar 29 10:06:35 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:06:35 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:06:40 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:06:40 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:06:45 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:06:45 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:06:50 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:06:50 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:06:55 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1
Tue Mar 29 10:06:55 2022 daemon.info dnsmasq-dhcp[14521]: DHCPOFFER(br-lan) 10.0.4.15  00:16:bb:bb:cc:c1
Tue Mar 29 10:07:00 2022 daemon.info dnsmasq-dhcp[14521]: DHCPDISCOVER(br-lan) 00:16:bb:bb:cc:c1

so i guess there is something wrong with my dnsmasq & dhcp server but i cant find the problem

hi,

when you restart lxc the veth interface is re-generated under a new random name by default. so adding vethX5JBWQ to br-lxc will not be a permanent solution. also if you configure lxc to use your br-lxc bridge then it should automatically add any new veth interface generated by lxc framework, it is a bit strange it does not work like this. third, config device section looks vague to me, i'd assume the name should be a direct reference to the interface it is related to: in your case it may be lxc1 rather than lxc if you mean to set mac address of your choice to lxc1 interface.

i would check the /<lxc instance>/config file too which should look similar to below:

lxc.net.0.type = veth
lxc.net.0.link = <your bridge>
lxc.net.0.flags = up
#lxc.net.0.name = eth0  # if you want to set the internal interface name within the container
lxc.net.0.veth.pair = veth_<container name> # i suggest to name you veth after the container instead of random name like vethX5JBWQ
lxc.net.0.hwaddr = xx:xx:xx:xx:xx:xx # and set the container's mac address so dhcp can offer always the same ip

Thanks for your help.
Well yes, its strange that the new "veth..." device is not added automatically.

So i tried to generate a new network device (lets say "lxc") and assigned the lxc-container with lxc.net.0.veth.pair=lxc

This works also, but the same error occurs again. After a short time, I'm not reaching any other interface (wan...)
When I restart, it keeps the settings (the lxc device in the bridge & the lxc container gets the same device) and works for a short time.

not reaching any other interface? let's wrap up:

  • you have a bridge which is used to connect lxc containers (the veth interface external leg) to
  • this bridge has an interface on top. this interface belongs to which zone?
  • what is the firewall zone config? in which direction forward is allowed for example?

ok,
let's make it simple (without the wireguard interfaces)

So I only have one lxc-container. I created the br-lxc only for this container!
This container should reach the outside over the wan interface. Of course this firewall zone is allowed to reach the wan zone.

> config defaults
>         option output 'ACCEPT'
>         option synflood_protect '1'
>         option input 'ACCEPT'
>         option forward 'ACCEPT'
> 
> config zone
>         option name 'wan'
>         option output 'ACCEPT'
>         option masq '1'
>         option mtu_fix '1'
>         option input 'ACCEPT'
>         option forward 'ACCEPT'
>         list network 'wan'
>         list network 'wan6'
> 
> 
> 
> config zone
>         option name 'lxc'
>         option input 'ACCEPT'
>         option output 'ACCEPT'
>         option forward 'ACCEPT'
>         list network 'lxc1'
> 
> config forwarding
>         option src 'lxc'
>         option dest 'wan'

hm, from fw concept point of view this looks ok (to me).

yes it's really strange.
I think that there is something wrong with my lxc setting.
I already installed a new openwrt vm tried it again (without any other interfaces) same problem.

always this problem with the dhcprequest i can see in syslog

I think i could have found the mistake, but I do not know how to solve it:
test1@eth0
Why eth0? Looks like the lxc host attaches to the eth0 device (which is connected my wan interface).

The settings for the lxc container itself are correct.

If i do not assign any pair ( lxc.net.0.veth.pair ) this will happen:
looks a bit strange for me

can you share your lxc config?

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = linux64

# Container specific configuration
lxc.rootfs.path = dir:/mnt/lxc/test1/rootfs
lxc.uts.name = test1

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = lxcbr
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:aa:aa:aa:c1
#lxc.net.0.name = eth0
lxc.net.0.veth.pair = test1

according to man " By default, lxc chooses a name for the network device belonging to the outside of the container, but if you wish to handle this name yourselves, you can tell lxc to set a specific name with the lxc.net.[i].veth.pair option (except for unprivileged containers where this option is ignored for security reasons)."

so if you set veth.pair to 'test1' i dont see how you end up with vethJDxxx ... (unless unprivileged case of course)

are you sure this is the right config file?

If i set up veth.pair=test1 of course i will NOT get a vethJDxxx....
I just also tested this option, because you mentioned it.

Unfortunate both ways are working but only for a few seconds.
What me confuses is always this: @eth0

test1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxcbr state UP qlen 1000

NAME  STATE   AUTOSTART GROUPS IPV4                        IPV6 UNPRIVILEGED
test1 RUNNING 0         -      10.20.1.19, 169.254.218.142 -    false

if you restart lxc interface do you see anything outstanding in syslog? can you verify if traffic is not leaving lxc container, or not entering into host network?

I can not explain why it mostly (works) when i restart the interface. I was trying to find out, what happens after this time (about 40 sec) and why its not working any more: all i see is the "DHCPOFFER..." logs

Yes traffic is not leaving lxc container. I was using tcpdump on the host, but I did not see anything!
But:
If I trying to ping the host (10.0.4.1) it works and the host answers. Always!
=> so "usually" it sounds for me just like a routing problem, but it isnt.

maybe someone has some ideas, let's see ...

1 Like

Does anyone knows what @eth0 means?

I guess it is the socket to which LXC will connect.
How can I change it. Of course eth0 does not make any sense, this is my wan interface.

I solved the problem :slight_smile:

The lxc-container cased the problem itself. I created this container on kubuntu_21.02.
Probably this network problem occurred after installing lxqt & vnc-server on it.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.