Running OpenWrt in a Docker container

fir3drag0n · June 27, 2021, 12:38pm

After restarting, I cannot see any SSID anymore, but the docker is running. What can I do now?

Sun Jun 27 12:29:41 2021 daemon.info dnsmasq-dhcp[1696]: read /etc/ethers - 0 addresses


Sun Jun 27 12:33:17 2021 daemon.warn odhcpd[1260]: A default route is present but there is no public prefix on lan thus we don't announce a default route!


Sun Jun 27 12:34:08 2021 daemon.info procd: Instance sysntpd::instance1 s in a crash loop 6 crashes, 42 seconds since last crash


Sun Jun 27 12:38:41 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 13: can't create /proc/sys/net/ipv6/conf/eth1/hop_limit: Read-only file system


Sun Jun 27 12:38:41 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 14: can't create /proc/sys/net/ipv6/conf/eth1/mtu: Read-only file system


Sun Jun 27 12:39:08 2021 daemon.warn odhcpd[1260]: A default route is present but there is no public prefix on lan thus we don't announce a default route!


Sun Jun 27 12:47:15 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 13: can't create /proc/sys/net/ipv6/conf/eth1/hop_limit: Read-only file system


Sun Jun 27 12:47:15 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 14: can't create /proc/sys/net/ipv6/conf/eth1/mtu: Read-only file system


Sun Jun 27 12:48:17 2021 daemon.warn odhcpd[1260]: A default route is present but there is no public prefix on lan thus we don't announce a default route!


Sun Jun 27 12:54:36 2021 daemon.warn odhcpd[1260]: A default route is present but there is no public prefix on lan thus we don't announce a default route!


Sun Jun 27 12:55:18 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 13: can't create /proc/sys/net/ipv6/conf/eth1/hop_limit: Read-only file system


Sun Jun 27 12:55:18 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 14: can't create /proc/sys/net/ipv6/conf/eth1/mtu: Read-only file system


Sun Jun 27 13:00:48 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 13: can't create /proc/sys/net/ipv6/conf/eth1/hop_limit: Read-only file system


Sun Jun 27 13:00:48 2021 daemon.notice netifd: wan6 (1463): /lib/netifd/dhcpv6.script: line 14: can't create /proc/sys/net/ipv6/conf/eth1/mtu: Read-only file system


Sun Jun 27 13:02:32 2021 daemon.warn odhcpd[1260]: A default route is present but there is no public prefix on lan thus we don't announce a default route!

jmandawg · June 27, 2021, 5:38pm

So i posted above about getting "nat loopback" working using hairpin, unfortunatley this seems to cause issues with DHCP. Not exactly sure why though.

jmandawg · June 28, 2021, 12:49pm

@fir3drag0n what do you mean by restart, restart container or restart computer?

I think if you restart docker container it gets new PID so wlan0 is not attached anymore to netns anymore.

Are you using the "make clean" and "make run" to start stop it?

fir3drag0n · June 28, 2021, 1:43pm

Restart Container. How do I attach WiFi again?

jmandawg · June 28, 2021, 2:55pm

@fir3drag0n You have to use the script to start stop the container, instead of restart use the "make clean" and "make run" it should clean up and tear down the netns correctly.

smo · July 10, 2021, 11:46pm

Has Anyone successfully achieved Ap+sta in openWRT Docker on A Respberry Pi using this script? I have forced a second wlan device before running, but still no cigar

jmandawg · August 18, 2021, 10:16pm

@oofnik couple quick notes/questions:

one thing i've noticed is the ntpd doesn't work because I suspect we don't have permissions to change the system time in docker. I see these errors in the log:

 procd: Instance sysntpd::instance1 s in a crash loop 25 crashes, 1 seconds since last crash

Maybe we should disable ntpd. or pass the --cap-add SYS_TIME to docker.

I've mounted -v /mnt/local-drive/local-folder:/etc/config and it seems to be persisting settings well, not sure if there are any other folders that need mounted.
I had to set the br-lan to promiscuous mode for nat loopback to work correctly (access external IP from inside LAN), does anyone know if this will cause any issues.

My setup is a mini PC and I bought one of these usb dual nic adapters off ebay:

I pass through both nics into the network namespace, and i setup my old R7000 as an access point. Everthing seems to be working good so far. Never seen a router boot up so fast.

Thanks for all the work on this.

oofnik · August 27, 2021, 5:17am

Thank you for the feedback!

I noticed the ntpd issue but it didn't bother me enough to do anything about it. I'll have to read about the security implications of adding SYS_TIME capability to the container since this container will in many cases (including mine) be directly exposed to the internet. We're already adding NET_ADMIN and NET_RAW so it shouldn't be too problematic.

Using a bind mount for the config is a good idea. I considered this approach instead of docker cp'ing the templated config into the container on creation. Some time has passed since I worked on the project so I'll have to take a closer look.

Re: promiscuous mode on br-lan being required for loopback connections, that's interesting, I don't remember having that problem, but I also don't need loopback. Looking at the config for the LAN bridge in my configuration now (inside the container namespace) I see that promiscuity is off. Traceroute reports a couple of hops to reach my external IP from inside the LAN, but it does eventually get there.

Oni07R · September 6, 2021, 9:54am

THANK YOU!
I had to shout it out. I wanted to make my custom home (cabled) router and your docker image works perfectly!

I have one bug to report though, running on RPi 4B:

busybox starts the command askfirst, whos only purpose is to type "Press Enter to activate this console".
The funny part is that when you connect with ssh to openwrt, it doesn't even prompt that sentence.
But askfirst process takes up 100% CPU (single core)! I was absolutely stunned when I saw this.

I solved this by editing /etc/inittab and disabling three lines where askfirst is launched during boot

::sysinit:/etc/init.d/rcS S boot
::shutdown:/etc/init.d/rcS K shutdown
# ttyAMA0::askfirst:/usr/libexec/login.sh
# ttyS0::askfirst:/usr/libexec/login.sh
# hvc0::askfirst:/usr/libexec/login.sh

Oni07R · November 4, 2021, 9:54am

I've noticed that there is no flow offloading in Firewall settings.
I read in another post that this is caused by the non-openwrt kernel you are using for this docker image.

Can and will it be implemented in a future release?
Or should I switch to KVM if I need flow offloading?

slh · November 5, 2021, 12:57am

As explained there, the problem are missing features in your host kernel - none of OpenWrt's business.

Oni07R · November 5, 2021, 8:45am

Can and will it be implemented in a future release?

As explained there, the problem are missing features in your host kernel - none of OpenWrt's business.

With "future release" I meant future release of oofnik's docker image, not future release of OpenWRT.

xorbug · November 6, 2021, 12:34pm

No container image can fix your host kernel features.
Also see the excellent @slh answer about mainline SFO support here: Software flow offloading for containerized OpenWrt - #2 by slh

Oni07R · December 17, 2021, 5:14pm

Sorry, I read your answer.
Thank you for the link, his/her post was very interesting.

averyfreeman · February 2, 2022, 9:43pm

Sorry to bump the necrothread, but I found it through google looking for an error code from systemd-nspawn: Failed to resize receive buffer: Operation not permitted

I thought I might try and run OpenWRT as a gateway for Open_vSwitch in systemd-container because pretty accessible from the outside still, but am running up against this error when trying to boot.

However, in regards to your project, I remember seeing a person who detailed setting up OpenWRT on raspberry Pi using LXC, and wanted to share it with you: http://www.makikiweb.com/Pi/lxc_openwrt.html

Oh OK the forum is mentioning you already got a copy of this here: Running OpenWrt in a Docker container - #49 by fvlaicu

Did you manage to make any progress on your docker container?

Thanks

PRodri · February 22, 2022, 6:21pm

Hello @oofnik

First I'would like to share my appreciation for the great work at this project!
I've been using it as main router with ISP / lan & docker netorks connection, has been quiet stable and performance is great.
From past two weeks, I've been trying to extend its functions to firewall a few IOT wifi devices using an wifi usb dongle, in which I'vent suceed yet:

I've started with tp TPL anchor T2U, then TL-WN722N which didnt supported "set_wiphy_netns". On my last order I've got the TL Archer T3U (Realtek RTL8812BU) which is the closer chipset I could find to the one you are using. Ref: https://wikidevi.wi-cat.ru/TP-LINK_Archer_T3U_Plus
Drivers: https://github.com/cilynx/rtl88x2bu.git

Nevertheless, it stills not ok, so let me try to describe:

If i run "make run" with .conf file to include the wifi, I can see the docker boots, but as soon as I try to open the GUI both the docker & server (debian10) crashes, requiring an hard reboot.

If I run "make run", removing the wifi config at /etc/config/wireless.tpl (No wireless config at owrt)

The wlan / phy0 is properly migrated into the docker pid and gone from the server as expected , I can even see the neighbour network at owrt GUI. (Attached)

root@vrouter:~#  ip link show wlx5ca6e639da36
3: wlx5ca6e639da36: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 5c:a6:e6:39:da:36 brd ff:ff:ff:ff:ff:ff

root@vrouter:~# iw phy0 info
Wiphy phy0
	wiphy index: 0
	max # scan SSIDs: 9
	.....

Although as soon as I try commit the wifi phy = phy0, both server and docker crashes:

config 'wifi-device'    'radio0'
    option 'type'       'mac80211'
    option 'phy'        phy0

Apologies, for the extent of details, your support would be highlihy apreciated here.
Any additonal logs/info required I'm keen to share, just let me know.

mikma · February 22, 2022, 7:37pm

I'm wondering if it's possible to use OpenWrt's wifi packages (hostapd etc) with a kernel that doesn't have the patches from OpenWrt.

PRodri · March 3, 2022, 11:28am

Hi Mikma, it is, I managed to get it working, using diferente drivers, but the connection was unstable. Therefore implemented an easier solution, where hostpad (full wifi) is managed by the host, and DHCP managed by docker openwrt. Using a macvlan to bundle the "usb" interface, as "parent" into the docker nertwork. Working flawlessly till now, if anyone following the same need more details, just let me know and I will be happy to help. Thanks

jmandawg · April 5, 2022, 10:08am

Is it possible to enable SQM inside the docker image?

slh · April 6, 2022, 2:00am

Please read the thread, even if it's a little long, many of these topics have been covered already.

It's difficult, mostly for two reasons:

on the one hand your host kernel needs to provide all the features your docker container needs (and expects! OpenWrt relies on certain kernel features and preconfigurations to exist, if that is no longer the case, as it's not running on its own kernel/ sysctl configuration anymore, you will see breakage of varying extents)
you do need to duplicate quite a few settings between docker host (hypervisor) and your container, adding significant complexity to your configuration (and even more for future configuration changes).
on the other hand you would have to grant your container a level of host access (into its networking subsystem) that goes far beyond what's considered to be 'safe' in a docker environment.

If you can, use full system virtualization (kvm) for OpenWrt, instead of docker - or run it on the bare iron.