I would like to share my experimental OpenWrt Docker Image. I created it to run OpenWrt on a Raspberry Pi 5 and on a Weidmueller UC20-M4000 PLC with u-OS. But I tested it also with a Ubuntu 23.10 and Ubuntu 24.04 host so this image should be generic.
Main features:
KVM acceleration via qemu
Supports CPU architectures x86_64 and aarch64
Web-based viewer for tty console
Attaches two physical Ethernet interfaces (LAN/WAN) exclusively into the docker container
Create virtual LAN between OpenWrt and host system (LAN only)
USB passthrough e.g. for modem or Wi-Fi
Automatic config migration when OpenWrt container is updated (experimental)
I would like to get your feedback on that! For example is there a general interest in a OpenWrt container or what do you think to run OpenWrt on an embedded device via virtualization?
There is certainly no interest in running OpenWrt in a container, as that is broken by design - with grave functionality and serious security issues. OpenWrt does hard-depend on un-/loading kernel modules at will, configuring sysctl settings and querying the (networking-) hardware (this hardware may be virtual, but it needs to be in charge of it, without the containerization filtering or failing those actions).
containerization, like docker, lxc, lxd, virtuozzo, OpenVZ, v-server: no
full system virtualization, like qemu-kvm, virtualbox, hyper-v, vmware: yes
real hardware/ bare-iron: yes
This has been discussed many times before, so the above is the short form of it.
tl;dr: running OpenWrt in a container is broken and insecure.
…and importing the ready-made OpenWrt disk images into the virtualization option of your choosing shouldn't take yo more than 2 minutes either, I certainly wouldn't trust any third party with that for the sake of convenience.
--
I'm feeling like a broken record already, this is the third time this has been raised this week alone…
I understand your point but inside the Docker container I run qemu-kvm so technical speaking I do a full system virtualization and I use the ready made OpenWrt images for that.
Do you still think that this is "broken and insecure"?
I think I need to explain a little but more i detail why I created this Docker image.
Two years ago I started experimenting with OpenWrt in LXC and Docker (just the rootfs). So I used the host Linux kernel instead of the OpenWrt Linux kernel. Surprisingly a lot of things just worked fine but I ran into limitations pretty soon. For me it was mandatory to connect a 4G/5G modem to it. With the host kernel it was not possible because of many different reasons. I didn't check this thread "Running OpenWrt in a Docker container" in detail but I think the limitations are discussed there.
With this experience I was looking for an alternative approach. After tests with QEMU and the QEMUs USB pass-through feature I was successful connecting a 4G/5G modem to OpenWrt. In this case I used the OpenWrt Linux kernel of course.
You may ask why I still want to use Docker? In the industrial world you have Linux systems where it is no possible to install software via a package manager (like apt or opkg for example). Usually the base Linux OS is packed as a firmware and runs in read-only mode. Because customers are requesting customer specific features many different vendors started to add Docker to the base system. Yes, Docker is running on DIN rail mounted embedded systems! In my case I wanted to run OpenWrt as a software router on top of the base system the only way is to pack everything into a Docker image. So I'm using Alpine as image base layer and install qemu into the image and run OpenWrt in qemu. With USB pass-though, macvtap and some Linux network namespace magic it is possible to run OpenWrt completely isolated from the host system.
Because OpenWrt is running as a VM it can be used without any limitations!
It is a pleasure that you checked the code . Honestly!
But let me add some comments:
I added it because opkg gives me the error SSL error: ERROR - Generic error and could not find a reason to it. With --no-check-certificate it is working fine. If you have a solution let me know it! The current stable version 23.05.5 doesn't have this issue.
Full log:
/ # opkg update
Downloading https://downloads.openwrt.org/snapshots/targets/x86/64/packages/Packages.gz
SSL error: ERROR - Generic error
*** Failed to download the package list from https://downloads.openwrt.org/snapshots/targets/x86/64/packages/Packages.gz
You are right that not to check the web certificate is insecure. I'm totally unhappy doing this but since this it is just for the snapshots I'm fine for it for the moment. Furthermore, I think the risk is pretty low that a men-in-the-middle attacker is sitting between the Azure cloud and the OpenWrt web server. But this is a different story and we don't want to discuss it here.
In the latest versions, I changed from two images (kernel and ext4 rootfs) to just one image generic-squashfs-combined.img. As a positive side effect, the --no-check-certificate is gone!
When I implemented the automated tests for the OpenWrt Docker image, it came in my mind that this could be also interesting for the OpenWrt developers. In my case I raise up 3 VMs (WAN VM, router VM with OpenWrt docker and LAN VM) and I'm running my tests completely cloud based in Github Actions. With this approach, a bunch of hardware independent features OpenWrt can be tested.
That sounds cool. Because I wanted to contribute just some minor patches to packages and realized there is no standardized way to test that in the repo nor the documentation.
I just want to announce that I added two new features to the openwrt Docker image.
1. PCI and PCIe pass-through
# PCI and PCIe pass-through
# Host system requirements
# - IOMMU CPU virtualization feature (e.g. Intel VT-d) has to enabled in bios
# - Host kernel has to switch on intel_iommu=on or amd_iommu=on
# - Host kernel module 'vfio-pci' has to exists
# - Ubuntu 24.04 is working by default
# Format <bus:device.function> like 'lspci' is given
PCI_1: "00:1f.6"
2. User command or script to run after OpenWrt is booted
# User command or script to run after OpenWrt is booted
# Please note that all commands and script will be executed as root
#OPENWRT_AFTER_BOOT_CMD: "a=$$((1+1)); echo The result of 1+1 is $${a}" # Shell script example
OPENWRT_AFTER_BOOT_CMD: "cat /etc/banner" # Custom commands example
I have a modem connected via USB to OpenWRT, is it possible that it will work with this container? On GitHub I noticed you only listed 2 supported USB devices - Are those just ones you have tested personally?
Thanks in advance.
For the listed two USB devices, the Linux drivers are installed by default. Your modem will also work, but you need to install the drivers/kernel modules by yourself.
See this comment by @slh regarding USB devices passed from host to VMs. In short, you should expect issues, unless you do PCIe pass-through on the USB controller, in which case you may still get issues (or not). You will just have to try and see.
P.s. I am not speaking from personal experience, just bringing to your attention the opinion expressed by someone else.
My personal experience is different from @slh. Most of the USB devices will work. I'm using USB pass-through regularly (USB to Ethernet, USB to Wi-Fi, USB to cellular network incl. 5G) with virtual machines (VirtualBox and qemu).
During my daily work, I gained the following experience:
Test it, most of the USB devices will work.
On some USB devices, FW uploading is not working - that's a hard blocker, thus this device cannot be used.
Especially out-of-tree Linux kernel drivers are affected.
If a device is working, it will work stable.
Because all low-level USB packages are forwarded in software, there is a significant performance lost. That means you will most likely not getting Gigabit speeds.
But also with PCIe pass-through (IOMMU) I discovered that the most PCIe devices will work but not all.
But biggest issue is not non-working USB or PCIe devices. This issue is finding the right OpenWrt driver. Because this container is generic, the most Linux drivers (kernel modules) are not installed, and thus the devices will not work. Furthermore, OpenWrt's support for USB Wi-Fi is limited compared with Ubuntu. Here is an example from a user (https://github.com/AlbrechtL/openwrt-docker/issues/31). Here, it helps to use an Internet search engine with the USB device ID and vendor ID.
Just guessing here, but what may be affecting the outcome is the hardware in the host. Possibly this works better on a more capable x86 host than on a less capable RISC host.
. Honestly!
.
