Using systemd-nspawn to run OpenWrt

Community Builds, Projects & Packages
1

This is based on amd64. Replace the rootfs image where applicable.
The networking part of OpenWrt and systemd machinectl part is left out as there's plenty of documentation available.
Note that there could be security concern with this kind of containerization approach vs qemu sort of full virtualization. Thanks to @slh for bringing this up.

# curl -O https://downloads.openwrt.org/releases/22.03.5/targets/x86/64/openwrt-22.03.5-x86-64-rootfs.tar.gz
# mkdir openwrt; cd openwrt; tar xf ../openwrt-22.03.5-x86-64-rootfs.tar.gz; cd ..
# echo "console::askfirst:/usr/libexec/login.sh" >> openwrt/etc/inittab 
# systemd-nspawn -bD openwrt
Spawning container openwrt on /root/openwrt.
Press Ctrl-] three times within 1s to kill container.
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
Please press Enter to activate this console.



BusyBox v1.35.0 (2023-04-27 20:28:15 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 22.03.5, r20134-5f15225c1e
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@openwrt:/#
2

Keep in mind that this (like lxc/ docker) is not a supported configuration for running OpenWrt. OpenWrt relies on a number of kernel patches changing kernel behaviour (basically changing a bunch of sysctl settings) and expects to be able to load- and unloaded (its own-) kernel modules as required, neither of this can work within a container (but it would work on full system virtualization, like kvm, virtualbox, hyper-v, vmware, etc.).

While you will obviously get a shell that way, many things won't work and you will tear open a number of security issues (wrong/ unexpected sysctl configuration). and cause 'quirky' behaviour for the stations behind this router.

tl;dr: don't do it, really, don't.

3

Do the same warnings also apply to the lxc guide at https://openwrt.org/docs/guide-user/virtualization/lxc ? If not, what is the difference between lxc and systemd-nspawn that makes lxc safe to use with openwrt but systemd-nspawn unsafe?

4

It applies to any container based approach that uses the host kernel, instead of virtualizing- and booting the OpenWrt image (including its kernel, procd, etc.); so -among others- lxc, lxd, docker, systemd-nspawn, virtuozzo, chroot, etc.

5

Would you be able to share what changes to sysctl settings are required to make it safe to use openwrt in lxc or systemd-spawn? I'd like to edit the wiki page if possible so others don't make the same mistake as me and assume that openwrt is officially supported in lxc because of the lack of warnings on that wiki page.

6

I'm a bit interested in this, as I tried to accomplish this long ago and failed. I have a couple questions regarding running in systemd-nspawn:

  • I recall there's an .img of OW, does that not work with machinectl?
  • what does echo "console::askfirst:/usr/libexec/login.sh" >> openwrt/etc/inittab do, is this necessary to get the OW boot?
  • you don't provide any interface argument, so if both host and OW configures the same IF, what happens? (I actually have the same doubt for all the hardware arguments for systemd-nspawn/machinectl, I'll be very grateful if you can teach me on that topic)

Congrats on getting this to work.

7

I think it will work(didn't test it), but not comfortable as just decompress rootfs on host file system.

No, OW will boot without this thing, but if you want to get an shell while it boots, you need to add this to let you login.

Short answer: You can't.
By default, systemd-nspawn didn't allow containers to manage networks, unless you give it an private network namespace.
Will, i think you can give OW cap sys admin directly, then it will be like...using systemd-netwokd and networkmanager at the same time?

8

Then is there any supports for systemd-vmspawn? It will be nice if OpenWRT in vmspawn can manage by machinectl.

9

No.

nspawn is a container (respectively a slightly beefed up chroot), not a VM.

10

No, i mean "systemd-vmspawn", not typo for nspawn, which is also an systemd tool but run subsystem in qemu(/kvm) and not use namespaces

11

You won't get a different answer by re-iterating the question.

Take a step back and answer it yourself.
What is "systemd-vmspawn", a chroot/ container or a VM? There you have your answer.

12

Interesting. Wish there were more details on this "In many ways it is similar to systemd-nspawn(1), but launches a full virtual machine instead of using namespaces." Personally I've opted the lxc way. See this

NAME
       systemd-vmspawn - Spawn an OS in a virtual machine

SYNOPSIS

       systemd-vmspawn [OPTIONS...] [ARGS...]

DESCRIPTION
       systemd-vmspawn may be used to start a virtual
       machine from an OS image. In many ways it is similar
       to systemd-nspawn(1), but launches a full virtual
       machine instead of using namespaces.

       File descriptors for /dev/kvm and /dev/vhost-vsock
       can be passed to systemd-vmspawn via systemd's native
       socket passing interface (see sd_listen_fds(3) for
       details about the precise protocol used and the order
       in which the file descriptors are passed), these file
       descriptors must be passed with the names "kvm" and
       "vhost-vsock" respectively.

       Note: on Ubuntu/Debian derivatives systemd-vmspawn
       requires the user to be in the "kvm" group to use the
       VSOCK options.
1 Like