Running QEMU guests on OpenWrt with qemu-bridge-helper


for a project I want to host a QEMU guest on a x86 device running OpenWrt (maybe I'll write a more detailed post about what I'm doing later). Previously I often had problems setting up the network between guest and host, but I realized that it's really easy with OpenWrt and qemu-bridge-helper and I wanted to share a quick recipe.
Using that, you can connect the guest to every bridge network of your OpenWrt host (every Interface with option type 'bridge').

You need the following packages on your device: kmod-tun qemu-bridge-helper qemu-x86_64-softmmu. If your hardware supports it, also install kmod-kvm-amd or kmod-kvm-intel for better performance.
If you install these packages via opkg, reboot afterwards.
Now you're ready to run QEMU guests. In my example the guest is a Debian Stretch installation on /dev/sda, but you can use an image as well. Just use a distribution that comes with virtio drivers by default.
The complete command looks like this:

qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 2G \
    -drive file=/dev/sda,cache=none,if=virtio,format=raw \
    -device virtio-net-pci,mac=E2:F2:6A:01:9D:C9,netdev=br0 \
    -netdev bridge,br=br-lan,id=br0

Line 3 creates a virtio net device with a fixed MAC address and is told to use the network backend with id=br0, which gets created in line 4. The network backend is of type bridge and br=br-lan tells it to connect to the bridge network br-lan of the OpenWrt host.
That's it, if your guest uses DHCP it should receive an address by the server running in br-lan.

To automatically start the VM at boot and shutting it down cleanly before a reboot, I created this init script:

#!/bin/sh /etc/rc.common


start() {
qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 2G \
    -drive file=/dev/sda,cache=none,if=virtio,format=raw \
    -device virtio-net-pci,mac=E2:F2:6A:01:9D:C9,netdev=br0 \
    -netdev bridge,br=br-lan,id=br0 \
    -qmp tcp:,server,nowait \
    -daemonize &> /var/log/qemu.log

stop() {
nc localhost 4444 <<QMP 
{ "execute": "qmp_capabilities" }
{ "execute": "system_powerdown" }

sleep 10s

It uses QMP to initiate the VM shutdown.

I didn't run a benchmark yet, but the guest is performing really well so far.

iperf3 results between a separate host in my LAN and the VM are fine:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   945 Mbits/sec  345             sender
[  4]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec

The host is an Intel Celeron 2955U (2 x 1,4 GHz), so I think native Gbit speed is great for a VM guest. The CPU load is about 70% during the tests.

Hi Sebastian, thank you for your work.
Do you think is possible to compile libvirt to remote manage the vms?

Well, the OpenWrt build system ist very flexible and in theory, you can build anything with it. Considering that Alpine Linux packages libvirt, it seems to work fine with musl, too (the C library OpenWrt uses).
However, on your average distro, libvirt has quite a lot of runtime dependencies. I don't know if it can be build without them.
That said, I don't know if it's worth the effort to port libvirt to OpenWrt. If you want a system to manage multiple VMs, there are plenty of alternatives like Proxmox VE or oVirt, for example.

Sure, i know proxmox, but my dream is to ingrate a lot of new capabilities in my OpenWRT server x86.
I have found this make file about libvirt
Already tried to compile with 18.06.2 but it fails...
I think it is a good starting point

1 Like

Finally i can compile it correctly.
If someone want to try i can share my makefile

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.

include $(TOPDIR)/
include $(INCLUDE_DIR)/
define Package/libvirt
  TITLE:=Libvirt Virtualization API
  DEPENDS:= +gnutls-utils +libgnutls +libdevmapper +libxml2 +yajl +libnl +libcurl +libdbus +librpc \
            +libblkid +libavahi +libncursesw +libsasl2 +libavahi-client +libreadline
define Package/libvirt/description
	Libvirt is an open source API, daemon and management tool for managing platform virtualization.
define Build/Configure
	$(call Build/Configure/Default,--with-yajl=yes --with-macvtap=no --with-xen=no --with-xen-inotify=no --with-qemu=yes --with-uml=no --with-openvz=no --with-vmware=no --with-phyp=no --with-xenapi=no --with-libxl=no --with-vbox=no --with-lxc=yes --with-esx=no --with-hyperv=no --with-storage-dir=yes --with-storage-fs=no --disable-nls)
define Package/libvirt/install
	$(CP) $(PKG_INSTALL_DIR)/etc $(1)/
	$(CP) $(PKG_INSTALL_DIR)/usr $(1)/
	$(INSTALL_DIR) $(1)/etc
	$(INSTALL_DIR) $(1)/etc/init.d
	$(INSTALL_BIN) ./files/etc/init.d/libvirt-init $(1)/etc/init.d/
$(eval $(call BuildPackage,libvirt))

Does not work for me on 19.07..........

install: cannot stat './files/etc/init.d/libvirt-init': No such file or directory

Keep getting that error message


Inside the package folder create the folders "files/etc/init.d/"

And create the file libvirt-init

#!/bin/sh /etc/rc.common
boot() {
start() {
	# Create /var/cache/libvirt if not exists
	if [ ! -d  /var/cache/libvirt ]; then
		mkdir -p /var/cache/libvirt/qemu
	# Create /var/lib/libvirt if not exists
	if [ ! -d  /var/lib/libvirt ]; then
		mkdir -p /var/lib/libvirt/boot
		mkdir -p /var/lib/libvirt/dnsmasq
		mkdir -p /var/lib/libvirt/filesystems
		mkdir -p /var/lib/libvirt/images
		mkdir -p /var/lib/libvirt/lockd/files
		mkdir -p /var/lib/libvirt/lxc
		mkdir -p /var/lib/libvirt/network
		mkdir -p /var/lib/libvirt/qemu
		mkdir -p /var/lib/libvirt/uml
	# Create /var/log/libvirt if not exists
	if [ ! -d  /var/log/libvirt ]; then
	        mkdir -p /var/log/libvirt/lxc
        	mkdir -p /var/log/libvirt/qemu
        	mkdir -p /var/log/libvirt/uml
	# Create /var/run/libvirt if not exists
	if [ ! -d  /var/run/libvirt ]; then
	        mkdir -p /var/run/libvirt/lockd
        	mkdir -p /var/run/libvirt/lxc
        	mkdir -p /var/run/libvirt/network
        	mkdir -p /var/run/libvirt/qemu
        	mkdir -p /var/run/libvirt/uml
	# Starting libvirtd daemon
	echo "Starting libvirt daemon..."
	exec /usr/sbin/libvirtd -d
stop() {
	echo "Stoping libvirt daemon..."
	start-stop-daemon -K -p /var/run/

Trying it now with your suggestions,

Ok, I think I have it working it compiled into my new image. Do we need to upgrade libvirt or can we? I just noticed there are newer versions. Have you already tried them? Thanks for all the great work.


Did you have problems connecting virt-manager to libvirt on the router?

Acting like I dont have ssh running