Running QEMU guests on OpenWrt with qemu-bridge-helper

Hey,

for a project I want to host a QEMU guest on a x86 device running OpenWrt (maybe I'll write a more detailed post about what I'm doing later). Previously I often had problems setting up the network between guest and host, but I realized that it's really easy with OpenWrt and qemu-bridge-helper and I wanted to share a quick recipe.
Using that, you can connect the guest to every bridge network of your OpenWrt host (every Interface with option type 'bridge').

You need the following packages on your device: kmod-tun qemu-bridge-helper qemu-x86_64-softmmu. If your hardware supports it, also install kmod-kvm-amd or kmod-kvm-intel for better performance.
If you install these packages via opkg, reboot afterwards.
Now you're ready to run QEMU guests. In my example the guest is a Debian Stretch installation on /dev/sda, but you can use an image as well. Just use a distribution that comes with virtio drivers by default.
The complete command looks like this:

qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 2G \
    -drive file=/dev/sda,cache=none,if=virtio,format=raw \
    -device virtio-net-pci,mac=E2:F2:6A:01:9D:C9,netdev=br0 \
    -netdev bridge,br=br-lan,id=br0

Line 3 creates a virtio net device with a fixed MAC address and is told to use the network backend with id=br0, which gets created in line 4. The network backend is of type bridge and br=br-lan tells it to connect to the bridge network br-lan of the OpenWrt host.
That's it, if your guest uses DHCP it should receive an address by the server running in br-lan.

To automatically start the VM at boot and shutting it down cleanly before a reboot, I created this init script:

#!/bin/sh /etc/rc.common

START=99
STOP=1

start() {
qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 2G \
    -drive file=/dev/sda,cache=none,if=virtio,format=raw \
    -device virtio-net-pci,mac=E2:F2:6A:01:9D:C9,netdev=br0 \
    -netdev bridge,br=br-lan,id=br0 \
    -qmp tcp:127.0.0.1:4444,server,nowait \
    -daemonize &> /var/log/qemu.log
}

stop() {
nc localhost 4444 <<QMP 
{ "execute": "qmp_capabilities" }
{ "execute": "system_powerdown" }
QMP

sleep 10s
}

It uses QMP to initiate the VM shutdown.

I didn't run a benchmark yet, but the guest is performing really well so far.

iperf3 results between a separate host in my LAN and the VM are fine:

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   945 Mbits/sec  345             sender
[  4]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec

The host is an Intel Celeron 2955U (2 x 1,4 GHz), so I think native Gbit speed is great for a VM guest. The CPU load is about 70% during the tests.

Hi Sebastian, thank you for your work.
Do you think is possible to compile libvirt to remote manage the vms?

Well, the OpenWrt build system ist very flexible and in theory, you can build anything with it. Considering that Alpine Linux packages libvirt, it seems to work fine with musl, too (the C library OpenWrt uses).
However, on your average distro, libvirt has quite a lot of runtime dependencies. I don't know if it can be build without them.
That said, I don't know if it's worth the effort to port libvirt to OpenWrt. If you want a system to manage multiple VMs, there are plenty of alternatives like Proxmox VE or oVirt, for example.

Sure, i know proxmox, but my dream is to ingrate a lot of new capabilities in my OpenWRT server x86.
I have found this make file about libvirt
http://wiki.clommunity-project.eu/soft:node-software
Already tried to compile with 18.06.2 but it fails...
I think it is a good starting point

1 Like

Finally i can compile it correctly.
If someone want to try i can share my makefile

#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk
 
PKG_NAME:=libvirt
PKG_VERSION:=1.3.5
PKG_RELEASE:=1
 
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://libvirt.org/sources/
PKG_MD5SUM:=f9dc1e63d559eca50ae0ee798a4c6c6d
PKG_INSTALL:=1
  
include $(INCLUDE_DIR)/package.mk
 
define Package/libvirt
  SECTION:=libs
  CATEGORY:=Libraries
  TITLE:=Libvirt Virtualization API
  URL:=http://libvirt.org
  DEPENDS:= +gnutls-utils +libgnutls +libdevmapper +libxml2 +yajl +libnl +libcurl +libdbus +librpc \
            +libblkid +libavahi +libncursesw +libsasl2 +libavahi-client +libreadline
 
endef
 
define Package/libvirt/description
	Libvirt is an open source API, daemon and management tool for managing platform virtualization.
endef
 
define Build/Configure
	$(call Build/Configure/Default,--with-yajl=yes --with-macvtap=no --with-xen=no --with-xen-inotify=no --with-qemu=yes --with-uml=no --with-openvz=no --with-vmware=no --with-phyp=no --with-xenapi=no --with-libxl=no --with-vbox=no --with-lxc=yes --with-esx=no --with-hyperv=no --with-storage-dir=yes --with-storage-fs=no --disable-nls)
endef
 
define Package/libvirt/install
	$(CP) $(PKG_INSTALL_DIR)/etc $(1)/
	$(CP) $(PKG_INSTALL_DIR)/usr $(1)/
 
	$(INSTALL_DIR) $(1)/etc
	$(INSTALL_DIR) $(1)/etc/init.d
	$(INSTALL_BIN) ./files/etc/init.d/libvirt-init $(1)/etc/init.d/
endef
 
$(eval $(call BuildPackage,libvirt))
1 Like