OpenWrt QEMU/KVM Host on a Pi 4B

I was using this part of the documentation as a starting point: opkg install kmod-tun qemu-bridge-helper qemu-x86_64-softmmu kmod-kvm-intel

Unfortunately it seems that most of those packages aren't available for the ARM target architecture for the 4b. So my next thought was to attempt compiling it following this: https://www.qemu.org/download/#source

Unfortunately I ran into another problem there with the version of Python that is built for this image of openwrt: https://downloads.openwrt.org/releases/21.02.0/targets/bcm27xx/bcm2711/openwrt-21.02.0-bcm27xx-bcm2711-rpi-4-ext4-factory.img.gz

Would anyone have any input on how to get this working on a Pi using OpenWRT? My goal was to have OpenWRT using 1 core and 1GB of RAM for the main OS< and then run a ubuntu server on the remaining resources.

Thanks!

Start with https://github.com/openwrt/packages/tree/master/utils/qemu - you're probably the first to try this on the RPi.

disregard

if you install master you can give this a go;

http://rpi4.wulfy23.info/utilities/qemu.tar.gz

[root@dca632 /usbstick/_QEMU-2022 55°]# /usr/bin/qemu-aarch64 --help
usage: qemu-aarch64 [options] program [arguments...]
Linux CPU emulator (compiled for aarch64 emulation)

Options and associated environment variables:

Argument             Env-variable      Description
-h                                     print this help
-help                                  
-g port              QEMU_GDB          wait gdb connection to 'port'

[...]

I'm a little unclear as to the link you shared. Does installing that package require rebuilding a fresh image rather than the prebuilt ones?

I tried to run that utility and I ran into an error. It installed just fine and ran the help line you did okay. But when I tried to run it on the image, ./usr/bin/qemu-aarch64 ubuntu-20.04.3-preinstalled-server-arm64\+raspi.img.xz it responds with: "./usr/bin/qemu-aarch64 ubuntu-20.04.3-preinstalled-server-arm64+raspi.img.xz "

This could be my own mistake as it's the first time I'm trying qemu from the CLI.

1 Like

ok... disregard for now then... if you come across a list of needed packages and a nice one liner to fire up from command line let me know an I can test it some more...

see: https://azeria-labs.com/emulate-raspberry-pi-with-qemu/ etc. etc.

I ran through the initial steps on setting up a build system. updated and installed all the feeds. When I run find ./ -name qemu in the working repository it does find it in the following places:

./package/feeds/packages/qemu
./feeds/packages/utils/qemu

But when I go ahead to run make menuconfig I can't seem to find qemu under any of the options. Does it just get built into the image by default?

there is nothing currently packaged for this ARCH in OpenWrt
(only visibly when x86_64 is selected or similar)

QEMU_DEPS_IN_GUEST := @(TARGET_x86_64||TARGET_armvirt||TARGET_malta)
QEMU_DEPS_IN_HOST := @(TARGET_x86_64||TARGET_sunxi)

Technically speaking it does not require building a complete firmware image, but doing it that way would still be the easiest approach.

Qemu needs quite a lot of dependencies to be installed (kernel modules in particular), which might not be available (yet) on your desired target architecture, so aside from adding your arch to https://github.com/openwrt/packages/blob/master/utils/qemu/Makefile#L32 (you should be able to mirror the things necessary for sunxi), you might have to fix a number of reverse dependencies as well. Getting this sorted is easier with the full OpenWrt buildroot, selecting everything you need statically for the sysupgrade image, compared to working piecemeal with the SDK.

Success or failure depends on a lot of things in this case, among others if the bootloader leaves the kernel in the correct mode, so while it should be possible, there might by quite a few hurdles in the way.

1 Like

I'll give a it a shot later today by editing that one line.

Not going to sink too much time into it because honestly a lot of what you said didn't make too much sense.

For instance I can't tell you the difference between buildroot and the sdk. I thought the sdk was the gothib repo that you use to build custom images?

If I were to try this"Getting this sorted is easier with the full OpenWrt buildroot, selecting everything you need statically for the sysupgrade image" am I understanding you correctly that it would attempt to add the qemu package to sysupgrade image found here, https://downloads.openwrt.org/releases/21.02.0/targets/bcm27xx/bcm2711/openwrt-21.02.0-bcm27xx-bcm2711-rpi-4-ext4-sysupgrade.img.gz , or build my own sysupgrade image?

Sorry if this seems like a noobish way to approach this. I have built my own image only once before and compiled some of my own packages from source on a laptop; this definitely is as in depth as I've gone.

Alrighty, I did try to add my architecture to Line 32 in the Makefile of my working directory. Attempted both aarch64 & ARMv8-A

When using make menuconfig qemu does not appear as long as I have the BCM2711 selected as the target, but it is present if I have x86 as the target for the image.

I think getting this to work is a little over my head. Probably too niche to commit much time to this but it would be neat in the future maybe.

That needs to be bcmwhatever, sorry not familiar with the RPi intricacies.

knock yourself out (master not 21.02)
https://github.com/wulfy23/rpi4/tree/master/utilities/qemu-ipk

Interesting! I'll give that image a shot, not exactly sure when I'll get that time next but I'm excited to give it a try. Thank you very much for sharing!

1 Like

qemu aarch64_generic build with openwrt-21.02.2 source.

https://bin.cooluc.com/aarch64_generic/packages/

enable kvm acceleration for openwrt arm architecture

cat >> target/linux/xxxx(model)/config-default <<EOF
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_ARM_HOST=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_INDIRECT_VECTORS=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_VFIO=y
CONFIG_VHOST_NET=y
EOF
1 Like