OpenWrt Forum Archive

Topic: ARMv8 Multiplatform and Qemu

The content of this topic has been archived on 18 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hello,

I've been struggling with running OpenWrt ARM images on qemu for awhile now and I think I should really ask some questions at this point to clear up my misunderstandings.  I'm on Ubuntu, by the way.

I clone the buildroot repository for OpenWrt, enter ~/openwrt/, select the ARMv8 Multiplatform target, makedefconfig, update and install feeds, run menuconfig and add a few packages/utils (tcpdump, some disk/file system operations in busybox, and my own project I'm trying to port), and start the build.  It builds fine.

The output in ~/openwrt/bin/arm64 includes the following files:

  • openwrt-arm64-qemu-virt.Image

  • openwrt-arm64-qemu-virt-initramfs.Image

  • openwrt-arm64-vexpress-foundation.axf

The page on virtualizing openwrt with Qemu gives this instruction:

qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic \
-smp 1 -m 2048 \
-kernel bin/arm64/openwrt-arm64-qemu-virt-initramfs.Image \
--append "console=ttyAMA0"

I really only tried using the .Image files, but that extension is unfamiliar to me.

I'm trying to get it to a place where I can have a virtual system that has a mountable, writable file system that I can manipulate outside of Qemu (this is for research, but it's probably boring so I won't go into it).

So, I've noticed that it doesn't seem to matter whether I pass openwrt-arm64-qemu-virt.Image or openwrt-arm64-qemu-virt-initramfs.Image as the kernel argument.  That means that I can't have any settings or configuration files other than the default ones from when the image is built because there's no persistent file system.

I've fiddled around with block mount, fstab configurations, and passing -device and -drive options with a disk image file to use as the file system, but since any changes I make to the configuration files or init scripts revert back to the default on shutting down or rebooting.  As such, I can't change the scripts at boot to call switch_root/pivot_root to follow the extroot instructions. I'm at a loss, here.

So, what I'm asking is...

  • What's the difference between the openwrt-arm64-qemu-virt.Image and openwrt-arm64-qemu-virt-initramfs.Image files?

  • How can I get a persistent root file system for this target?

  • Is what I'm trying to do possible, and does what I'm describing make sense?

  • Is there any further reading than what's on the "OpenWrt in Qemu" page that gives more explanation and detail?

  • Is there a better way to run an ARM-based OpenWrt image on a VM? (realview has some serious issues, namely that I can only have 1 network port)

Thanks for your time, and anyone who can help me out here would be my savior.

I don't use qemu exactly like this, but I think that you're going to need to define a storage device for the virtual hardware specification.

tim+openwrt.org wrote:

I don't use qemu exactly like this, but I think that you're going to need to define a storage device for the virtual hardware specification.

First, thank you for the reply.

I must have forgotten to mention this in my original post, but I tried setting my build configuration to produce a root file system image in the form of a .ext4 (it comes out as "openwrt-arm64-root.ext4"). 

I run qemu with this command:

qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine type=virt -nographic \
-smp 1 -m 2048 \
-kernel openwrt-arm64-qemu-virt.Image \
-drive file=openwrt-arm64-root.ext4,if=none,id=sd0,format=raw \
-device virtio-blk-device,drive=sd0 \
--append "root=/dev/vda

When I add a virtual drive with an ext4 disk image in it (and a -device argument to go along with it because it gets ignored as "orphaned" otherwise), it shows up on /dev/vda.

I get that it's because it's going in as a virtual device, but with the way I'm more used to (qemu-system-arm -M realview-eb-mpcore ...<options options options>... -sd openwrt-root-ext4 --append "root=/dev/mmcblk0"), the image passed to the -sd arg shows up on /dev/mmcblk0 and appending "root=/dev/mmcblk0" gets it to properly boot with that image as the root file system.

In the aarch64 case above, if I do that, it just creates /dev/root as a simlink pointing to /dev/vda (like /dev/root -> /dev/vda), but it's not the root file system.  Like, this is what I get for a few commands to check what's mounted and such:

root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
tmpfs                   970.8M     28.0K    970.7M   0% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev

^ Those are there regardless

root@OpenWrt:/# mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,nosuid,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,mode=600)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)

^ That "rootfs on / type rootfs (rw)" line is there regardless, as well.

root@OpenWrt:/# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda  254:0    0  48M  0 disk

Note that it's not mounted.

If I try

mount /dev/vda /

, it doesn't seem to change anything, either:

root@OpenWrt:/# mount /dev/vda /
[  610.544979] EXT4-fs (vda): couldn't mount as ext3 due to feature incompatibilities
[  610.552327] EXT4-fs (vda): couldn't mount as ext2 due to feature incompatibilities
[  610.600801] EXT4-fs (vda): mounted filesystem without journal. Opts: (null)

^ In my other machines, that dialogue normally appears during boot.  Now if I do lsblk...

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda  254:0    0  48M  0 disk /

With mount...

root@OpenWrt:/# mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,nosuid,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,mode=600)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)
/dev/vda on / type ext4 (rw,relatime,block_validity,delalloc,barrier,user_xattr)

Any ideas?  By the way, feel free to assume that I don't know things that are crucially important but are obvious to others.  I feel like there's just some gap missing in my understanding of all of this (not just as it pertains to OpenWrt, either).

Hi all,
   I'm also trying the same scenario, and getting the same issue

Anyone of you help me to resolve this issue.

Thanks in advance

-Regards,
-Vishnu

The discussion might have continued from here.