What is the best way to ensure a QEMU Guest OS(es) cannot consume Host resources to the point that it affects routing/firewall duties?
As routing/firewall happens in the kernel is it automatically prioritized over the user-space QEMU processes? But what about other 'essential' services like VPN and DNS?
What are you working with hardware wise? How is your network set up?
In addition:
Just to be clear: your Host or Guest is running OpenWrt, correct?
Host is running OpenWRT. Three QEMU guests run on it (web server, print server and Home Assistant). Hardware is a 4 Core Xeon, 64GB ram.
I ask because occasionally HomeAssistant has to compile some C++ binaries for ESP home and when it does this it looks like QEMU is using 100% CPU on all four physical cores.
Obviously could limit QEMU to 2 cores but this is wasteful because it can't use the extra processing power when it needs to. All I want is to ensure that OpenWRT host processes always get priority over the QEMU guest processes. Could this just be done with nice?
Is luci-app-statistics installed? Because that looks like an incredible percentage.
Now where did my dunce cap fall off to? 
Likely because the OP uses the machine to compile code, and route, and print, and be a webserver (among other things):
Yes it's normal when compiling, it's a very processor intensive task.
Yes, that's common on a computer when compiling. Aren't you able to configure the VM's CPU processing?
TBH, I've never used QEMU with OpenWrt or other non GUI Linux distros; but in all Hypervisors, I've been able to configure the CPU usage percentage, etc. This is a VirtualBox example (which uses QEMU):
Have you tried?
You need to add something to your esphome YAML file for the build:
compile_process_limit (Optional, int): The maximum number of simultaneous compile processes to run. Defaults to the number of cores of the CPU which is also the maximum you can set.
Not yet, was going to see if anyone had any other bright ideas. You can limit the CPU% using QEMU but then you are limiting it at all times (essentially wasting spare CPU cycles if the host isn't doing anything).
I think 'nice' is the right answer, but wanted to check.
Trying it out now, I notice OpenWRT runs collectd with a nice of 5 probably for similar reasons.
Well it's 'working' but knowing whether it truly 'works' is harder!
Here is the QEMU service script for reference.
#!/bin/sh /etc/rc.common
START=99
STOP=1
qemu_pidfile="/var/run/qemu-home-assistant.pid"
touch $qemu_pidfile
start() {
nice -10 qemu-system-x86_64 -enable-kvm -cpu host -smp 4 -m 16G \
-bios /vms/bios.bin \
-drive file=/vms/haos.qcow2,if=virtio \
-usb -device usb-host,hostbus=3,hostport=3.4 \
-device virtio-net-pci,mac=E2:F2:6A:01:FF:CD,netdev=br0 \
-netdev bridge,br=br-iot,id=br0 \
-qmp tcp:127.0.0.1:4447,server,nowait \
-pidfile $qemu_pidfile \
-daemonize &> /var/log/qemu-home-assistant.log \
-serial telnet:localhost:4321,server,nowait
echo "QEMU: Started VM with PID $(cat $qemu_pidfile)."
}
stop() {
echo "QEMU: Sending 'system_powerdown' to VM with PID $(cat $qemu_pidfile)."
nc 127.0.0.1 4447 <<QMP
{ "execute": "qmp_capabilities" }
{ "execute": "system_powerdown" }
QMP
if [ -e $qemu_pidfile ]; then
pid=$(cat $qemu_pidfile)
if [ -e /proc/$pid ]; then
echo "QEMU: Waiting for VM shutdown."
while [ -e /proc/$pid ]; do sleep 1s; done
echo "QEMU: VM Process $pid finished."
else
echo "QEMU: Error: No VM with PID $pid running."
fi
rm -f $qemu_pidfile
else
echo "QEMU: Error: $qemu_pidfile doesn't exist."
fi
}
It shouldn't be too hard - just compile something and check while running.
True, but the proof would be that the router/firewall functions truly do get priority, so I'd need to stress the router at the same time. I'll try streaming some 4K video over a VPN connection and see if it handles that while compiling.