RS-232 to USB serial adapters

This can happen if your serial speed is correct for BIOS but the OS installed in the USB drive is running at a different speed.

What is the serial speed (baud)? What is the device? What is the OS in the USB drive?

I also get some garbled output for the first few seconds of boot with OpenWrt. Then it clears up and is usable again.

Also, on some devices the serial was garbled but it was just because the serial port pins are old and dirty, just disconnect and ri-connect the adapter 5 or 10 times will clean them and make good electrical contact.

It should not show garbled text, the configuration should be the same also in IPFire. Does OpenWrt work? Maybe it is a problem of IPFire.

Did you use the iso or the "flash" image? https://www.ipfire.org/download/ipfire-2.27-core160 Try with both