I'm building a custom OpenWRT image for a MikroTik HexLite router containing WireGuard and custom settings, including a different IP address. The image isn't much larger than the official build, but I'm still unable to flash it because RAM is just not large enough.
When netbooting the initramfs image, I end up using 19.2MB of the 26.8MB sized tmpfs for the running system. Uploading the squashfs image of about 7MB is enough to crash the system before it starts flashing.
Now I'm thinking about what's the best solution for this. I found four options:
Build two different images, one for an initramfs with less packages, but still with the custom IP. But building two configs is cumbersome.
Boot OpenWRT 23.05 from network and flash 25.12. The 23.05 is small enough for this to work. But that's a weird solution and I'd like to boot the same OS I flash.
Remove files from the running initramfs, like /lib/modules to create space. That turns out to be pretty efficient and I like it somewhat. It would be interesting how much one could delete and still be able to flash.
Remove packages from the image. For example, MikroTik default packages add kmod-ath9k, but the HexLite has no WiFi. On the one hand, this seems to be the proper way. On the other hand, my image would fit into flash, so I would like to be able to flash it.
I wasn't able to find any guidance for this issue and I think there should be some. What are your thoughts about this? How would you solve it?
Do the first installation using default prebuilt images. Then you can add (or remove ath9k for example) packages using self-made sysupgrades or owut if they are too big for less compressed overlayfs
Please report a bug, or make a PR removing (-kmod-ath9k) the package.
The issue with using the prebuilt images is that they come with a default 192.168.1.1 IP address. For legacy reasons, this still conflicts with my company network. I did that in the past and it was a hassle. The Raspberry Pi for automatic setup regularily got lost in routing somehow.
Removing kmod-ath9k from the device config is a great idea, I'll get on it.
It's an automated install also generating local WireGuard config and pushing the public key and IP to a server for me to add to the VPN. I do a lot of OpenWRT installs. I could probably setup a virtual NAT on the Pi to work around that. Or even use a real NAT with another router.
Anyhow, I was still wondering if there is some general solution to the issue of having not enough RAM for an image that would fit into flash. Also, the generic images currently still fit, but if they keep growing in size, it will become an issue with them as well.
What are you talking about? The 750-r2-Image here is 6.9MB for the initramfs (packed) and 7.3MB for the squashfs image.
Edit: the prebuilt images are not default config, they have LuCI. And I prefer having the webinterface, too. But as far as I understand, reducing packages is the reasonable way.
Edit 2: just booted it, the prebuilt initramfs uses 18.5MB when unpacked, leaving 8.3MB. That 1MB after uploading the image is still enough, but from my experience, 200-300kB is leading to failure.
Building an image without (the unneeded) kmod-ath9k and it's dependencies got me down to 6.1MB initramfs and 6.4MB squashfs. initramfs unpacked uses 15.9MB, leaving a whole 10.9MB of tmpfs free.
My build was even with wireguard and dependencies, but with ppp and some other stuff removed.
5.9 with wireguard and ppp, but without wpad. I'll open a PR.
Do the small install first, then sysupgrade to custom image with rest of packages
few hundred kb trimmed (wpad+ath9k) will make that even more likely to succeed.
As I build my own image anyway, I'll just add the patch of my PR until it's merged. Been doing that for 23.05 as well, as RB750r2 support unfortunately didn't land there in time and I was eager to update the fleet away from 19.07.
Due to the ar71xx to ath79 change, I couldn't update while keeping the config. Since then I build the image with embedded config so I could update in the field (with some custom script to carry over uplink IP and the wireguard config). Every update was scary two minutes of waiting for the ping to come back, but every updated worked. Except for the one device where IPv4 config was set on the wan6 interface...
