I don't know much about OpenWRT and its build system and the possible options, someone built this image at my job, he left, no documentation or anything, now that I must upgrade OpenWRT its the last point I'd like to address before releasing it if possible.
Was wondering about the size of the OpenWRT image I build. We have an old 18.05 image here that is compressed to 22MB, gunzip it, it is 35MB.
Now I updated OpenWRT to 23.05, its 40MB compressed (I added some packages), but gunzip it and its 16GB, how comes?
Anything I can do to reduce its size?
there was a thread recently where someone compared recent versions, and some lib (?) made the image grow a lot.
I'm out the door, and can't find it atm, but it could be related.
but if you have 14GB disk, I don't really see the problem.
Its not really a problem, its more a question I'm asking because the OpenWRT 18.05 image size is 22MB and when I gunzip it its 40MB big, but when doing the same thing on OpenWRT 23.05, the image size is 40MB but when I gunzip it its 16GB.
I believe it has something to do with the CONFIG_TARGET_ROOTFS_PARTSIZE=15000 configuration.
I believe it also affects the image creation, OpenWRT build phase is using dd with a block size of 16GB:
0+1 records in
1+0 records out
15728640000 bytes (16 GB, 15 GiB) copied, 35.0909 s, 448 MB/s
So I need 16GB of free RAM to build this image, I find it weird?
We are using python, shadow, coreutils and openvpn for our iot stuff. collectd is not included in our "release" image. But thanks for the heads up. I'm not too concerned about how much space it takes on the router but more about the image size once uncompressed.
What do you mean bad gunzip params? I am using the image openwrt-x86-64-generic-squashfs-rootfs.img.gz, gunzip only uncompress files, it will not change the original file size.
the increase in size seems about right, it you follow the size increases of openwrt (and the linux kernel) ... definitley not alarming.
this is related to the size if the entire image including reserved free space ...meaning you have 15gb+ of free space reserved for memory and less than 40mb are openwrt. You can adjust this if you have something else to do with the abundance of free space. However, nothing you have provided comes as unusual or alarming
Ok, any reason on why this changed compared to OpenWRT 18.05?
My sysupgrade now takes over 400 seconds where it only takes 80 seconds on the old image.
18.05 is ancient in terms of openwrt . Everything has gotten bigger since then. 18.05 supported 4mb devices ... beyond very basic config, 4mb dvices can no longer be supported. If i recall, 8mb devices can no longer be supported ..and 16mb is barely enough for a decent openwrt setup with luci and all. Not sure about the extra flash time. That would take some looking into, but I am pretty certain it has to do with build configuration, nothing with openwrt
I have a 16GB SSD on my router, so making very tiny images is not really my goal, my issue is that the image built with OpenWRT 18.05 is 22MB compressed and only 35MB uncompressed. With still the partition size set to 15000.
This is my old image:
-rw-rw-r-- 1 buildbot buildbot 35M Oct 14 15:37 openwrt-2021-03-10-11-00-12.img
-rw-rw-r-- 1 buildbot buildbot 22M Oct 14 15:37 openwrt-2021-03-10-11-00-12.img.gz
losetup loop999 openwrt-2021-03-10-11-00-12.img
fdisk /dev/loop999
Disk /dev/loop999: 34.98 MiB, 36683264 bytes, 71647 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x2d20a05d
Device Boot Start End Sectors Size Id Type
/dev/loop999p1 * 512 33279 32768 16M 83 Linux
/dev/loop999p2 33792 30753791 30720000 14.6G 83 Linux
The new image:
-rw-r--r-- 1 buildbot buildbot 15G oct. 15 13:13 openwrt.img
-rw-r--r-- 1 buildbot buildbot 46M oct. 15 13:13 openwrt.img.gz
losetup loop999 openwrt.img
fdisk /dev/loop999
Disk /dev/loop999: 14.66 GiB, 15745941504 bytes, 30753792 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x05bd2c8b
Device Boot Start End Sectors Size Id Type
/dev/loop999p1 * 512 33279 32768 16M 83 Linux
/dev/loop999p2 33792 30753791 30720000 14.6G 83 Linux
If its an issue with my build system or build options, I'm open to any suggestion. It's more of an optimization issue more than an usability. If the image cannot be shrunk like the old image, then too bad for me, we'll have to wait longer when flashing the upgrade.
I believe sysupgrade is significantly longer because it needs to uncompress the image, and the new one is 15GB instead of 30MB.
I made a script that analyses the empty blocks, trim the image to the size of empty blocks + some extra zeroed blocks, the new image is 59MB big and flash only took a few seconds. Guess I don't have anything better for now.
That way, I don't have my not enought memory error I sometimes happen to have, because that thing just write 1 big block of data, unfortunatly, it writes 1 block with a size of the whole image: dd if=/workdir/openwrt/build_dir/target-x86_64_musl/linux-x86_64/tmp/openwrt-x86-64-generic-squashfs-rootfs.img.gz of=/workdir/openwrt/build_dir/target-x86_64_musl/linux-x86_64/tmp/openwrt-x86-64-generic-squashfs-rootfs.img.gz.new bs=15728640000 conv=sync
Maybe I should open an issue for that, I believe it is not expected to required your whole ram to just pad an image.
Do you need this root partition size anymore? You specify the size, but don’t like the result? It’s confusing. Image building changed a lot in 6 years, of course.