Image size creep

For Developers
1

I have been building OpenWRT image from the svn, (now from git) for many years for my trusty TL-WR1043ND v1 (and later v2). It has been basically the same configuration all the time, just updated to the latest OS and packages.

The last year I started to notice that the image size has been increasing to the point that in the last build (around October 2018) I had to remove some (big packages) in order to fit everything in (8 MB flash). I am using only "functional packages" (no web UI, or such) and the size creep seemed to be proportional to the size of the code, not that just one package just got suddenly significantly bigger.

Guess what, when running build now, the already reduced image does not fit again and mktplinkfw says I am again ~ 100kB over the limit.

I wonder, what is changing that it makes the code consistently bigger? Is it a compiler which produces more bloated code or some build settings, or are the packages getting "better" and bigger?

Anyone has an idea?

2

You'll have to review your build configuration (both as a whole and relative to the defaults/ diffconfig), a default ath79 release image with luci included weighs around 4.5 MB and I can cram quite some additional features and services into an 8 MB image (RAM is a much bigger problem for the tl-wr1043nd v1, not so much for the v2; obviously when buying new devices one should favour ones with more flash/ RAM).

3

Also, as software development progresses so does binary size in most cases. Worth having in mind is that as MIPS is slowly "dying" focus is moved towards ARM platforms which doesn't have the same limitations as MIPS which in turn may shift focus slightly from size to performance instead.

Edit: ...and before someone mentions it, TL-WDR1043ND (1.X-3.X) are somewhere between a 4-9y old devices and they weren't very beefy during that time so perhaps its time to move on. Version 2.X and higher have at least 64Mbyte of RAM so they're still usable to some extent.

4

What release?

5

@slh @anon50098793

I have been always building the image from the current branch (git master now) so it was not tied to any particular release. I have been using my own configuration (script) which handled the individual modules and packages, because I put in packages like vim (which alone takes ~ 2MB), or ntpd with ntpd-utils, openvpn with openssl, which are quite big and needed to eliminate anything I did not need.

I understand that there are different configuration available (the official image one among them) which use much smaller size. But it was not my point.

I was saying that I have been building basically the very same config (just adapted to the changes made in the default config) of the same list of packages for years and it was becoming lately consistently too large.

As I wrote, I have been using always the newest versions of anything, which included also the tools and the toolchain, and I unfortunately I cannot replicate the old builds (easily), so what I can only conclude from the older logs that the image is becoming bigger, and I am wondering why.

FWIW here is the list of packages I am installing, with the respective sizes:
https://pastebin.com/raw/53cKSsap

I know that if I get rid of vim for example I will be (again) good, but this would not help me to understand why is this happening. I am not primarily looking for the solution, but for the explanation (if there is any particular apart from "it just gets bigger, deal with it").

1 Like
6

I think to best understand you have to take a broad look at software evolution.

I remember installing windows 3.1... tiny....

More hardware + more features = larger codebase.... it's that simple.

Yes, priorities and focus can limit that... but it's the pace of hardware that stamps the core outcomes... nothing else.

7

vim and its dependencies alone take up roughly 2.5 MB, you really don't need to wonder about size issues on an 8 MB flash device with that included.

I do own a tl-wr1043nd v1 myself, with (a fairly large subset of-) strongswan/ openssl, wpad-openssl, usb-storage/ ext4, luci, adblock, bcp38, p910nd, nlbwmon and a fairly large assortment of packages intended for a 'router' - and I still have ~1.1 MB of free flash (RAM is a serious issue though).

# df -h
/dev/root                 5.0M      5.0M         0 100% /rom
/dev/mtdblock4            1.3M    260.0K      1.1M  19% /overlay

https://openwrt.org/supported_devices/432_warning

1 Like
8

Don't forget that the kernel keeps getting bigger. It's not just openwrt or it's packages.

1 Like
9

Also, mbed TLS is much smaller than OpenSSL and at least when I looked at it using OpenVPN med TLS performed better than OpenSSL on MIPS.

1 Like
10

This is what I did, removed vim and added as much features as possible to the busybox version. So I should be good for some time :slight_smile:.

I believe I actually tried openvpn-mbedtls twice and it never worked for me (and I did not figure out why), so at the end I always came back to the openssl. Right now I am migrating my openvpn setup to wireguard and it looks promising so far, so I might get rid of both soon.

Just for the info, I was running the iperf3 test (using default TCP parameters) between two identical TL-WR1043ND v2 routers (with the same image) connected over the internet and set up with either openvpn or wireguard vpn tunnel.

Both VPNs reached basically same speed (~9.5 Mbps), but when openvpn was running top reported ~35% idle CPU time, while with wireguard the idle time was ~75% (and 0% usr). Quite impressive.

11

Yes, wireguard is quite a bit more efficient but it does have some downsides depending on your setup. The only "issue" I know of is that mbed TLS doesn't or at least didn't accept 1024-bit keys while OpenSSL did otherwise they're interchangable at least in my experience. You can do ~15-18mbit at best using OpenVPN on such devices depending on encryption and bit length so either way both OpenVPN and Wireguard should do fine in that regard.

1 Like