Package Makefile: Compress Binary with UPX


I'm currently working on telegraf package for OpenWRT. Because the default binary size is about 147M I started to exclude plugins and reached 47M. With the help of the telegraf developers I excluded the debug symbols and was able to reduce the binary size to 36M.

Since most routers don't have that much flash capacity I want to reduce the size even further. I found the tool upx which compresses the binary while leave it executable. You can find a nice test overview here.
I also found in other threads here that people are using it too.
(In a test I reduced my 36M to awesomely 9M 6M, which are still much)

Now I want to include this compressing into my Makefile for OpenWRT packages and are unsure how to do this. And than I found this discussion that it is somehow conflicting with the compressing of the squashfs..

Now I am totally confused.

  • Is upx an option for openwrt ?
  • If yes, how do I declare the host dependency in the Makefile (if even possible) ?
  • Do you have experiences with upx ?

Thanks! :slight_smile:

1 Like

is it a static build ?

did you try to strip the binary ?

An ancient version of upx seems to be among the openwrt build tools, but I have never seen discussion about it, or anything that uses it. (Well, search reveal 2-3 PRs, one of whicvh you alread linked.)

The filesystems (squashfs, ubifs, jffs2, etc.) that we commonly use, already offer filesystem level compression, so the additional benefit might not be that great.

You should maybe test and first create a firmware image without the new package, and then include the package in the build and then compare how much the firmware size actually grows.


Sounds like he did, at least to some extent:

being able to compress image 6:1 from 36 MB to 6 MB practically means that there is lots of something repeatable easily compressable, like background images, icons etc.

File system compression should handle that well enough.

I meant using the command strip, not by removing unneeded functionality ...

Thanks for your replies!

I am not used to the command strip, but as I understood it, it does the same like compiling with argument -s -w, which excludes debug symbols.

Did you mean something different ? Which symbols should I remove with strip ?

To my knowledge, the size-constraint-oriented OpenWrt build system uses strip by default (with the sstrip as the default strip tool).

(and in general, stripping usually removes the debug symbols, which you have apparently configured to left out already at the build phase)

1 Like

I did a small test on my RPI 3 B with OpenWRT last snapshot.

On the host

$ upx --lzma -9 ../../build_dir/target-aarch64_cortex-a53_musl/telegraf-1.19.1/telegraf_small 
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  34734080 ->   6341984   18.26%   linux/arm64   telegraf_small                

Packed 1 file.

$ du -s ../../build_dir/target-aarch64_cortex-a53_musl/telegraf-1.19.1/telegraf_small 
6208	../../build_dir/target-aarch64_cortex-a53_musl/telegraf-1.19.1/telegraf_small

On the RPI before scp-ing the binary

root@openwrt~# df 
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             15285368    307900  14961084   2% /
tmpfs                   470284        60    470224   0% /tmp
/dev/mmcblk0p1           65390     18046     47344  28% /boot
tmpfs                      512         0       512   0% /dev

And after:

root@openwrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             15285368    314108  14954876   2% /
tmpfs                   470284        60    470224   0% /tmp
/dev/mmcblk0p1           65390     18046     47344  28% /boot
tmpfs                      512         0       512   0% /dev

Which is a total difference of ~6M (314108−307900=6208).

Did I do something wrong in my test ?

For me it seems to work. Any idea how to integrate that into my Makefile ?

You didn't test if it was necessary, or would the file system compression do similar job?

What happens if you store normal un-upxed file?

Of course!

$ du -s ../../build_dir/target-aarch64_cortex-a53_musl/telegraf-1.19.1/telegraf_small 
34052	../../build_dir/target-aarch64_cortex-a53_musl/telegraf-1.19.1/telegraf_small


$ ssh root@openwrt df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             15285368    203772  15065212   1% /
tmpfs                   470284        56    470228   0% /tmp
/dev/mmcblk0p1           65390     18046     47344  28% /boot
tmpfs                      512         0       512   0% /dev


$ ssh root@openwrt df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             15285368    237820  15031164   2% /
tmpfs                   470284        56    470228   0% /tmp
/dev/mmcblk0p1           65390     18046     47344  28% /boot
tmpfs                      512         0       512   0% /dev

Resulting in 237820−203772=34048 which are just 4 byte less than the original 34052.

So I guess the result is that it seems to be helpful to use upx compression.

Now my question:

  • How do I set a host dependency for compiling my package ? (because upx is not needed as a DEPENDS= on the target, but as a tool like automake or gcc or something on host side need to be installed)

Could you maybe help me again @Grommish ?

If there is already a package for upx, you'd install it to the build system by using HOST_BUILD_DEPENDS:=upx/host, assuming it's been kept up to call the Host build stuff. You'd have to look at the upx package.

If there isn't already one, or one that needs to be updated, then Congratulations! You've officially fallen down the rabbithole :smiley: You get to make/update it.

That should make sure it's installed in the staging_dir so it can be accessed by the build environment at compile time


thank you very much!

I will try that :slight_smile:

After starting to build my own upx package (which doesn't went that far), I found a git repo that does excatly what I wanted.

But am I right, that I need to make a pull request for that first, before I can push my package with PKG_DEPENDS:= ... upx/host ? :frowning:

That's a question for the repo-gods on the best way to proceed, given that one of your pre-reqs would be a PR on a package that you aren't the author of but would be willing to maintain it? Someone will be able to tell you what to do once you get it all working, I'm sure. I can't see your package going anywhere without the pre-req packages being in place though. Who would be able to build it without both.

I'm in a similar situation where my suricata6 package can't go ahead until rust-lang is finished But, you get them working, you submit the PR and point out that one is contingent on the other, and wait.. You'll still be able to build the package for your own builds, and anyone that really wants it can roll-their-own from the PRs.

Thanks for your thoughts!

I wrote with the author and said that the package was removed from openwrt because of squashfs.
Now I realized that, I am using ext4 on my RPIs.

So big edit to: Package Makefile: Compress Binary with UPX - #9 by jonnytischbein and Package Makefile: Compress Binary with UPX - #7 by jonnytischbein

I guess there is no flag to check whether my package is compiled for squashfs or ext4 image, right ? Or I didn't found on (but I also completely overlooked the HOST_BUILD* variables :upside_down_face: )

squashfs(-lzma) is highly compressed, but it only covers the packages that are part of OpenWrt's default (release-) firmware/ package set. User-installed packages end up in the overlay, which -generally- is jffs2, also compressed, but by far not as good as squashfs could (as it's a re-writeable filesystem).

So, no - unless you build OpenWrt yourself (including your package into the read-only squashfs part), squashfs won't help you to reduce the footprint of your package (how much jffs2's compression can reduce it remains to be tested).


Thanks for your explaination!
I do understand it a bit better now :slight_smile:

Would it make sense to you, to differntiate between packages for ext4 and jiffs2 systems ? Because it would make an impact on ext4 systems if the binary is compressed.
On the other side I guess that ex4 systems do have enough memory capacity to handle that, right ?

You can't, at least not programmatically - sure, you could offer two alternative (binary-) packages, one upx-compressed, on not compressed, but personally I wouldn't do that (or even consider that to be a bug).

The main question (apart from how well it compresses on jffs2, the other potential options as backing filesystems for the overlay usually tend to be on devices with larger storage sizes anyways) would be how likely your package is going to be installed on devices with << ~256 MB flash.

Using ext4 is rare in OpenWrt context. It mainly concerns routers without flash, with SD cards etc., in practice minicomputers like RPI and x86 systems...

Most normal routers have internal flash with limited storage space, so it makes sense to use
a compressing file system.


Looking at your PR, I see that telegraaf uses go - is that just a build-time dependency or does the golang package also need to be present at runtime? In the later case, you can pretty much save yourself the trouble to shrink telegraaf (even further) itself, as golang already weighs ~80 MB by itself (and therefore only fits on devices with very large storage to begin with).

1 Like