Today it struck me that the size of dynamic libraries in OpenWrt are almost always around 65kB, or a multiple of that:
~$ ls -lR /usr | grep ".so$"
lrwxrwxrwx 1 root root 17 Jan 29 23:36 ldd -> ../../lib/libc.so
-rw-r--r-- 1 root root 65836 Jan 29 23:36 libuclient.so
-rw-r--r-- 1 root root 65740 Jan 29 23:36 libudebug.so
-rwxr-xr-x 1 root root 65824 Jan 29 23:36 uhttpd_ubus.so
-rwxr-xr-x 1 root root 135188 Jan 29 23:36 app_confbridge.so
-rwxr-xr-x 1 root root 66576 Jan 29 23:36 app_dial.so
-rwxr-xr-x 1 root root 65744 Jan 29 23:36 app_echo.so
-rwxr-xr-x 1 root root 65904 Jan 29 23:36 app_mp3.so
-rwxr-xr-x 1 root root 66008 Jan 29 23:36 app_playback.so
-rwxr-xr-x 1 root root 66364 Jan 29 23:36 app_stack.so
-rwxr-xr-x 1 root root 65792 Jan 29 23:36 app_system.so
-rwxr-xr-x 1 root root 65840 Jan 29 23:36 bridge_builtin_features.so
-rwxr-xr-x 1 root root 65916 Jan 29 23:36 bridge_simple.so
-rwxr-xr-x 1 root root 66276 Jan 29 23:36 bridge_softmix.so
-rwxr-xr-x 1 root root 134144 Jan 29 23:36 chan_pjsip.so
-rwxr-xr-x 1 root root 66636 Jan 29 23:36 codec_a_mu.so
-rwxr-xr-x 1 root root 66708 Jan 29 23:36 codec_alaw.so
-rwxr-xr-x 1 root root 66388 Jan 29 23:36 codec_resample.so
-rwxr-xr-x 1 root root 66708 Jan 29 23:36 codec_ulaw.so
-rwxr-xr-x 1 root root 66376 Jan 29 23:36 format_wav.so
-rwxr-xr-x 1 root root 66136 Jan 29 23:36 func_callerid.so
-rwxr-xr-x 1 root root 66492 Jan 29 23:36 func_logic.so
-rwxr-xr-x 1 root root 65872 Jan 29 23:36 func_pjsip_aor.so
-rwxr-xr-x 1 root root 65872 Jan 29 23:36 func_pjsip_contact.so
-rwxr-xr-x 1 root root 65848 Jan 29 23:36 func_pjsip_endpoint.so
-rwxr-xr-x 1 root root 68336 Jan 29 23:36 func_strings.so
-rwxr-xr-x 1 root root 65888 Jan 29 23:36 func_timeout.so
-rwxr-xr-x 1 root root 67252 Jan 29 23:36 pbx_config.so
-rwxr-xr-x 1 root root 66316 Jan 29 23:36 res_crypto.so
-rwxr-xr-x 1 root root 66264 Jan 29 23:36 res_http_websocket.so
-rwxr-xr-x 1 root root 66544 Jan 29 23:36 res_phoneprov.so
-rwxr-xr-x 1 root root 66456 Jan 29 23:36 res_pjproject.so
-rwxr-xr-x 1 root root 334944 Jan 29 23:36 res_pjsip.so
-rwxr-xr-x 1 root root 65920 Jan 29 23:36 res_pjsip_acl.so
-rwxr-xr-x 1 root root 66004 Jan 29 23:36 res_pjsip_authenticator_digest.so
-rwxr-xr-x 1 root root 65916 Jan 29 23:36 res_pjsip_caller_id.so
-rwxr-xr-x 1 root root 66228 Jan 29 23:36 res_pjsip_config_wizard.so
-rwxr-xr-x 1 root root 65904 Jan 29 23:36 res_pjsip_dialog_info_body_generator.so
-rwxr-xr-x 1 root root 65996 Jan 29 23:36 res_pjsip_diversion.so
-rwxr-xr-x 1 root root 65800 Jan 29 23:36 res_pjsip_dlg_options.so
-rwxr-xr-x 1 root root 65824 Jan 29 23:36 res_pjsip_dtmf_info.so
-rwxr-xr-x 1 root root 65776 Jan 29 23:36 res_pjsip_empty_info.so
-rwxr-xr-x 1 root root 65788 Jan 29 23:36 res_pjsip_endpoint_identifier_anonymous.so
-rwxr-xr-x 1 root root 66372 Jan 29 23:36 res_pjsip_endpoint_identifier_ip.so
-rwxr-xr-x 1 root root 65804 Jan 29 23:36 res_pjsip_endpoint_identifier_user.so
-rwxr-xr-x 1 root root 66432 Jan 29 23:36 res_pjsip_exten_state.so
-rwxr-xr-x 1 root root 66328 Jan 29 23:36 res_pjsip_header_funcs.so
-rwxr-xr-x 1 root root 66660 Jan 29 23:36 res_pjsip_history.so
-rwxr-xr-x 1 root root 66088 Jan 29 23:36 res_pjsip_logger.so
-rwxr-xr-x 1 root root 66164 Jan 29 23:36 res_pjsip_messaging.so
-rwxr-xr-x 1 root root 66456 Jan 29 23:36 res_pjsip_mwi.so
-rwxr-xr-x 1 root root 65760 Jan 29 23:36 res_pjsip_mwi_body_generator.so
-rwxr-xr-x 1 root root 66008 Jan 29 23:36 res_pjsip_nat.so
-rwxr-xr-x 1 root root 66268 Jan 29 23:36 res_pjsip_notify.so
-rwxr-xr-x 1 root root 65812 Jan 29 23:36 res_pjsip_one_touch_record_info.so
-rwxr-xr-x 1 root root 65836 Jan 29 23:36 res_pjsip_outbound_authenticator_digest.so
-rwxr-xr-x 1 root root 66204 Jan 29 23:36 res_pjsip_outbound_publish.so
-rwxr-xr-x 1 root root 67080 Jan 29 23:36 res_pjsip_outbound_registration.so
-rwxr-xr-x 1 root root 65936 Jan 29 23:36 res_pjsip_path.so
-rwxr-xr-x 1 root root 65832 Jan 29 23:36 res_pjsip_phoneprov_provider.so
-rwxr-xr-x 1 root root 65816 Jan 29 23:36 res_pjsip_pidf_body_generator.so
-rwxr-xr-x 1 root root 65764 Jan 29 23:36 res_pjsip_pidf_digium_body_supplement.so
-rwxr-xr-x 1 root root 65760 Jan 29 23:36 res_pjsip_pidf_eyebeam_body_supplement.so
-rwxr-xr-x 1 root root 66104 Jan 29 23:36 res_pjsip_publish_asterisk.so
-rwxr-xr-x 1 root root 132812 Jan 29 23:36 res_pjsip_pubsub.so
-rwxr-xr-x 1 root root 66516 Jan 29 23:36 res_pjsip_refer.so
-rwxr-xr-x 1 root root 66256 Jan 29 23:36 res_pjsip_registrar.so
-rwxr-xr-x 1 root root 65872 Jan 29 23:36 res_pjsip_rfc3326.so
-rwxr-xr-x 1 root root 66644 Jan 29 23:36 res_pjsip_sdp_rtp.so
-rwxr-xr-x 1 root root 65856 Jan 29 23:36 res_pjsip_send_to_voicemail.so
-rwxr-xr-x 1 root root 132648 Jan 29 23:36 res_pjsip_session.so
-rwxr-xr-x 1 root root 65800 Jan 29 23:36 res_pjsip_sips_contact.so
-rwxr-xr-x 1 root root 66220 Jan 29 23:36 res_pjsip_t38.so
-rwxr-xr-x 1 root root 66096 Jan 29 23:36 res_pjsip_transport_websocket.so
-rwxr-xr-x 1 root root 65852 Jan 29 23:36 res_pjsip_xpidf_body_generator.so
-rwxr-xr-x 1 root root 199208 Jan 29 23:36 res_rtp_asterisk.so
-rwxr-xr-x 1 root root 65936 Jan 29 23:36 res_sorcery_astdb.so
-rwxr-xr-x 1 root root 65984 Jan 29 23:36 res_sorcery_config.so
-rwxr-xr-x 1 root root 65852 Jan 29 23:36 res_sorcery_memory.so
-rwxr-xr-x 1 root root 65920 Jan 29 23:36 res_sorcery_realtime.so
-rwxr-xr-x 1 root root 66012 Jan 29 23:36 res_srtp.so
-rwxr-xr-x 1 root root 65812 Jan 29 23:36 res_timing_timerfd.so
-rw-r--r-- 1 root root 65736 Jan 29 23:36 lucihttp.so
-rwxr-xr-x 1 root root 66480 Jan 29 23:36 nixio.so
-rwxr-xr-x 1 root root 65872 Jan 29 23:36 ubus.so
-rwxr-xr-x 1 root root 65848 Jan 29 23:36 ip.so
-rwxr-xr-x 1 root root 65848 Jan 29 23:36 jsonc.so
-rwxr-xr-x 1 root root 65812 Jan 29 23:36 output_alsa.so
-rwxr-xr-x 1 root root 65652 Jan 29 23:36 output_dummy.so
-rwxr-xr-x 1 root root 65940 Jan 29 23:36 pppoatm.so
-rwxr-xr-x 1 root root 66588 Jan 29 23:36 pppoe.so
-rwxr-xr-x 1 root root 65932 Jan 29 23:36 file.so
-rwxr-xr-x 1 root root 65852 Jan 29 23:36 iwinfo.so
-rwxr-xr-x 1 root root 66056 Jan 29 23:36 luci.so
-rwxr-xr-x 1 root root 65900 Jan 29 23:36 rpcsys.so
-rwxr-xr-x 1 root root 65844 Jan 29 23:36 rrdns.so
-rwxr-xr-x 1 root root 65964 Jan 29 23:36 ucode.so
-rwxr-xr-x 1 root root 65980 Jan 29 23:36 digest.so
-rwxr-xr-x 1 root root 66028 Jan 29 23:36 fs.so
-rwxr-xr-x 1 root root 131244 Jan 29 23:36 html.so
-rwxr-xr-x 1 root root 65688 Jan 29 23:36 log.so
-rw-r--r-- 1 root root 65740 Jan 29 23:36 lucihttp.so
-rwxr-xr-x 1 root root 65724 Jan 29 23:36 math.so
-rwxr-xr-x 1 root root 65944 Jan 29 23:36 nl80211.so
-rwxr-xr-x 1 root root 131488 Jan 29 23:36 rtnl.so
-rwxr-xr-x 1 root root 65980 Jan 29 23:36 ubus.so
-rwxr-xr-x 1 root root 65864 Jan 29 23:36 uci.so
-rwxr-xr-x 1 root root 65948 Jan 29 23:36 uloop.so
-rwxr-xr-x 1 root root 65848 Jan 29 23:36 core.so
Why is that? I could imagine that for some reason a library is always a multiple of exact 64KiB, due to blocksizes or whatever. But that is not the case here. BTW, this is mips_24kc. But I looked on aarch64_cortex-a53, and there I see the same pattern.
I imagine it's due to something like the musl static library being included in each of the library items. I would have thought with some linker trimming there would be a bit more variation, but perhaps there is some reason that the trimming isn't dropping that much out...
It might be worth checking what kind of symbols are in each library to see if there is a common set of unneeded musl functions being carried around.
The plot thickens. I failed to look at the link table, because objdump on my platform couldn't handle mips code.
But I tried binwalk to show the entropy of 3 random libraries:
The "near 64 kB multiples" .so size seems true for 64-bit aarch64, but not for 32-bit ARM. Both examples below are up-to-date 1-2 days old main/master builds with the default compilation settings (and pretty similar package selection).
mediatek/filogic MT6000 (Flint2):
root@router6000:~# cat /etc/openwrt_release
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='SNAPSHOT'
DISTRIB_REVISION='r33165-2256cfac68'
DISTRIB_TARGET='mediatek/filogic'
DISTRIB_ARCH='aarch64_cortex-a53'
DISTRIB_DESCRIPTION='OpenWrt SNAPSHOT r33165-2256cfac68'
DISTRIB_TAINTS='no-all busybox'
root@router6000:~# uname -a
Linux router6000 6.12.71 #0 SMP Fri Feb 20 19:44:26 2026 aarch64 GNU/Linux
root@router6000:~# ls -lR /usr | grep ".so$" | head -n 20
lrwxrwxrwx 1 root root 17 Feb 20 21:44 ldd -> ../../lib/libc.so
-rwxr-xr-x 1 root root 131075 Feb 20 21:44 libelf-0.192.so
lrwxrwxrwx 1 root root 15 Feb 20 21:44 libelf.so.1 -> libelf-0.192.so
-rwxr-xr-x 1 root root 138842 Feb 20 21:44 libiptext.so
-rwxr-xr-x 1 root root 65890 Feb 20 21:44 libiptext4.so
-rwxr-xr-x 1 root root 65890 Feb 20 21:44 libiptext6.so
-rwxr-xr-x 1 root root 65539 Feb 20 21:44 libiptext_arpt.so
-rwxr-xr-x 1 root root 65690 Feb 20 21:44 libiptext_ebt.so
lrwxrwxrwx 1 root root 11 Feb 20 21:44 libpng.so -> libpng16.so
-rw-r--r-- 1 root root 65539 Feb 20 21:44 libuclient.so
-rw-r--r-- 1 root root 65585 Feb 20 21:44 libudebug.so
-rw-r--r-- 1 root root 65674 Feb 20 21:44 libunet.so
-rwxr-xr-x 1 root root 65618 Feb 20 21:44 uhttpd_ubus.so
-rwxr-xr-x 1 root root 65546 Feb 20 21:44 conntrack.so
-rwxr-xr-x 1 root root 65594 Feb 20 21:44 cpu.so
-rwxr-xr-x 1 root root 65539 Feb 20 21:44 cpufreq.so
-rwxr-xr-x 1 root root 65539 Feb 20 21:44 exec.so
-rwxr-xr-x 1 root root 65570 Feb 20 21:44 interface.so
-rwxr-xr-x 1 root root 65554 Feb 20 21:44 iwinfo.so
-rwxr-xr-x 1 root root 65546 Feb 20 21:44 load.so
ipq806x R7800:
root@router78:/usr# cat /etc/openwrt_release
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='SNAPSHOT'
DISTRIB_REVISION='r33181-8e33c40c7f'
DISTRIB_TARGET='ipq806x/generic'
DISTRIB_ARCH='arm_cortex-a15_neon-vfpv4'
DISTRIB_DESCRIPTION='OpenWrt SNAPSHOT r33181-8e33c40c7f'
DISTRIB_TAINTS='no-all busybox'
root@router78:/usr# uname -a
Linux router78 6.12.71 #0 SMP Sat Feb 21 18:24:45 2026 armv7l GNU/Linux
root@router78:/usr# ls -lR /usr | grep ".so$" | head -n 20
lrwxrwxrwx 1 root root 17 Feb 21 20:24 ldd -> ../../lib/libc.so
-rwxr-xr-x 1 root root 77827 Feb 21 20:24 libelf-0.192.so
lrwxrwxrwx 1 root root 15 Feb 21 20:24 libelf.so.1 -> libelf-0.192.so
-rwxr-xr-x 1 root root 69570 Feb 21 20:24 libiptext.so
-rwxr-xr-x 1 root root 8374 Feb 21 20:24 libiptext4.so
-rwxr-xr-x 1 root root 8374 Feb 21 20:24 libiptext6.so
-rwxr-xr-x 1 root root 4098 Feb 21 20:24 libiptext_arpt.so
-rwxr-xr-x 1 root root 8270 Feb 21 20:24 libiptext_ebt.so
-rw-r--r-- 1 root root 20489 Feb 21 20:24 libuclient.so
-rw-r--r-- 1 root root 8217 Feb 21 20:24 libudebug.so
-rw-r--r-- 1 root root 36938 Feb 21 20:24 libunet.so
-rwxr-xr-x 1 root root 16426 Feb 21 20:24 uhttpd_ubus.so
-rwxr-xr-x 1 root root 4102 Feb 21 20:24 conntrack.so
-rwxr-xr-x 1 root root 8222 Feb 21 20:24 cpu.so
-rwxr-xr-x 1 root root 8194 Feb 21 20:24 cpufreq.so
-rwxr-xr-x 1 root root 4102 Feb 21 20:24 dhcpleases.so
-rwxr-xr-x 1 root root 4098 Feb 21 20:24 entropy.so
-rwxr-xr-x 1 root root 49154 Feb 21 20:24 exec.so
-rwxr-xr-x 1 root root 8210 Feb 21 20:24 interface.so
-rwxr-xr-x 1 root root 8202 Feb 21 20:24 iwinfo.so
My guess would be that linker wants to place something at the next data page boundary, meaning either 64 kB (65536) for 64bit and 4 kB (4096) for 32bit.
I cannot see how that can be true. The data in squashfs is highly compressed, so a 64KiB library segment will take much less than an erase-block. And I doubt an compressed page can be mmaped directly to ram, although it might be possible, with some abstraction layer.
It is not about flash erase blocks, which typically are 64 or 128 kB even in the older routers.
It is likely about memory addressing architecture, memory "page size".
4 kB is typical for some processors, but apparently 64 kB is used by the gcc linker for 64bit Arm targets.
The curious part is that in kernel options, the 4 kB page size is set also for 64 bit Arm targets. So, I wonder if there is some kind of config error for GCC linker.
(@Mijzelf mentioned mips_24kc which is 32bit architecture, so it is maybe not just aarch64 targets)
From my research it appears to be related to upstream changes in how executables and library files are padded in relation to support for Full RELRO. Padding changed from using commonpagesize to using maxpagesize.
Using checksec on the library files and executables that are padded to past 64k boundaies does show Full RELRO. Went back to 19.07.9 and checked a file there - it has the small size but checksec still shows Full RELRO. (checked rrdtool.so).
This sounds like something that the gcc & binutils experts might take a look into.
Not quite sure who, but @hauke , @robimarko and @PolynomialDivision have been active regarding gcc and binutils.
Some other downstream users of gcc & binutils do build with > 4k page sizes even on 32 bit architectures usually as a way to support > 16TB block devices. An example would be some NAS distributions. Capping max page size caused binaries to fail to load on those distributions. Can see mention of it if you follow enough links to related bugs and the discussions in the bug reports linked in posts in this thread.
For each OpenWrt arch we could double check all targets only build using 4k pages and if that is the case force max page size to 4k for that architecture. The binaries - both executables and libraries would not longer have the extra padding.
Well, the generic config defines 4 kB pages, so binutils linker should maybe just check that option and if so, then keep the maxpagesize at 4 kB for ELF programs:
But there are many options related to that. Example about ARM64 and PPC:
So, having a foolproof way for downstream will likely be hard. But keeping OpenWrt coherent should be possible, but I am not sure if there is an elegant solution. I have no binutils expertise, so no idea what would be the optimal way to set the option.
And we haven't yet tested if that really is the solution.
It is total BS, 64kB mprotect is of meaning only for Android and aarch64 that can boot with 4k or 64k pages with same binaries. It is a total amok run to protect 7 extra pages on eg MIPS which will never bolt on anything else than 4k pages.
