Have installed a local buildroot and put my local template config into squasfs.
Fine.
But df shows me that I still have 60% (268 of rare 448 blocks) of /overlay in use.
In /overlay/upper, I find old versions of my safed config.
I tried firstboot, jffs2reset and manual remove of the files in /overlay/upper, but after every reboot, they where back again.
How can I get rid of them?
You have a custom image, and you're asking us why it doesn't work ?
Good joke 
My image does work as expected, but some of the building blocks don't.
I did not fiddle with the internals of overlayfs, jffs2reset or firstboot.
I just was expecting that upper of overlay whould be quite empty after that process, as described here:
https://openwrt.org/docs/guide-user/troubleshooting/failsafe_and_factory_reset#factory_reset
Both commands even "threaten" to do what I want:
Erase all config and installed packages (in the overlay) and revert to "factory" (my build) - but after reboot, it's still there.
Less than half a megabyte of free flash means space on your image is very tight.
On first run the scripts in /etc/uci-defaults are run (then deleted but you can still see them in /rom/etc/uci-defaults) which amongst other things produce the various default config files which all take up some space.
First step: confirm you really are that limited in flash space - is the device really that old/limited or have you made some mistakes or poor choices building the image making it too large. So describe the device and what you have included in the image and how it is built.
Did a sysupgrade -n -v now.
Part of my old files (outside of /etc) is gone now, so are the config files for services not even installed in the custom image any more.
However, /etc/config is repopulated in the jffs2 aka upper whith what looks like a mere copy of the lower in squashfs.
May be there are reasons to do so, but it's no, but it's not the behaviur I'd expect after reading documentation or programs "help".
And in my particulare case, it' sheer waste of rare flash: 4 k firewall configuration for the firewall which I could not manage to exclude from build, and another 4k for collectd, which is identical on all devices and perfectly kept in squashfs.
Beyond that, df says 236 k used in overlay, but du can only find 26 k of data.
The rest is overhead? Hidden magic?
confirm you really are that limited in flash space
yes, that's the point.
I tried skd of extroot to net storage, but gave up on that.
It's a TP-Link CPE210 directional outdoor Access Point with 8MB of flash.
I require full hostapd for hostapd.wpa_psk, snmpd,lldpd and perl to integrate it in my infrastructure.
Luci and opkg are somewhat "luxury" - I even was happy that it fitted to flash beyond my expectation.
The firewalf and netfilter stuff I think I could drop, but did not manage to do so.
I still have inet6 addresses in lo and br-lan, and luci for ipv6 and ppp.
Maybe, I could drop the CA-bundle, since opkg is still operable without.
And heck - I don't get https connection on luci ..
This is my package selection
(used bash syntax to concat long line strings to be finally delivered to make)
:~/test/openwrt/imageBuilder/openwrt-imagebuilder-23.05.5-ath79-generic.Linux-x86_64$ cat do_make_image.sh
#!/bin/bash
# https://openwrt.org/docs/guide-user/additional-software/imagebuilder#building_image
# https://openwrt.org/docs/guide-user/additional-software/saving_space
PKG="base-files busybox dropbear fstools kmod-ath9k kmod-gpio-button-hotplug libc libgcc logd mtd netifd procd procd-seccomp procd-ujail swconfig uboot-envtools uci uclient-fetch urandom-seed urngd"
# PKG="$PKG -ppp -ppp-mod-pppoe -firewall4 -nftables -odhcpd -dnsmasq " # -ca-bundle"
PKG="$PKG hostapd-mbedtls -wpad-basic-mbedtls -wpad -wpad-basic"
PKG="$PKG collectd collectd-mod-cpu collectd-mod-df collectd-mod-iwinfo collectd-mod-network collectd-mod-wireless"
PKG="$PKG lldpd snmpd zlib "
PKG="$PKG opkg bridge perl rsync "
PKG="$PKG luci"
PKG="$PKG -ppp -ppp-mod-pppoe -firewall4 -nftables -odhcpd -dnsmasq " # -ca-bundle"
PKG="$PKG -firewall4 -fwtool -luci-proto-ipv6 -luci-proto-ppp -odhcp6c -odhcpd-ipv6only"
make image \
PROFILE="tplink_cpe210-v3" \
PACKAGES="$PKG" \
CONFIG_IPV6=n \
DISABLED_SERVICES="dnsmasq firewall odhcpd" \
FILES="files"
And this is what came out.
$ cat openwrt-23.05.5-ath79-generic-tplink_cpe210-v3.manifest
base-files - 1562-r24106-10cc5fcd00
bridge - 1.7.1-1
busybox - 1.36.1-1
ca-bundle - 20230311-1
cgi-io - 2022-08-10-901b0f04-21
collectd - 5.12.0-49
collectd-mod-cpu - 5.12.0-49
collectd-mod-df - 5.12.0-49
collectd-mod-iwinfo - 5.12.0-49
collectd-mod-network - 5.12.0-49
collectd-mod-wireless - 5.12.0-49
dropbear - 2022.82-6
firewall4 - 2023-09-01-598d9fbb-1
fstools - 2023-02-28-bfe882d5-1
fwtool - 2019-11-12-8f7fe925-1
getrandom - 2022-08-13-4c7b720b-2
hostapd-common - 2023-09-08-e5ccbfc6-8
hostapd-mbedtls - 2023-09-08-e5ccbfc6-8
iw - 5.19-1
iwinfo - 2023-07-01-ca79f641-1
jansson4 - 2.14-3
jshn - 2023-05-23-75a3b870-1
jsonfilter - 2024-01-23-594cfa86-1
kernel - 5.15.167-1-da89f936189e8280762513898b74a850
kmod-ath - 5.15.167+6.1.110-1-1
kmod-ath9k - 5.15.167+6.1.110-1-1
kmod-ath9k-common - 5.15.167+6.1.110-1-1
kmod-cfg80211 - 5.15.167+6.1.110-1-1
kmod-crypto-aead - 5.15.167-1
kmod-crypto-ccm - 5.15.167-1
kmod-crypto-cmac - 5.15.167-1
kmod-crypto-crc32c - 5.15.167-1
kmod-crypto-ctr - 5.15.167-1
kmod-crypto-gcm - 5.15.167-1
kmod-crypto-gf128 - 5.15.167-1
kmod-crypto-ghash - 5.15.167-1
kmod-crypto-hash - 5.15.167-1
kmod-crypto-hmac - 5.15.167-1
kmod-crypto-manager - 5.15.167-1
kmod-crypto-null - 5.15.167-1
kmod-crypto-rng - 5.15.167-1
kmod-crypto-seqiv - 5.15.167-1
kmod-crypto-sha512 - 5.15.167-1
kmod-gpio-button-hotplug - 5.15.167-3
kmod-lib-crc32c - 5.15.167-1
kmod-mac80211 - 5.15.167+6.1.110-1-1
kmod-nf-conntrack - 5.15.167-1
kmod-nf-conntrack6 - 5.15.167-1
kmod-nf-flow - 5.15.167-1
kmod-nf-log - 5.15.167-1
kmod-nf-log6 - 5.15.167-1
kmod-nf-nat - 5.15.167-1
kmod-nf-reject - 5.15.167-1
kmod-nf-reject6 - 5.15.167-1
kmod-nfnetlink - 5.15.167-1
kmod-nft-core - 5.15.167-1
kmod-nft-fib - 5.15.167-1
kmod-nft-nat - 5.15.167-1
kmod-nft-offload - 5.15.167-1
kmod-random-core - 5.15.167-1
libblobmsg-json20230523 - 2023-05-23-75a3b870-1
libc - 1.2.4-4
libcap - 2.69-1
libevent2-7 - 2.1.12-1
libgcc1 - 12.3.0-4
libiwinfo-data - 2023-07-01-ca79f641-1
libiwinfo20230701 - 2023-07-01-ca79f641-1
libjson-c5 - 0.16-3
libjson-script20230523 - 2023-05-23-75a3b870-1
libltdl7 - 2.4.7-1
liblua5.1.5 - 5.1.5-11
liblucihttp-ucode - 2023-03-15-9b5b683f-1
liblucihttp0 - 2023-03-15-9b5b683f-1
libmbedtls12 - 2.28.9-1
libmnl0 - 1.0.5-1
libnetsnmp - 5.9.1-7
libnftnl11 - 1.2.6-1
libnl-tiny1 - 2023-07-27-bc92a280-1
libpci - 3.10.0-1
libpcre2 - 10.42-1
libpopt0 - 1.19-1
libpthread - 1.2.4-4
libubox20230523 - 2023-05-23-75a3b870-1
libubus20230605 - 2023-06-05-f787c97b-1
libuci20130104 - 2023-08-10-5781664d-1
libuclient20201210 - 2023-04-13-007d9454-1
libucode20230711 - 2024-07-11-1a8a0bcf-3
libustream-mbedtls20201210 - 2023-02-25-498f6e26-1
lldpd - 1.0.17-5
logd - 2022-08-13-4c7b720b-2
luci - git-24.346.66847-1bb28ba
luci-app-firewall - git-24.346.66847-1bb28ba
luci-app-opkg - git-24.346.66847-1bb28ba
luci-base - git-24.346.66847-1bb28ba
luci-light - git-24.346.66847-1bb28ba
luci-mod-admin-full - git-24.346.66847-1bb28ba
luci-mod-network - git-24.346.66847-1bb28ba
luci-mod-status - git-24.346.66847-1bb28ba
luci-mod-system - git-24.346.66847-1bb28ba
luci-proto-ipv6 - git-24.346.66847-1bb28ba
luci-proto-ppp - git-24.346.66847-1bb28ba
luci-theme-bootstrap - git-24.346.66847-1bb28ba
mtd - 26
netifd - 2024-01-04-c18cc79d-2
nftables-json - 1.0.8-1
openwrt-keyring - 2022-03-25-62471e69-2
opkg - 2022-02-24-d038e5b6-2
perl - 5.28.1-9
procd - 2023-06-25-2db83655-2
procd-seccomp - 2023-06-25-2db83655-2
procd-ujail - 2023-06-25-2db83655-2
rpcd - 2023-07-01-c07ab2f9-1
rpcd-mod-file - 2023-07-01-c07ab2f9-1
rpcd-mod-iwinfo - 2023-07-01-c07ab2f9-1
rpcd-mod-luci - 20240305-1
rpcd-mod-rrdns - 20170710
rpcd-mod-ucode - 2023-07-01-c07ab2f9-1
rssileds - 4
rsync - 3.2.7-1
snmpd - 5.9.1-7
swconfig - 12
uboot-envtools - 2023.04-1
ubox - 2022-08-13-4c7b720b-2
ubus - 2023-06-05-f787c97b-1
ubusd - 2023-06-05-f787c97b-1
uci - 2023-08-10-5781664d-1
uclient-fetch - 2023-04-13-007d9454-1
ucode - 2024-07-11-1a8a0bcf-3
ucode-mod-fs - 2024-07-11-1a8a0bcf-3
ucode-mod-html - 1
ucode-mod-math - 2024-07-11-1a8a0bcf-3
ucode-mod-nl80211 - 2024-07-11-1a8a0bcf-3
ucode-mod-rtnl - 2024-07-11-1a8a0bcf-3
ucode-mod-ubus - 2024-07-11-1a8a0bcf-3
ucode-mod-uci - 2024-07-11-1a8a0bcf-3
ucode-mod-uloop - 2024-07-11-1a8a0bcf-3
uhttpd - 2023-06-25-34a8a74d-2
uhttpd-mod-ubus - 2023-06-25-34a8a74d-2
urandom-seed - 3
urngd - 2023-11-01-44365eb1-1
usign - 2020-05-23-f1f65026-1
wireless-regdb - 2024.10.07-1
zlib - 1.2.13-1
Ok, I understand.
And they are rerun after jffs2reset and after a rm -rf /overlay/upper/* as well?
And even so, this accounts for the 25 k in /etc, but not for the other 90% 
root@OpenWrt:/etc/config$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 5120 5120 0 100% /rom
tmpfs 28560 72 28488 0% /tmp
tmpfs 28560 68 28492 0% /tmp/root
tmpfs 512 0 512 0% /dev
/dev/mtdblock6 448 236 212 53% /overlay
overlayfs:/overlay 448 236 212 53% /
root@OpenWrt:/etc/config$ du -h -d 1 /overlay/*
24.5K /overlay/upper/etc
1.0K /overlay/upper/usr
25.5K /overlay/upper
0 /overlay/work/work
0 /overlay/work
You think that' not enough for stable long term operation?
So better go without luci and CA-bundle?
For ubifs volumes df doesn't really report space used but an estimate of how much more can be stored and it errors on the side of caution - at least that much can be stored.
see:
Memory Technology Device (MTD) Subsystem for Linux. - Why does df report too little free space?
This seems like a waste of time - Is there a reason you are trying to use such an old device? At some point unless there are unusual circumstances getting new software to run on an old device is more trouble than it's worth. You can strip down OpenWrt and make it a static firmware - even remove opkg / package management - and build and install a new firmware when modifications/updates are desired. Lot of extra work and you lose a major benefit of OpenWrt - that it can be modified in place.
I already have them, and might even consider to get some more.
And I simply haven't found yet any narrow beam directional outdoor AP below 300 € - other than the CPE210.
Might rent an excavator instead and dig a ditch to the middle of the area and place an ordinary round beam AP there.
Or do modern Mu-Mimo a similiar job in automagically focussing narrow beams toward devices far away? (the CPE 210 has Mu-Mimo already)
Testing my build, if found that I forgot some packages
libev libuhttpd-mbedtls px5g-mbedtlsip-bridgehostapd.WPA_PSK (and may be vlan-related config of network and wireless as well) by inwards scp , I need openssh-sftp-serverAfter installing those, I'm left with 56 k free on / aka /overlay.
I'd like to have diff at hand as well, but opkg refuses with pkg diffutils needs 109.
I don't see any other problems yet to run that short on flash space.
However, with dynamic management of dozens of vlans, may be we end up in not being able to edit config files at some at some point?
df doesn't really report space used but an estimate of how much more can be stored and it errors on the side of caution
Ah, that's what I have seen when I installed the packages for https: The figure for free space was falling slower than the amount of kB installed.
root@OpenWrt:/overlay/upper/usr/lib/opkg/info$ du -s -d1 /overlay/upper/
28 /overlay/upper/etc
435 /overlay/upper/usr
463 /overlay/upper/
df ....
/dev/mtdblock6 448 392 56 88% /overlay
Since this was my initial question, you earned the "solution" tag 
But now, towards 'the end', empty space and pkg size installed move in parallel, again.
So either I manage to strip out the unused firewall stuff, or I have to sacrify luci.
Searching the forum, I found many reports of the problem, but not really a solution beyond manual checking of package dependencies.
As it seems, the imagebuilder does evaluate these dependencies, so it might be nice to find some debug output of this process, instead of tedious and error prone manual evaluation.
You are probably correct that an existing deployment of multiple devices is one case where getting a few more years of use can be worth the extra effort.
Might be helpful to use a device with more flash and an easy Factory reset procedure while working out what packages can be removed without things breaking. Won't help with the size issue since it would be a different router but would help you figure out a workable set of packages. Add/remove packages in Luci and if you soft brick the device do a factory reset to get back to the OpenWrt as it was first installed.
well, yes, sort of. That's what I did on DAP-X1860.
However, maybe if I have started Q-Bridge-mib on narrow flash, I had tried it on ash from the beginnin, instead of going the eaysy perl way, that requires 1 MB now...
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.