That's a somewhat useful article but I don't have any USB ports (hence wanting to use RAM) and I don't want to maintain a custom build (hence not just adding the required packages directly to one) so I'm not quite following how it's relevant to my question?
I tried:
mkdir /overlay-mem
mount -t tmpfs -o size=2097152 tmp-overlay /overlay-mem
mkdir /overlay-mem/upper /overlay-mem/work
mount -t overlay -o remount,lowerdir=/,upperdir=/overlay-mem/upper,workdir=/overlay-mem/work overlayfs:tmpfs /
Which didn't error, but didn't work. When I start I have [discarding irrelevant e.g. dev/proc/tmp entries]:
/dev/root on /rom type squashfs (ro,relatime)
/dev/mtdblock5 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlay (ro,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmp-overlay on /overlay-mem type tmpfs (rw,relatime,size=2048k)
and running the final mount doesn't change anything despite echo $? showing 0!
If I drop the remount option and run instead:
mount -t overlay -o lowerdir=/,upperdir=/overlay-mem/upper,workdir=/overlay-mem/work overlayfs:tmpfs /
My mtab does change:
/dev/root on /rom type squashfs (ro,relatime)
/dev/mtdblock5 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmp-overlay on /overlay-mem type tmpfs (rw,relatime,size=2048k)
overlayfs:tmpfs on / type overlay (rw,relatime,lowerdir=/,upperdir=/overlay-mem/upper,workdir=/overlay-mem/work)
But if I touch /testing123 it still creates a file "/overlay/upper/testing123" instead of "/overlay-mem/upper/testing123". And df still shows the same tiny available size for both my overlayfs:tmpfs and the original overlayfs:/overlay:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 3840 3840 0 100% /rom
tmpfs 61700 1100 60600 2% /tmp
/dev/mtdblock5 320 236 84 74% /overlay
overlayfs:/overlay 320 236 84 74% /
tmpfs 512 0 512 0% /dev
tmp-overlay 2048 0 2048 0% /overlay-mem
overlayfs:tmpfs 320 236 84 74% /
When I umount / my own futile attempt goes away, but subsequent calls just happily and silently do nothing. Why does it let me mount my "overlayfs:tmpfs" but then not really use it? Why does trying with the remount option not do anything at all?
Opkg package manager > Non-standard installation destinations
DIR="/etc /etc/init.d /bin /sbin /usr/bin /usr/sbin /usr/lib /usr/lib64"
for DIR in ${DIR}
do find /tmp${DIR} -mindepth 1 -maxdepth 1 -exec \
ln -s {} ${DIR} \; 2> /dev/null
done
2 Likes
I think I tracked down why "remount" does work. As quoted at https://unix.stackexchange.com/a/404008/42732:
[the remount option] does not change device or mount point.
I.e. I think all you can do with remount is tweak other options like ro/noatime. Of course the lowerdir/upperdir settings are options but maybe they don't count because they are rather more ambitious options?
I didn't go that route initially because it sounded unlikely to work based on the notes there. But I went ahead and got:
# opkg -d ram install tcpdump-mini
Installing tcpdump-mini (4.9.3-3) to ram...
Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/base/tcpdump-mini_4.9.3-3_mips_24kc.ipk
Installing libpcap1 (1.9.1-3) to ram...
Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mips_24kc/base/libpcap1_1.9.1-3_mips_24kc.ipk
Configuring libpcap1.
Configuring tcpdump-mini.
Collected errors:
* opkg_conf_write_status_files: Can't open status file //usr/lib/opkg/status: No such file or directory.
Despite the errors, I did end up with an executable at "/tmp/usr/sbin/tcpdump". But it is a rather unhappy executable:
# /tmp/usr/sbin/tcpdump
Error loading shared library libpcap.so.1: No such file or directory (needed by /tmp/usr/sbin/tcpdump)
[…snipped several dozen errors about specific pcap symbols missing…]
I can run it via LD_LIBRARY_PATH=/tmp/usr/lib /tmp/usr/sbin/tcpdump though. So if I can convince Wireshark sshdump to do the same from my main machine then I might be able to live with this. Still curious if there's any way to remount / with a different overlay though — it's unclear to me how the ExtRoot stuff in fstab is doing anything different than I should be able to do from the command line?
The above method with symlinks should basically work.
Remounting may require to switch_root or pivot_root.
That probably needs to reinit, or run at the preinit stage.
1 Like
you may wish to checkout this for inspiration
what you are attempting to do is fairly specialized so anticipate doing most of the gruntwork yourself...
1 Like
Ah, that's probably the key difference then.
Not going to try figure out the exact OpenWrt boot sequence (there's some general Linux discussion at e.g. https://unix.stackexchange.com/questions/107828/how-is-etc-fstab-accessed-before-root-is-mounted and https://unix.stackexchange.com/questions/242747/what-is-the-purpose-of-listing-the-rootfs-in-etc-fstab one could start with in a pinch) but as the default /etc/fstab is completely empty it's probably safe to assume the initial overlayfs gets set up at a special time and setting up a new one is unlikely once the system is up and running.
Just to re-iterate and followup in the context of (one of) my original goals:
It is possible to use this half-installed tcpdump via a Wireshark sshdump session. The trick is that the "Remote capture command" sort of overrides a lot of the other options (at least in the macOS GUI of Wireshark v3.8.4) so you need a more complete command with all the interface/filter/output stuff baked in like:
LD_LIBRARY_PATH=/tmp/usr/lib /tmp/usr/sbin/tcpdump -U -i 'eth0' -w - 'not ((host 10.0.0.142 or host clientmachine::whatever::42) and port 22)'
Or, the symlink trick added into Can I stack on a second overlayfs from RAM to temporarily install more packages? - #5 by vgaetera might avoid the need for this. Pick your poison I guess 
21.02 release builds are failing for WPA8630Pv2 due to small available space (6MiB), so it looks like the only option is to move it to the tiny target. This works surprisingly well with almost no loss in functionality, with 1MB free space: https://github.com/openwrt/openwrt/pull/4500 . Hopefully it can be backported to 21.02...
1 Like
are you sure you can't expand root partition by adding other non-contiguous partitions with mtd-concat driver?
see here for mtd-concat https://github.com/openwrt/openwrt/blob/ec6293febc244d187e71a6e54f44920be679cde4/target/linux/generic/pending-5.4/497-mtd-mtdconcat-add-dt-driver-for-concat-devices.patch
1 Like
This 6MB available already includes the os-image and file-system.
mtd-concat looks very cool though, I don't know why it wasn't considered in the initial support for this device.
Here are the partition layouts of all the models. There is a 1MB region for the PLC firmware (default-nvm and default-pib which could be used by OpenWRT. They are included in all known firmware files, so they are recoverable.
However one problem is that the only known way of reverting back to stock for this unit without opening it up is by using tplink-safeloader convert_firmware to make a sysupgrade image from the OEM firmwares. Currently that only supports restoring os-image and file-system and partition-table, so that would need to be extended to include those extra partitions (e.g. by specifying for each device). Maybe there is a way of triggering the OEM HTTP restore using uboot-envtools (which would be awesome), but noone has found it yet.
In contrast, the tiny target looks easy to achieve giving 1MB free without too many changes, so it's probably more likely at this stage. mtd-concat is worth looking into though...
afaik it is an OpenWrt kernel patch, so it is not well-known, many contributors don't know it exists. Which is why I always ask when I see these situations.
I don't think you need to add these partitions to the tplink-safeloader.
If I understand correctly the wiki entry, the powerline will still work fine since it has its own SPI chip with the firmware https://openwrt.org/toh/tp-link/tl-wpa8630p_v2#powerline_firmware_and_configuration
So the device will still be able to go back to stock firmware and work correctly without modifying the tplink-safeloader.
Those partitions will now contain garbage but according to wiki they are only used to store new PLC firmware during stock firmware upgrade procedure (WTF TP-Link?! Why waste flash like that).
During stock firmware upgrade they will be overwritten with the new PLC firmware blobs from the new firmware image anyway, regardless of previous contents.
But I don't have the device, so I can't guarantee this.
afaik you can have both at the same time, and they are two different changes so you can make two independent PRs.
1 Like
Yes, that's worth testing. If there being garbage in the PLC partitions temporarily doesn't cause any issues until the next OEM flash (e.g. PLC bricking from either the firmware or PIB?). The PLC has its own SPI flash, so it should be safe to backup everything and test this out.
1 Like
To use mtd-concat, it looks like the main blocker is that tplink-safeloader build_image() doesn't yet support splitting up the OpenWRT firmware across multiple non-contiguous partitions. build_image(), read_file() and maybe a few others would need to be changed. Looks doable but tricky.
Glad to see many others interested in this device and the storage space situation. I still haven't gotten to 100% proving this out but fwiw I did spent some more time looking into how the "ram-root" code mentioned here works:
I think the scripts in that download are more ambitious/flexible than than they strictly "need" to be for my purposes (quick spin up more space temporarily) but iiuc they do show the answer to the walls I hit earlier with just mount itself. I think the key trick is to use pivot_root, something roughly like:
mount -o noatime,nodiratime,move /proc $NEW_ROOT/proc
pivot_root $NEW_ROOT $NEW_ROOT/$OLD_ROOT
for mount_dir in /dev /sys /tmp; do
mount -o noatime,nodiratime,move /$OLD_ROOT$mount_dir $mount_dir
done
mount -o noatime,nodiratime,move $NEW_OVERLAY /overlay
Again the above is untested by me and just cribbed from the ram-root.sh from the Extroot using (z)ram drive with the backup&restore option over NFS share - #2 by lleachii thread @anon50098793 linked to.
(Oh, and I see Can I stack on a second overlayfs from RAM to temporarily install more packages? - #8 by vgaetera mentions this trick also.)
1 Like
Here is an unofficial BETA build of 21.02.1 using the ath79-tiny patch mentioned above. This should be identical to the official release images if the patch gets merged in (and 21.02.2 is released). After installing, there is 1MB free space, and that includes LuCi.
Keep in mind this is a "BETA" release, so you if you don't want to risk anything you can wait until its officially included.
Release note:
If you are sysupgrading from the earlier ath79-generic snapshot images to this new ath79-tiny image, you will see this message and should follow its instructions:
The flash erase blocksize has changed to 4k from the 64k in previous versions, so the JFFS2 settings partition MUST be reformatted to avoid data corruption. Backup your settings before upgrading, then during sysupgrade, de-select "Keep settings" and select "Force" to continue (equivilant to "sysupgrade -n -F)
If you forget to do this and your system becomes unstable after upgrading, you can do this to format the partition and recover:
Reboot Press RESET when Power LED blinks during boot to enter Failsafe mode SSH to 192.168.1.1 Run "firstboot" and reboot
If anyone tests it, let me know how you get on!
The images should be safe to use, and definitely much safer than making custom cut-down images with ImageBuilder, which can be a loaded foot-gun as removing the wrong package can make the unit inaccessible. (Happened to me before...)
1 Like
I've tested your unofficial 21.02.1 build on a TP-Link WPA8630P v2.1 (EU) and it works well. Thank you!
(I upgraded from a previous development build to I haven't tested the factory image)
I think I figured out how to mount a tmpfs overlay on root. The following works on my device, and Id think it would work on others, but idk for sure how portable it is.
Basically, this sets up an identical root filesystem at /newroot using bind mounts, with the only difference being that the overlay uses a tmpfs directory for upper overlay, and uses the /overlay/upper:/rom for lower overlay.
This makes it so that when when you write to the filesystem at / it goes to persistent overlay, and when you write to the filesystem at /newroot it goes to tmpfs overlay.
running pivot_root /newroot /newroot/oldroot switches the system to use the tmpfs overlay.
running pivot_root /oldroot /oldroot/newroot switches the system back to using the persistent overlay. Rebooting also would do this, but is not required to get the original configuration back.
Heres the code. Hope it is of some use to you.
# create directories and setup a dedicated tmpfs for the overlay
mkdir -p /newroot /oldroot /overlay_tmpfs
mount -t tmpfs tmpfs /overlay_tmpfs
chmod 755 /newroot /oldroot /overlay_tmpfs
mkdir -p /overlay_tmpfs/rom /overlay_tmpfs/lower /overlay_tmpfs/upper /overlay_tmpfs/work
# setup read-only bind mounts of '/rom' and '/overlay/upper' into '/overlay_tmpfs'
mount -o bind,ro /rom /overlay_tmpfs/rom
mount -o bind,ro /overlay/upper /overlay_tmpfs/lower
# setup tmpfs overlay (tmpfs <-- /overlay/upper <-- /rom) at '/newroot'
mount -t overlay -o rw,noatime,lowerdir=/overlay_tmpfs/lower:/overlay_tmpfs/rom,upperdir=/overlay_tmpfs/upper,workdir=/overlay_tmpfs/work/ overlayfs:/overlay_tmpfs /newroot
# bind-mount everything else into '/newroot'
cat /proc/mounts | awk '{print $2}' | grep -Fv '/newroot' | grep -Ev '^\/$' | while read -r nn; do mkdir -p "/newroot${nn}"; findmnt "/newroot${nn}" 1>/dev/null 2>/dev/null || mount -o bind "${nn}" "/newroot${nn}"; done
# THIS ENABLES THE TMPFS OVERLAY
# pivot root to '/newroot'
pivot_root /newroot /newroot/oldroot
wifi up
/etc/init.d/wpad restart
/etc/init.d/network restart
# THIS DISABLES THE TMPFS OVERLAY
# pivot root to '/oldroot'
pivot_root /oldroot /oldroot/newroot
wifi up
/etc/init.d/wpad restart
/etc/init.d/network restart
2 Likes
Yep I also started thinking on 4/32 devices it could be a valid strategy to
- expand overlay with tmpfs ramdisk
- re-install
luci-sslon every boot
Will give this a shot.