Help creating extroot (and custom image for it)

Hello,
I have a TP-Link TD-W8970 v1 router that I have used with OpenWRT (ROOter flavour) for a long time. Recently I have updated the firmware inside the router with a vanilla OpenWRT version 23.05.2 downloaded from https://firmware-selector.openwrt.org/?version=23.05.2&target=lantiq%2Fxrx200&id=tplink_tdw8970

The vanilla firmware doesn't do everything I want, as I may need to attach to it and use a Huawei 4G USB modem (which is why I was using ROOter originally) as an emergency measure if my new 4G router (which I use just as a modem) fails. The TP-Link router doesn't have enough space to install all the required packages for it post-installation, so I have used image builder (actually, the request build feature within the firmware download page for the device) to include the following packages: kmod-usb-net-huawei-cdc-ncm luci-proto-ncm picocom comgt-ncm kmod-usb-serial-option in addition to those already listed on the webpage.
The generated firmware gets installed fine and everything works. For reference later, this is the output of df with this firmware installed:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 4608      4608         0 100% /rom
tmpfs                    27752       108     27644   0% /tmp
/dev/mtdblock4             448       232       216  52% /overlay
overlayfs:/overlay         448       232       216  52% /
tmpfs                      512         0       512   0% /dev

Also for reference later, when I go to the Software page in Luci I have 216 KiB of free space to install further packages.

Now, despite everything working with this configuration, I would like to install some other packages, although they're not a must. For this, I thought I'd go the extroot way. Again, even with a vanilla firmware, I don't have enough space to install all the required packages, so I built a different custom firmware using the webpage. This time, I added the following packages (which are also those suggested in the OpenWRT wiki in the extroot guide page), again on top of those already present in the webpage: block-mount kmod-fs-ext4 kmod-usb-storage kmod-usb-ehci. Notice that since my router has USB2 ports I decided to install kmod-usb-ehci and skipped kmod-usb-ohci and kmod-usb-uhci instead. I have formatted a 1 GB pen drive with ext4 and it gets recognised by the firmware, so I assume the packages I installed are enough.
However, I'm having all sorts of issues with this firmware, which is basically unusable. To begin with, this is the output of df:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 5120      5120         0 100% /rom
tmpfs                    27752       116     27636   0% /tmp
tmpfs                    27752        80     27672   0% /tmp/root
overlayfs:/tmp/root      27752        80     27672   0% /
tmpfs                      512         0       512   0% /dev

This is the output of block info:

/dev/mtdblock3: UUID="72f49f62-19221b68-08514f4a-36428fae" VERSION="4.0" MOUNT="/rom" TYPE="squashfs"
/dev/sda1: UUID="aab27360-8f9b-4b75-8b97-36d4574337ec" LABEL="extroot" VERSION="1.0" TYPE="ext4"

And this is the output of block detect:

config 'global'
        option  anon_swap       '0'
        option  anon_mount      '0'
        option  auto_swap       '1'
        option  auto_mount      '1'
        option  delay_root      '5'
        option  check_fs        '0'

config 'mount'
        option  target  '/mnt/sda1'
        option  uuid    'aab27360-8f9b-4b75-8b97-36d4574337ec'
        option  enabled '0'

As you can see, /overlay doesn't get created (the folder is present in the filesystem, but it's empty). As a result, any configuration that I apply after firmware installation doesn't survive a reboot. Further, if I go to the Software page in Luci the available space is now about 27 MiB, which must be wrong if you compare it to the value I get with a working firmware. Obviously, I cannot proceed with the extroot creation process, and I had to revert the firmware to the one I talked about at the beginning of the post.

Does anyone have any idea of what's happening, and how can I solve the issue? In case someone asks, both firmware images are below the 8 MB of the router flash space (the first one is 7.2 MB and the second one 7.4 MB).

Your fstab is showing that the drive is nominally mounting as /mnt/sda1 instead of /

This is the extroot recipe I use:

mount /dev/sda1 /mnt ; tar -C /overlay -cvf - . | tar -C /mnt -xf - ; umount /mnt

block detect > /etc/config/fstab; \
   sed -i s/option$'\t'enabled$'\t'\'0\'/option$'\t'enabled$'\t'\'1\'/ /etc/config/fstab; \
   sed -i s#/mnt/sda1#/overlay# /etc/config/fstab; \
   cat /etc/config/fstab;

The second part will create an fstab entry that will properly mount the usb drive a / and therefore the overlay will be on the external drive.

1 Like

I tried to do that part, but it doesn't work. As I said, any configuration I add after installation is lost on reboot, including that one. Besides, forget about the usb drive for a moment. Shouldn't /overlay be created properly by mounting /dev/mtdblock4 (as it happens with the first custom or the vanilla firmwares) before I configure the extroot? None of that happens with the second custom firmware.

Try resetting and starting from scratch.

Do these steps first to ensure your flash drive is empty.

mount /dev/sda1 /mnt; rm -rf /mnt; umount /mnt

(it will throw an error at the end that it cannot delete /mnt because it is busy, ignore that).

Then pick up exactly as I showed in my recipe above.
Copy and paste any output/errors so that we can see what is happening, and then reboot and check

mount
df -h

I always installed the different firmwares using sysupgrade -n each time; correct me if I'm wrong but that should equate to your request of resetting. I formatted the pen on my linux laptop just before inserting it into the router but earlier today I've also tried to mount it in the router and I've verified that it's empty.
In your recipe from above (I did try to follow the guide on the OpenWRT website and then some different command in the forum before I realised what it was that was going wrong) I already fail at the first command, as /overlay is empty (I think what should be in /overlay is actually in /tmp/root), so tar exits with an error (I don't remember it exactly, but something about it being short; that makes sense because the created tar file is actually empty).

I also have a TD-W8970 v1 and the same lack-of-memory issue; I always build my image with Imagebuilder instead of using Firmware Selector and Extroot works like a charm since OpenWrt 21.xx.x.: if you are willing to use Imagebuilder, I can share with you my "how to" steps hoping they can help you...

Anyway, to do the job I always use the following packages:

* kmod-usb-core
* kmod-usb-dwc2
* kmod-usb-ehci
* kmod-usb-ledtrig-usbport
* kmod-usb-roles
* kmod-usb-storage 
* kmod-usb-uhci
* kmod-usb2 
* block-mount
* kmod-fs-ext4
* e2fsprogs
* fdisk

I'd like to specify that I use kmod-usb-uhci (instead of kmod-usb-ohci) only to drive an USB 1.1 printer.

I tried using imagebuilder in a VM as well to build the firmware. At the moment I can't quite remember which firmware I tried to build with it (maybe the extroot version) but I think I was having the same issue as with the version created from the website (I told the program to use the same packages I listed in my first post). However, if you tell me your configuration works with your router I'll definitely give it a try. I don't have time this time around (I'm leaving in a few hours), but next time I'll be here at my parents' place I'll try and update you all. Just a curiosity: I don't see luci listed in your packages, but imagebuilder doesn't load it by default. Do you not install the web interface at all? Also, have you tried building and installing 23.05.2 or which is the latest version you tried?

I'm currently running a custom 23.05.2 image I've built with the following make image command:

make image PROFILE="tplink_tdw8970" PACKAGES="base-files br2684ctl bspatch busybox ca-bundle cgi-io -dnsmasq dropbear -dsl-vrx200-firmware-xdsl-a -dsl-vrx200-firmware-xdsl-b-patch firewall4 fstools fwtool getrandom jansson4 jshn jsonfilter kernel -kmod-atm kmod-crypto-acompress kmod-crypto-aead kmod-crypto-crc32c kmod-crypto-des kmod-crypto-hash kmod-crypto-manager kmod-crypto-null kmod-gpio-button-hotplug kmod-leds-gpio kmod-lib-crc-ccitt kmod-lib-crc32c kmod-lib-lzo -kmod-ltq-atm-vr9 kmod-ltq-deu-vr9 kmod-ltq-ifxos kmod-ltq-ptm-vr9 -kmod-ltq-vdsl-vr9 -kmod-ltq-vdsl-vr9-mei kmod-ltq-vectoring kmod-nf-conntrack kmod-nf-conntrack6 kmod-nf-flow kmod-nf-log kmod-nf-log6 kmod-nf-nat kmod-nf-reject kmod-nf-reject6 kmod-nfnetlink kmod-nft-core kmod-nft-fib kmod-nft-nat kmod-nft-offload -kmod-ppp -kmod-pppoa -kmod-pppoe -kmod-pppox kmod-slhc libblobmsg-json20230523 libbz2-1.0 libc libgcc1 libiwinfo-data libiwinfo20230701 libjson-c5 libjson-script20230523 liblucihttp-ucode liblucihttp0 libmbedtls12 libmnl0 libnftnl11 libnl-tiny1 libpthread librt libubox20230523 libubus20230605 libuci20130104 libuclient20201210 libucode20230711 libustream-mbedtls20201210 -linux-atm logd -ltq-dsl-base -ltq-vdsl-vr9-app -ltq-vdsl-vr9-vectoring-fw-installer luci luci-app-firewall luci-app-opkg luci-base luci-light luci-mod-admin-full luci-mod-network luci-mod-status luci-mod-system -luci-proto-ipv6 -luci-proto-ppp luci-ssl luci-theme-bootstrap mtd netifd nftables-json -odhcp6c -odhcpd-ipv6only openwrt-keyring opkg -ppp -ppp-mod-pppoa -ppp-mod-pppoe procd procd-seccomp procd-ujail px5g-mbedtls rpcd rpcd-mod-file rpcd-mod-iwinfo rpcd-mod-luci rpcd-mod-rrdns rpcd-mod-ucode ubi-utils ubox ubus ubusd uci uclient-fetch ucode ucode-mod-fs ucode-mod-html ucode-mod-math ucode-mod-ubus ucode-mod-uci uhttpd uhttpd-mod-ubus urandom-seed urngd usign kmod-fs-ext4 e2fsprogs fdisk zram-swap block-mount kmod-usb-storage kmod-usb2 kmod-usb-uhci " FILES="files" BIN_DIR="/home/openWRT/23.05.2"

Then, as this image already contains all the packages needed for Extroot (i.e. what I listed in my previous answer), I follow exactly what described here https://openwrt.org/docs/guide-user/additional-software/extroot_configuration?s[]=overlay from step 2 to step 6.

Hope this helps.

Perfect, thank you very much! I'll give it a try at the next occasion and I'll let you know how it goes.

I'm resuming this old conversation because I've reached my parents' place and I've managed to setup the extroot. Thank you @WildCarrot for your help! I tried your configuration but that killed the wifi. Still, your comment made me realise that I had to include all the packets so I ended up installing the default image, getting a list of all installed packages and then generating a custom build based on that. This is my final package configuration (I used the image builder from the openwrt website rather than installing it locally):

base-files br2684ctl bspatch busybox ca-bundle cgi-io dnsmasq dropbear -dsl-vrx200-firmware-xdsl-a -dsl-vrx200-firmware-xdsl-b-patch firewall4 fstools fwtool getrandom hostapd-common iw iwinfo jansson4 jshn jsonfilter kernel kmod-ath kmod-ath9k kmod-ath9k-common -kmod-atm kmod-cfg80211 kmod-crypto-acompress kmod-crypto-aead kmod-crypto-ccm kmod-crypto-cmac kmod-crypto-crc32c kmod-crypto-ctr kmod-crypto-des kmod-crypto-gcm kmod-crypto-gf128 kmod-crypto-ghash kmod-crypto-hash kmod-crypto-hmac kmod-crypto-manager kmod-crypto-null kmod-crypto-rng kmod-crypto-seqiv kmod-crypto-sha512 kmod-gpio-button-hotplug kmod-leds-gpio kmod-lib-crc-ccitt kmod-lib-crc32c kmod-lib-lzo -kmod-ltq-atm-vr9 kmod-ltq-deu-vr9 kmod-ltq-ifxos kmod-ltq-ptm-vr9 -kmod-ltq-vdsl-vr9 -kmod-ltq-vdsl-vr9-mei kmod-ltq-vectoring kmod-mac80211 kmod-nf-conntrack kmod-nf-conntrack6 kmod-nf-flow kmod-nf-log kmod-nf-log6 kmod-nf-nat kmod-nf-reject kmod-nf-reject6 kmod-nfnetlink kmod-nft-core kmod-nft-fib kmod-nft-nat kmod-nft-offload kmod-nls-base -kmod-ppp -kmod-pppoa -kmod-pppoe -kmod-pppox kmod-random-core kmod-slhc kmod-usb-core kmod-usb-dwc2 kmod-usb-ledtrig-usbport kmod-usb-roles libblobmsg-json20230523 libbz2-1.0 libc libgcc1 libiwinfo-data libiwinfo20230701 libjson-c5 libjson-script20230523 liblucihttp-ucode liblucihttp0 libmbedtls12 libmnl0 libnftnl11 libnl-tiny1 libpthread librt libubox20230523 libubus20230605 libuci20130104 libuclient20201210 libucode20230711 libustream-mbedtls20201210 -linux-atm logd -ltq-dsl-base -ltq-vdsl-vr9-app -ltq-vdsl-vr9-vectoring-fw-installer luci luci-app-firewall luci-app-opkg luci-base luci-light luci-mod-admin-full luci-mod-network luci-mod-status luci-mod-system luci-proto-ipv6 luci-proto-ppp luci-ssl luci-theme-bootstrap mtd netifd nftables-json odhcp6c odhcpd-ipv6only openwrt-keyring opkg -ppp -ppp-mod-pppoa -ppp-mod-pppoe procd procd-seccomp procd-ujail px5g-mbedtls rpcd rpcd-mod-file rpcd-mod-iwinfo rpcd-mod-luci rpcd-mod-rrdns rpcd-mod-ucode ubi-utils ubox ubus ubusd uci uclient-fetch ucode ucode-mod-fs ucode-mod-html ucode-mod-math ucode-mod-nl80211 ucode-mod-rtnl ucode-mod-ubus ucode-mod-uci ucode-mod-uloop uhttpd uhttpd-mod-ubus urandom-seed urngd usign wireless-regdb wpad-basic-mbedtls block-mount kmod-fs-ext4 e2fsprogs kmod-usb-storage kmod-usb2 kmod-usb-uhci kmod-usb-ehci kmod-zram zram-swap

Thank you again! We can consider this closed now.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.