Mochabin NXP 9089

No, no, no, do not use the globalscale DTB.

Just add:

ranges = <0x81000000 0x0 0xf9010000 0x0 0xf9010000 0x0 0x10000
		  0x82000000 0x0 0xc0000000 0x0 0xc0000000 0x0 0x20000000>;

To the Mochabin DTS, I am using that and it works just fine with QCA6390.

Also, does the card even show up on PCI, use lspci to check

Thanks for the help.

I have changed the mochabin DTS. Sorry for the lame question. Should it look like so?

/* miniPCI-E (J5) */
&cp0_pcie2 {
	status = "okay";

	pinctrl-names = "default";
	pinctrl-0 = <&cp0_pcie_reset_pins>;
	phys = <&cp0_comphy5 2>;
	phy-names = "cp0-pcie2-x1-phy";
	reset-gpio = <&cp0_gpio1 9 GPIO_ACTIVE_LOW>;
	ranges = <0x81000000 0x0 0xf9010000 0x0 0xf9010000 0x0 0x10000
		  0x82000000 0x0 0xc0000000 0x0 0xc0000000 0x0 0x20000000>;
};

Output of lspci after changes:

00:00.0 PCI bridge: Marvell Technology Group Ltd. 88F60x0/88F70x0/88F80x0/CN913x ARM SoC
01:00.0 Unassigned class [ff00]: Qualcomm QCA6390 Wireless Network Adapter

Apologies for the even lamer questions but I'm having the same issues when I've tried using the Openwrt image builder to add the correct software and create me a fresh image. I can post more detail later, but what I see after rebuild is the qca card is no longer recognised and there's some keen.err entries mentioning ath11 in the system log. Looking at the posts above, I'm guessing that's the solution but:

  1. Jeroslav - has it worked
  2. How do I change the Mochabin.dts ? Is it a specific file I can find with SSH access, or is it something I can do through Luci?

Thanks very much

I have changed the dts file in the image builder here:

openwrt-imagebuilder-mvebu-cortexa72.Linux-x86_64/build_dir/target-aarch64_cortex-a72_musl/linux-mvebu_cortexa72/linux-5.15.90/arch/arm64/boot/dts/marvell/armada-7040-mochabin.dts

and then built the image using

make image PROFILE="globalscale_mochabin" PACKAGES="luci"

then you can use the bin file to flash using luci:

openwrt-imagebuilder-mvebu-cortexa72.Linux-x86_64/bin/targets/mvebu/cortexa72

But I am not sure, if those are the right steps. Maybe I am missing the ath11 module. So I think I need to build the image completely from source (my guess).

Ah, I didn't realise it was a file that needed changing before creating the image itself. I'm not that advanced so was actually using their web based tool found here - OpenWrt Firmware Selector

To try to get the wireless card working I then just added a shed load of packages under the "customize installed packages" part to try to match all the packages included on the Mochabin before I messed it up by tinkering with it and trying to see if I could run it on the stable version 22.03.3. Interesting (to me at least) I found that when I tried to add the ath11k-firmware-qca6390 package when creating a custom 22.03.3 image on that website it threw up an error. When I changed to snapshot, all the packages (which I'll copy as a list below) went through OK and it created a usable image. However, the wireless card still isn't recognised, which I guess is down to the changes required to that DTS file. I'm going to try my hand at using the image builder guidance here - https://openwrt.org/docs/guide-user/additional-software/imagebuilder - which I guess at some point will give me that option to make the necessary changes to .dts file. Just in case anyone's interested, what I think are the relevant errors that came up in the system log for the image I created using the web tool are as follows (I've included some extra errors that came up, but the ones I think are relevant are in the middle - I've not included much of the other log to save space):

Thu Feb  2 23:22:12 2023 kern.err kernel: [    0.581708] orion-mdio f212a200.mdio: Cannot register MDIO bus (-517)
Thu Feb  2 23:22:12 2023 kern.err kernel: [    0.752311] leds-is31fl319x 1-0064: Failed to get shutdown gpio: -517

Thu Feb  2 23:22:12 2023 kern.info kernel: [    6.832085] usbcore: registered new interface driver qmi_wwan
Thu Feb  2 23:22:12 2023 kern.info kernel: [    6.838390] usbcore: registered new interface driver rndis_host
Thu Feb  2 23:22:12 2023 kern.info kernel: [    6.871074] ath11k_pci 0000:01:00.0: BAR 0: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)
Thu Feb  2 23:22:12 2023 kern.err kernel: [    6.881077] ath11k_pci 0000:01:00.0: failed to assign pci resource: -22
Thu Feb  2 23:22:12 2023 kern.err kernel: [    6.887753] ath11k_pci 0000:01:00.0: failed to claim device: -22
Thu Feb  2 23:22:12 2023 kern.warn kernel: [    6.893857] ath11k_pci: probe of 0000:01:00.0 failed with error -22
Thu Feb  2 23:22:12 2023 kern.info kernel: [    6.900727] usbcore: registered new interface driver cdc_mbim
Thu Feb  2 23:22:12 2023 user.info kernel: [    6.908629] kmodloader: done loading kernel modules from /etc/modules.d/*
Thu Feb  2 23:22:12 2023 user.notice dnsmasq: DNS rebinding protection is active, will discard upstream RFC1918 responses!

Thu Feb  2 23:22:12 2023 kern.warn kernel: [    0.971360] ahci f2540000.sata: supply ahci not found, using dummy regulator
Thu Feb  2 23:22:12 2023 kern.warn kernel: [    0.978738] ahci f2540000.sata: supply phy not found, using dummy regulator
Thu Feb  2 23:22:12 2023 kern.warn kernel: [    0.985825] platform f2540000.sata:sata-port@0: supply target not found, using dummy regulator
Thu Feb  2 23:22:12 2023 kern.warn kernel: [    0.994892] platform f2540000.sata:sata-port@1: supply target not found, using dummy regulator
Thu Feb  2 23:22:12 2023 kern.warn kernel: [    1.005899] ahci f2540000.sata: masking port_map 0x3 -> 0x3

So for info the package list I include are as below:
ath11k-firmware-qca6390 base-files busybox ca-bundle dnsmasq dropbear e2fsprogs ethtool-full firewall4 fstools hostapd-common iw-full kmod-ath kmod-ath11k kmod-ath11k-pci kmod-cfg80211 kmod-crypto-acompress kmod-crypto-aead kmod-crypto-ccm kmod-crypto-cmac kmod-crypto-ctr kmod-crypto-gcm kmod-crypto-gf128 kmod-crypto-ghash kmod-crypto-hmac kmod-crypto-manager kmod-crypto-michael-mic kmod-crypto-null kmod-crypto-rng kmod-crypto-seqiv kmod-crypto-sha256 kmod-fs-msdos kmod-fs-vfat kmod-gpio-button-hotplug kmod-hwmon-core kmod-lib-lzo kmod-mac80211 kmod-mhi-bus kmod-mii kmod-nft-offload kmod-nls-base kmod-nls-cp437 kmod-nls-iso8859-1 kmod-nls-utf8 kmod-qrtr kmod-qrtr-mhi kmod-thermal kmod-usb-core kmod-usb-net kmod-usb-net-rndis kmod-usb-wdm libatomic1 libblkid1 libblobmsg-json20220515 libc libevdev libgcc libiwinfo-lua libkmod liblucihttp-lua libnl-tiny libpci libubus-lua libudev-zero libusb-1.0-0 libustream-wolfssl logd lua luci luci-lib-base luci-lib-ip luci-lib-jsonc luci-lib-nixio luci-ssl mkf2fs mtd netifd nftables odhcp6c odhcpd-ipv6only opkg partx-utils pciids pciutils ppp ppp-mod-pppoe procd procd-seccomp procd-ujail px5g-wolfssl uboot-envtools uci uclient-fetch umbim uqmi urandom-seed urngd usb-modeswitch usbids usbutils usign wireless-regdb wpad-wolfssl wwan zlib

I don't really know if I need them all, or have the foggiest what most of them mean, but it allowed me to match the software list on the Mochabin that I saw before I then messed it all up. I'm guessing that if I want to ensure all those packages get included when using the image builder I'll need to include them when I run the make image command as follows:

make image PROFILE="globalscale_mochabin" PACKAGES="ath11k-firmware-qca6390 base-files busybox ca-bundle ... AND SO ON"

If it looks to anyone that by doing that I'm going to completely destroy the build, please do warn me. Also as a warning to the less tech savvy people such as myself, if using the custom packages option on https://firmware-selector.openwrt.org/?version=SNAPSHOT&target=mvebu%2Fcortexa72&id=globalscale_mochabin - do make sure to add luci as one of the packages, otherwise you'll have to refresh your SSH skills like I did (and I didn't have many to start off with, so it was a challenge).

Hi Jeroslav

I'm kind of proud that I managed to use that image builder, and made the same changes to the mochabin DTS file as you before copying your "make image" script, but with all the added packages I mentioned (2 removed as explained below) and managed to make my own Openwrt image. The only issue is that after flashing that to my Mochabin I'm now not seeing any wireless adaptor listed under the network tab in Luci, so am a bit stuck. I've run the command suggested by Robimarko and am getting the same as you as copied below:

root@OpenWrt:~# lspci
00:00.0 PCI bridge: Marvell Technology Group Ltd. 88F60x0/88F70x0/88F80x0/CN913x ARM SoC
01:00.0 Unassigned class [ff00]: Qualcomm QCA6390 Wireless Network Adapter

I had a dig around to see if any other commands might help so have copied the output of these below as well:

root@OpenWrt:~# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-wan state DOWN qlen 2048
    link/ether f0##full mac addresses removed## brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1508 qdisc mq state UP qlen 2048
    link/ether f0########### brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-wan state DOWN qlen 2048
    link/ether f0########## brd ff:ff:ff:ff:ff:ff
5: lan0@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether f0:ad:4e:28:9d:c7 brd ff:ff:ff:ff:ff:ff
6: lan1@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f0######### brd ff:ff:ff:ff:ff:ff
7: lan2@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f0######### brd ff:ff:ff:ff:ff:ff
8: lan3@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f0######### brd ff:ff:ff:ff:ff:ff
9: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether f0######### brd ff:ff:ff:ff:ff:ff
10: br-wan: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
    link/ether f0:######### brd ff:ff:ff:ff:ff:ff

For info, when using the full image-builder tool I got errors linked to the following packages when I tried to include them so these were removed:
ethtool-full
libblobmsg-json20220515

And so my full script looked like this:
make image PROFILE="globalscale_mochabin" PACKAGES="ath11k-firmware-qca6390 base-files busybox ca-bundle dnsmasq dropbear e2fsprogs firewall4 fstools hostapd-common iw-full kmod-ath kmod-ath11k kmod-ath11k-pci kmod-cfg80211 kmod-crypto-acompress kmod-crypto-aead kmod-crypto-ccm kmod-crypto-cmac kmod-crypto-ctr kmod-crypto-gcm kmod-crypto-gf128 kmod-crypto-ghash kmod-crypto-hmac kmod-crypto-manager kmod-crypto-michael-mic kmod-crypto-null kmod-crypto-rng kmod-crypto-seqiv kmod-crypto-sha256 kmod-fs-msdos kmod-fs-vfat kmod-gpio-button-hotplug kmod-hwmon-core kmod-lib-lzo kmod-mac80211 kmod-mhi-bus kmod-mii kmod-nft-offload kmod-nls-base kmod-nls-cp437 kmod-nls-iso8859-1 kmod-nls-utf8 kmod-qrtr kmod-qrtr-mhi kmod-thermal kmod-usb-core kmod-usb-net kmod-usb-net-rndis kmod-usb-wdm libatomic1 libblkid1 libc libevdev libgcc libiwinfo-lua libkmod liblucihttp-lua libnl-tiny libpci libubus-lua libudev-zero libusb-1.0-0 libustream-wolfssl logd lua luci luci-lib-base luci-lib-ip luci-lib-jsonc luci-lib-nixio luci-ssl mkf2fs mtd netifd nftables odhcp6c odhcpd-ipv6only opkg partx-utils pciids pciutils ppp ppp-mod-pppoe procd procd-seccomp procd-ujail px5g-wolfssl uboot-envtools uci uclient-fetch umbim uqmi urandom-seed urngd usb-modeswitch usbids usbutils usign wireless-regdb wpad-wolfssl wwan zlib"

I'm guessing that won't have made any difference leaving those two items out, but let me know if it could be relevant. I'm still getting the same ath11 errors in the system log as shown below:

Fri Feb  3 15:45:58 2023 kern.info kernel: [    6.912400] ath11k_pci 0000:01:00.0: BAR 0: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)
Fri Feb  3 15:45:58 2023 kern.err kernel: [    6.922336] ath11k_pci 0000:01:00.0: failed to assign pci resource: -22
Fri Feb  3 15:45:58 2023 kern.err kernel: [    6.928996] ath11k_pci 0000:01:00.0: failed to claim device: -22
Fri Feb  3 15:45:58 2023 kern.warn kernel: [    6.935141] ath11k_pci: probe of 0000:01:00.0 failed with error -22

Any thoughts welcome?

Thanks :slight_smile:

I think the modification to the dts file I did is wrong. Because the issue with memory assignment for PCIe still persist (as you can see in the kernel logs). Well, back to the docs then...

I still dont know, why it is such a big issue for Globalscale to just publish a complete image with all necessary changes. I only got a hint from them, that I should modify the dts and dtb files myself and build the image...

I've still not managed to get the wifi card working - another lesson for me about leaving things be when they are working (for info it was all working OK when I received it from Globalscales). Anyways, in the meantime I've started considering another solution. It's a different enough proposition to warrant a separate topic, but just thought I'd link that other topic here because I'd be interested in the thoughts of some of the posters from this topic - TP Link Omada Controller install and run in Openwrt? - Installing and Using OpenWrt - OpenWrt Forum

Best of luck to anyone still fighting to get their wifi working in Mochabin, I'll be interested to see where you get with it.

Although I managed to get the wifi drivers working properly with a lot of help from Jeroslav and the advice from Robimarko about the ranges addition to the DTS file, I was still faced with numerous issues including getting Luci to install on the latest Snapshot (dated 08/02/23 - it was just hanging on the Luci page, no login box).

Since I'm pretty novice at this stuff I've decided to put the last stable build (22.03.3) on the Mochabin and just use it as a router without wifi for the moment. It's replacing a Brume, and I have a Netgear WAC124 as a dumb wifi AP attached to that anyway so I'll just keep that structure for now. Once a stable build is released that adopts the required drivers, I might revisit using the Mochabin as an all in one router and wifi. I might add external antennae as well, or dependent on what comes up on the forums might even swap the Qualcomm card out for the Mediatek based cards as mentioned by Lord Vader here - Mochabin NXP 9089 - Hardware Questions and Recommendations - again probably adding external antennae. Alternatively, I might go down the route of sticking with seperate WAPs such as the Ubiquiti Unifi 6 ones listed here - Table of Hardware: Full details - wifi 6 WAPs.

I'd still be interested in how anyone else gets on. Best of luck

1 Like

Please share how you got the wifi drivers working. Where do the ranges need to be added to the armada-7040-mochabin.dts?

ranges = <0x81000000 0x0 0xf9010000 0x0 0xf9010000 0x0 0x10000
		  0x82000000 0x0 0xc0000000 0x0 0xc0000000 0x0 0x20000000>;

Anything else that needs to be considered?

Hi Chriz

I got forwarded the rebuild instructions which had a link to a working image, but still had some issues with Luci install etc. I'll hopefully be on the computer I was using later and will send over the instructions along with further detail on some of the issues I faced.

I got a response from Globalscale engineers with built image and following guide. To enter the u-boot console (step 5) please open the serial communication (e.g. via USB-A to micro-USB cable and putty on Windows).

Image
Guide

Good luck. :wink:

Ah Jeroslav beat me to it. That's the guide I used as well. Luci didn't install on the image they provided and when I tried some other snapshot build I got an issue where Luci just hung. If that's still an issue let me know and I'll see if I can share an image I did have Luci working on. Just checking as well, this got my Qualcomm WiFi card working, do you have that one or the NXP card in yours?

It is the Qualcomm WiFi card:
01:00.0 Unassigned class [ff00]: Qualcomm QCA6390 Wireless Network Adapter

I think all OpenWRT Mochabins have the Qualcomm adapter.

@jeroslav Thanks for sharing the files! Let me try them out. I will keep you updated.

This worked like a charm! I did not install the OpenWRT snapshot privded with the image but stopped after updating the recovery ramdisk (step 9) and continued using the ath11k enabled snapshot which I had already flashed and configured:

[    9.390540] ath11k_pci 0000:01:00.0: BAR 0: assigned [mem 0xc0000000-0xc0ffffff 64bit]
[    9.399106] ath11k_pci 0000:01:00.0: MSI vectors: 32
[    9.404120] ath11k_pci 0000:01:00.0: qca6390 hw2.0
[    9.408932] ath11k_pci 0000:01:00.0: FW memory mode: 0
[    9.571036] mhi mhi0: Requested to power ON
[    9.575511] mhi mhi0: Power on setup success
[   10.295940] mhi mhi0: Wait for device to enter SBL or Mission mode
[   10.394623] kmodloader: done loading kernel modules from /etc/modules.d/*
[   10.412064] ath11k_pci 0000:01:00.0: chip_id 0x0 chip_family 0xb board_id 0xff soc_id 0xffffffff
[   10.420910] ath11k_pci 0000:01:00.0: fw_version 0x10121492 fw_build_timestamp 2021-11-04 11:23 fw_build_id

The WiFi card is recognized now and works as expected. Thank you so much for your help!

Awesome, good to hear you got it working. I've ended up opting to stick with a stable build which doesn't allow me to use the built in WiFi card because I have a dumb ap configured anyways. Once there stable build is updated to a version that supports that WiFi card I'll revisit. I'm currently having a headache increasing the overlay size on my Mochabin. I've put a sata ssd in the device but have really struggled getting the overlay to switch to that device. Probably me being a bit of a biff, but I've also discovered that the image I've used only uses about 1GB of the built in eMMc, which is 16GB in total. I'm thinking of I can get the overlay to use 10GB I'll never fill that. Will try to work out out myself but likely end up opening another post to see if anyone knows how to do it. Ideally I'm hoping I'll be able to sort it with a script in image builder so that I can repeat it each time I do an upgrade.

Sorry, I digressed massively. Out of interest how stable are you finding the WiFi?

Brothers in thought! I have also installed a SATA SSD to have the overlay sourced from there. It did not work likely because of: https://github.com/openwrt/openwrt/issues/7352 however it seems this is being fixed currently.

For Mochabin WiFi it does not look like you can configure the ath11k card for both 2,4 GHz and 5 GHz. I have opted for 5 GHz and signal quality is not quite as good as from my old (dedicated) router. Perhaps I am holding it wrong :grinning:
Anyway I have increased Tx power which fixes this issue somewhat. Also ath11k crashes about once a week (not a real crash - the WiFi just seems to be down). Not a big issue as the WiFi only needs to be restarted. However compared to my old setup with no issues at all, i.e. rock-stable for months a step back.

Hi ChriZ

To save you some time on the extending overlay issue I managed to get that sorted over the weekend thanks mainly to another poster (when I say sorted I have a 10GB overlay that uses empty space on the built in eMMc). I'll hopefully get some time to double check the steps taken and then post that this evening or tomorrow. Off the top of my head is roughly 6 or 7 lines to copy into SSH and it's fairly straightforward (even I could do it). Will keep you posted.

Hi Chriz

I'm not going to have time to do a full write up of this tonight, but if you're anything like me you'll want to get your overlay sorted, so here's the untidy version of the full write up I'll do on this once I have time and have tested fully. Apologies if I put in stuff you already know, but just including roughly what I'll put together as a full guide. I'm wondering if the steps I'll detail below work out whether I'll be able to get this added to the device info page for the mochabin. For info, and I'll properly credit the genius of the people who people who's posts I used at a later date, the posts that I used were:

resizing the overlay when it is ext4 and not f2fs - interestingly the first time I managed to increase my overall, the instructions in here worked a bit better. However, when I did a new sysupgrade using the current stable image (which is squashfs) the steps that required you to be working on an ext4 filesystem didn't work, and after some head scratching I found the reason was that the partition my loop0 file was on was f2fs.

gl-mv1000 increasing_the_root_filesystem - from this I just took a couple of lines that allowed me to easily increase the partition size where loop0 resides. (only point 2 actually, and I used the resize option instead of creating and formatting a new partition)

So here goes:

  1. Do a df -h to check on the current partitions/system file sizes:
root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 3.5M      3.5M         0 100% /rom
tmpfs                     3.9G      1.1M      3.9G   0% /tmp
/dev/loop0               98.8M     61.5M     37.3M  62% /overlay
overlayfs:/overlay       98.8M     61.5M     37.3M  62% /
tmpfs                   512.0K         0    512.0K   0% /dev

Then install the packages you'll need by copying an pasting the following lines of code

opkg update
opkg install losetup e2fsprogs resize2fs block-mount kmod-fs-f2fs f2fs-tools cfdisk kmod-fs-ext4 parted rsync

You might not need all of these, it depends if you're running on ext4 or f2fs, but they all seem fairly small anyway.

Then confirm where the loop0 file resides using the following command. My results shown as well:

root@OpenWrt:~# losetup
NAME       SIZELIMIT  OFFSET AUTOCLEAR RO BACK-FILE  DIO LOG-SEC
/dev/loop0         0 3604480         1  0 /mmcblk0p2   0     512

I'm guessing you'll also have your loop0 on mmcblk0p2. If it's different though take note you'll need to change that in the next step.

root@OpenWrt:~# cfdisk /dev/mmcblk0

You can then use the down arrow key to select the second partition and left arrow key to go to “Resize” >> press return. Then use left arrow key to move cursor to new size entry, and change to the size you want, as below. Then press return, select “write” and yes to confirm. Then quit.

I chose not to use the full free space available, but that’s because I’m clueless and I don’t know if some free space is needed for the tmpfs to use, which was showing at 3.9GB. So I just chose to make mmcblk0p2 10GB in total because I can’t see myself ever needing that much anyway. If anyone who actually knows what they’re doing wants to correct me about whether I could use the whole of the free space please do.

I then just copied spence’s instructions as follows:

LOOP="$(losetup -n -O NAME | sort | sed -n -e "1p")"
ROOT="$(losetup -n -O BACK-FILE ${LOOP} | sed -e "s|^|/dev|")"
OFFS="$(losetup -n -O OFFSET ${LOOP})"

I then wanted to check it had done something so ran the following command with results shown (again something I picked up off Spence)

root@OpenWrt:~# echo ${OFFS} ${LOOP} ${ROOT}
3604480 /dev/loop0 /dev/mmcblk0p2

The next 2 lines then went fine as well:

LOOP="$(losetup -f)"
losetup -o ${OFFS} ${LOOP} ${ROOT}

But this is where things got a bit weird. From Spence's instructions, the next steps worked fine when I first ran them. It must be because somehow I had formatted my mmcblk0p2 partition as ext4 (no idea how I'd done that, or if it was because of using a different build image initially. But anyways, this initially worked.

fsck.ext4 -f ${LOOP}

If that command works for you, it may ask you to create a lost and found, just click yes or select whatever default options there are. Then you can run the next lines. If however you get an error, then skip to the lines a bit further down which work for the f2fs filesystem.
Final lines of code if running on ext4:

mount ${LOOP} /mnt
root@R4S-wrt:~# umount ${LOOP}
root@R4S-wrt:~# resize2fs ${LOOP}

If it tells you to "Please run 'e2fsck -f /dev/loop0' first.", then do that, and repeat the resize2fs ${LOOP} command.

Once that's done you might be tempted to do a df -h again and find to your dismay that the size of loop0 hasn't changed. Just do a:

reboot

And try your df -h again.

Different lines of code if you got an error after fsck.ext4 -f ${LOOP}. In my case, it was because I was on f2fs instead. So the lines of code to do are:

fsck.f2fs -f ${LOOP}  #and go for default options if given any
mount ${LOOP} /mnt
umount ${LOOP}
resize.f2fs ${LOOP}
reboot

C:\Users\happy>ssh root@192.168.1.1
root@192.168.21.1's password:


BusyBox v1.35.0 (2023-01-03 00:24:21 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 22.03.3, r20028-43d71ad93e
 -----------------------------------------------------
root@OpenWrt:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 3.5M      3.5M         0 100% /rom
tmpfs                     3.9G    276.0K      3.9G   0% /tmp
/dev/loop0               10.0G    495.2M      9.5G   5% /overlay
overlayfs:/overlay       10.0G    495.2M      9.5G   5% /
tmpfs                   512.0K         0    512.0K   0% /dev
root@OpenWrt:~#

One of the things I'm still scratching my head about it that doing this process seems to take up about 490M on the loop0 now, whereas before the resize only 61M was used? No idea why.

Hope that helps, and let me know if you have any suggestions for making my instructions clearer.

Ta

Happy

Thanks for the mention. :slight_smile:
90%+ of the info I shared came from other as linked to in my post.

I can also offer some potential help determining space usage of your overlay.

Oh, by the way, /tmp/ is normally using RAM and not any storage like an eMMC or an sd- card.
Look at the command mount to see some info.
In the output, You may see a line like:

...
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
...
tmpfs on /dev type tmpfs (rw,nosuid,noexec,noatime,size=512k,mode=755)`
...

If so, I suspect your device uses RAM for `/tmp/.

This leads to info for your overlay filesystem. There is a line in the mount output for it, Something like:
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
Notice that the writable area (called upperdir) is mounted on /overlay/upper.

Note that the higher used space might simply be filesystem overhead like a much larger directory structure and space reserved for file inodes and block usage etc.

Try using some more advanced options of du (disk usage), and ls to see more info.

Try du --help and ls --help to see a list of options available. Search the Internet or refer the the man pages on a linux system for more details.

Try du -chs /overlay/upper/ and compare output with that of adding 'x':
du -chsx /overlay/upper/
If without 'x' is close to the 490M but with 'x' is much smaller then df -h might be reporting info for a mounted or linked filesystem.

To see more details of space usage, look into sub-directories by adding a wildcard '*' and trailing '/' to indicate to show directories and not files. The 's' option controls some of this too:
du -chsx /overlay/upper/*/
Keep drilling down any path that is using a lot of space.
Command examples: ('s' option omitted)
du -chx /overlay/upper/usr/*
du -chx /overlay/upper/usr/bin/*

You could also show the full listing of files and look for what is using space.
To get an idea of how many files are on the writable area of /overlay/upper/, try using the line count utility:
ls -lARSh /overlay/upper/| wc -l
To see the list, sorted by size in each directory just take off the | wc -l:
ls -lARSh /overlay/upper/
...and scroll... lol.

find is another tool that can do this type of searching as well.

I hope this helps.