Unable to unmount rootfs partition

Hi @UncleDonny, I am facing the same problem when I try to keep setting while upgrade firmware using sysupgrade, but I don't understand how do you solve it.

@AP-0088:(unreachable)/root# ubiattach -p /dev/mtd11
[14181.981324] UBI: attaching mtd11 to ubi0
[14182.366031] UBI: scanning is finished
[14182.384984] gluebi (pid 18312): gluebi_resized: got update notification for unknown UBI device 0 volume 2
[14182.393591] UBI: volume 2 ("rootfs_data") re-sized from 1 to 258 LEBs
[14182.400834] UBI: attached mtd11 (name "rootfs", size 64 MiB) to ubi0
[14182.406365] UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[14182.413108] UBI: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[14182.419789] UBI: VID header offset: 2048 (aligned 2048), data offset: 4096
[14182.426668] UBI: good PEBs: 512, bad PEBs: 0, corrupted PEBs: 0
[14182.432572] UBI: user volume: 3, internal volumes: 1, max. volumes count: 128
[14182.439661] UBI: max/mean erase counter: 42/26, WL threshold: 4096, image sequence number: 348931895
[14182.448787] UBI: available PEBs: 0, total reserved PEBs: 512, PEBs reserved for bad PEB handling: 40
[14182.457929] UBI: background thread "ubi_bgt0d" started, PID 18313
[14182.458829] gluebi (pid 18312): gluebi_create: gluebi MTD device 24 form UBI device 0 volume 1 already exists
UBI device number 0, total 512 LEBs (65011712 bytes, 62.0 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)

@AP-0088:(unreachable)/root# mount -t ubifs ubi0:rootfs_data /tmp/overlay
mount: mounting ubi0:rootfs_data on /tmp/overlay failed: Device or resource busy

No problem when attach /dev/mtd11 as /dev/ubi0, but fail when mount 'rootfs_data' volume from /dev/ubi0 to /tmp/overlay.

I am using IPQ806x board.

Any comments or suggestions would be greatly appreciated, thank you!

Did you solve your mount problem? I am running into the same mount issue on ipq platform during sysupgrade

You have to pivot off the current root. This is handled by the exec call to stage2 in the sysupgrade process.

I am running into the same problem Karhoey has mentioned, sysupgrade fails to restore the backed up config on overlay partition.

1 Like

Do you have serial logs? sysupgrade is generally pretty good about saving /sysupgrade.tar

As Karhoey pointed out, the issue seems like corner case doesn't happen 95% of time. In failure cases the mounting of overlay fails and the backed up sysupgrade.tgz gets never copied over.

@zer0_0ne I'm still puzzled by the log/description presented above as

  • There's no interactive shell involved with sysupgrade (root# ubiattach -p /dev/mtd11)
  • The overlay is typically on ubiN_1, not ubiN

As there has been a recent flood of questions about pivoting root (whether their OP knows it or not), I had though yours was inquiry was around that, as the linked post certainly seems not to be from the sysupgrade process. Forgive me if I mistakenly associated your query with those other threads' content.

NP Jeff. I agree Karhoey, should have created a new thread instead of posting in this thread. sysupgrade code is straighforward, for some reason mount fails on UBI partition leading to problems.

Which device? If dual-firmware, is it only when flashing over OEM firmware, or over OpenWrt as well?

IPQ platform. It is flashing OpenWrt CC with 4.4 kernel which gets shipped as a part of QSDK

Not much we can help with here if running a QSDK image. While loosely based on an old version of OpenWrt, Qualcomm seems to have changed so many things that support, or even diagnosis, of their proprietary OS is not possible here.

Fair point.. The following issue is eerily similar, I am suspecting the issue is with 4.4 kernel and CC than with Qcom changes

There's some "funkiness" when writing to "raw" NAND, rather than using the UBI utilities. I don't know how QSDK handles upgrades, especially on NAND, but that would be something I would look at.

As one example of how current OpenWrt handles it, once the new root (tmpfs) has been prepared and procd has been signaled to enter sysupgrade mode, effectively doing a pivot_root, the exec-ed process for the Linksys EA8300 (IPQ4019) that I am familiar with is target/linux/ipq40xx/base-files/lib/upgrade/linksys.sh. nand_do_upgrade() can be found in package/base-files/files/lib/upgrade/nand.sh. mtd is a utility that is, apparently NAND-aware.

Notice that the issue you linked solved itself when I reinstalled OpenWrt again; I would blame a fault during the first installation, not a specific kernel version.

The problem is in

platform_copy_config() {
	local nand_part="$(find_mtd_part "ubi_rootfs")"
	local emmcblock="$(find_mmc_part "rootfs_data")"

	if [ -e "$nand_part" ]; then
		local mtdname=rootfs
		local mtdpart

		[ -f /proc/boot_info/$mtdname/upgradepartition ] && {
			mtdname=$(cat /proc/boot_info/$mtdname/upgradepartition)
		}

		mtdpart=$(grep "\"${mtdname}\"" /proc/mtd | awk -F: '{print $1}')
		ubiattach -p /dev/${mtdpart}
		mount -t ubifs ubi0:rootfs_data /tmp/overlay <=== mount fails
		cp /tmp/sysupgrade.tgz /tmp/overlay/
		sync
		umount /tmp/overlay
	elif [ -e "$emmcblock" ]; then
		mount -t ext4 "$emmcblock" /tmp/overlay
		cp /tmp/sysupgrade.tgz /tmp/overlay/
		sync
		umount /tmp/overlay
	fi
}

https://source.codeaurora.org/quic/qsdk/oss/system/openwrt/tree/target/linux/ipq/base-files/lib/upgrade/platform.sh?h=NHSS.QSDK.6.1.0.r6

I’d put ubinfo -a in just before the mount there (check the spelling). My guess is that either another UBI is present, the volume doesn’t exist, or it is zero size, or the like.

One "i" in that command

Added the ubinfo -a and when the mount fails and it output is identical to one from a working system```

root@OpenWrt:/# ubinfo -a
UBI version:                    1
Count of UBI devices:           1
UBI control device major/minor: 10:59
Present UBI devices:            ubi0

ubi0
Volumes count:                           3
Logical eraseblock size:                 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks:     624 (79233024 bytes, 75.6 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  40
Current maximum erase counter value:     214
Minimum input/output unit size:          2048 bytes
Character device major/minor:            248:0
Present volumes:                         0, 1, 2

Volume ID:   0 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        31 LEBs (3936256 bytes, 3.8 MiB)
State:       OK
Name:        kernel
Character device major/minor: 248:1
-----------------------------------
Volume ID:   1 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        339 LEBs (43044864 bytes, 41.1 MiB)
State:       OK
Name:        ubi_rootfs
Character device major/minor: 248:2
-----------------------------------
Volume ID:   2 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        210 LEBs (26664960 bytes, 25.4 MiB)
State:       OK
Name:        rootfs_data
Character device major/minor: 248:3

I also encountered this issue, which resulted in the upgrade being unable to save the original configuration. What is the final solution? The link you provided cannot be opened?

We couldn't narrow down the issue as the repro was difficult. Knowing this could potentially happen we had a common partition and we used a backup from that partition in case the copy failed.

The partition cannot be mounted with commond "mount -t ubifs ubi0:rootfs_data /tmp/overlay" after run_ramfs. You copy the sysupgrade.tgz to a common patition, then after reboot, you copy configuration to rootfs_data partition?